Why doesn't the error reporting go into detail (like: val "30" fails predicate: integer?) Is this intentional?
2016:05:25 12:44:17 sveri Hm, is it ok to use ::spec/name for my own specs? Also I have not found the source for ::spec/string, neither by searching github (which is a bit harder, because it ignores "::", nor by looking at the api.
2016:05:25 12:44:36 sveri Also, is it correct that ::spec/name has string? as predicate?
2016:05:25 12:59:24 kingoftheknoll @alexmiller: About a week ago you hinted on Reddit at improved error messages for Clojure. Don’t think anyone’s said that here but it seems like we just got a peek into how that might be accomplished if all of Clojure.core implements specs.
2016:05:25 14:08:08 mjhamrick What's the best way to have fdefs be checked during lein test? Instrument all works at the repl, but seems to not during test.
2016:05:25 14:10:12 anmonteiro @mjhamrick: what I’ve done is (clojure.spec.test/check-var #’fn-speced-with-fdef)
2016:05:25 14:14:49 mjhamrick @anmonteiro: are you just putting that in each test?
2016:05:25 14:15:03 anmonteiro so far, yes
2016:05:25 14:15:10 anmonteiro the project where I tried it was pretty small
2016:05:25 14:24:58 sveri @mjhamrick: I did put (s/instrument-all) at the top of each test namespace
2016:05:25 14:35:44 alexmiller @kingoftheknoll: yes, I have done additional work in this area
2016:05:25 14:44:55 alexmiller @sveri ::spec/name and ::spec/string don't exist - what is that from?
2016:05:25 14:46:37 sveri @alexmiller: Thats why I have not found it in the code. Not sure where exactly it came from, its just that cursive offered it via code completion. Maybe I typed it in a different namespace or whatever. Anyway, good to know.
2016:05:25 14:52:28 michaeldrogalis @richhickey @alexmiller: I have another proposed enhancement to explain-data. Several predicates, such as keyword?, or integer? are programmatically easily recognizable via :pred for non-conforming values. It's a straightforward equality check (e.g. (= 'keyword? (:pred error-map))). Other predicates are, to my understanding, more difficult to recognize. Take s/keys and s/tuple for example. In the former, keys yields a predicate when a required key is missing. While the predicate is somewhat understandable when you eye ball it, picking it out programmatically has been error-prone. The same goes for s/tuple when the number of arguments is incorrect.
2016:05:25 14:53:13 alexmiller an example would help make this specific
2016:05:25 14:53:16 michaeldrogalis Those functions have privileged access to which predicates are failing. It might be helpful to provide named predicates in those cases so that consumers of explain-data could be more intelligent. See - https://gist.github.com/MichaelDrogalis/016ac62cf3f1e899fb89fd79f2de2277
2016:05:25 14:54:02 alexmiller thx for the example! :)
2016:05:25 14:54:49 alexmiller oh the contains stuff that's inside keys
2016:05:25 14:55:25 alexmiller I'll have to defer that to Rich, I think he's not around atm
2016:05:25 14:55:39 moxaj It seems to me that inner conformed values are lost when using coll-of or map-of. Example:
returns [1 2 3] instead of [nil nil nil]. Meanwhile, cat and keys preserves them. How come?
2016:05:25 14:56:14 michaeldrogalis @alexmiller: Sure 🙂 Ill be back later.
2016:05:25 14:57:00 alexmiller @moxaj Rich said above "currently conform doesn't flow into coll-of, so the value is never conformed, only checked" which I think is likely related to this
2016:05:25 15:41:00 sveri I am converting a lib from schema to spec and in schema I was able to inline define the return values / args. Is there a way to do that in spec too? The alternative in this case is a lot more code.
2016:05:25 15:44:08 alexmiller for map keys, no - that's the whole point of the map / attributes split
2016:05:25 15:44:24 alexmiller the benefit is that you are creating named reusable semantics for ::entityname, ::ns, etc etc
2016:05:25 15:44:57 alexmiller so you can use those elsewhere too
2016:05:25 15:45:50 sveri @alexmiller: Yea, I understand the benefits, just wanted to make sure I did not miss anything
2016:05:25 15:48:19 mpenet Is it possible to hook ones own error message if a predicate fails? Ex we have schemas that takes a string that should be a valid parse of our internal query language, and if fails should return a nicely structured error with line/col num and a human readable message. With prismatic Schema it is possible to do with our own schema type extending a protocol for explain/spec.
2016:05:25 15:51:37 mpenet Correct me if I am wrong, but I think so far I can only get a generic "predicate foo? failed" type of message
2016:05:25 15:55:55 alexmiller there is not currently a way to provide a custom error for a failing predicate
2016:05:25 15:56:00 alexmiller you can use explain-data to detect problems and produce a custom error message
2016:05:25 15:56:31 alexmiller but @richhickey can comment on whether that might be something we could do
2016:05:25 15:57:30 mpenet that's what I suspected. Something like ex-info for specs could be useful
2016:05:25 16:05:40 alexmiller why not explain-data for that?
2016:05:25 16:05:55 alexmiller or maybe I misunderstand the suggestion
2016:05:25 16:05:59 mpenet I actually missed it, that could work yes
2016:05:25 16:06:39 alexmiller that's what instrument does on a failing function spec
2016:05:25 16:15:11 mpenet hmm I am not sure that's really equivalent. I would have to wrap all schemas that contains these values with a special validation fn I think. I would like to have my own datastructure returned by explain-data (or something else) when the predicate fails. I am not sure that's doable with the current approach where predicates return just a potentially truthy value.
2016:05:25 16:16:31 mpenet but I could be wrong. I played for 20min with specs so far.
2016:05:25 16:24:07 mpenet this might sound like a small concern, but in a larger context it's very useful: we have a whole api around the manipulation of data containing such queries, being able to use the same schema type and framework and not wrap the whole thing because of a single type in a potentially nested Schema is a nice feature, in our case it also integrates for free with frameworks that just follow this Schema format (rest server, swagger etc).
2016:05:25 17:38:55 eraserhd First piece of feedback is that clojure.spec makes me really want “boolean?"
2016:05:25 17:49:57 sveri @eraserhd: @richhickey said they consider adding this and some more core types (I felt the need for it too :-))
2016:05:25 17:52:23 eraserhd @sveri yay 🙂
2016:05:25 17:52:50 eraserhd I’m still playing with it, but the second thing I’m working through is the repetition, especially for function arguments.
2016:05:25 18:16:19 alexmiller @eraserhd: we have boolean? in almost every example namespace we wrote. so, agreed. :) definitely near the top of the wanted-predicates list.
2016:05:25 18:16:49 eraserhd @alexmiller: cool 🙂
2016:05:25 18:18:51 eraserhd Another interesting problem I encountered: I used a set for boolean: #{true false}. But sets aren’t treated differently in this context, although I might expect that they are. Namely #(#{true false} false) ;=> false. So having a set in a spec which contains nil or false is awkward.
2016:05:25 18:19:17 alexmiller yeah, don't do that :)
2016:05:25 18:19:58 alexmiller it is kind of an interesting wrinkle though that the obvious way to write boolean? is with #(instance? Boolean %)
2016:05:25 18:20:15 eraserhd It might at least deserve a call-out in docs.
2016:05:25 18:21:59 eraserhd Also, a lot of things seem to leak their internal implementations wrt error messages.
2016:05:25 18:23:31 alexmiller but Clojure uses (only) canonical boolean values
2016:05:25 18:23:57 alexmiller @eraserhd: prob better for a mention in the ref doc (which isn't there yet)
2016:05:25 18:24:20 alexmiller @eraserhd: example re errors?
2016:05:25 18:24:55 eraserhd @alexmiller: I encountered a couple. Just a minute and I’ll paste.
2016:05:25 18:55:31 sveri Anyone else seeing problems with reloading test namespaces in the REPL (cursive)?
I just commented out some tests, reloaded the test ns in the REPL, hit run-tests, but still, all tests were run.
I noticed something similar with test-refresh not recognizing commented fdefs.
I cannot point at anything right now, but, there seems to be something broken
2016:05:25 19:01:56 kovasb @cfleming: are there gonna be slick Cursive toolips for providing associated specs when editing functions arguments?
2016:05:25 19:02:32 kovasb I imagine I'm editing the arg to some function that takes a big map, and as I'm filling in the components of the map its telling me what each piece should conform to
2016:05:25 19:09:26 arohner What is the spec for ‘map? i.e. in core.typed, I’d write something like (t/ann map [ [ X -> Y] [X] -> [Y] ). i.e. it takes a collection of X, and a function of X->Y, and returns a collection of Y. Is there a way to do that in spec?
2016:05:25 19:25:15 bbrinck I’m missing something related to recursive specs. I want to define a vaguely hiccup-like structure where the first element is the name, followed by some attributes, followed by a vector of children. What am I doing wrong here? https://gist.github.com/bhb/6cfcb3b38757442aec4ba5db46148699
2016:05:25 19:38:41 hiredman if you add another (s/spec ....) around the ::tag I get a success, but I am not sure why
2016:05:25 19:42:32 hiredman I guess you are in the regex context that matches the outer vector, so you need another s/spec to create a new regex context for the list of child vectors, then you need another s/spec to get out of the regex context of the vector of children and specify an individual child
2016:05:25 19:48:21 anmonteiro I’d also be interested to see the spec for clojure.core/map
2016:05:25 19:48:24 bbrinck @hiredman: Ah, thanks! I had to read that a few times, but now it makes sense 🙂
2016:05:25 19:56:25 sveri Ok, I dont understand how to use instrument. My guess was, that if I put (s/instrument-all) into a namespace all functions are instrumented whenever I load the file into the REPL. But thats not the case, instead I have to execute that function everytime after I loaded a namespace.
2016:05:25 19:56:59 sveri Also, it works when I put it to the end of a ns
2016:05:25 20:02:33 dryewo Hi all, first I want to thank Rich&Co for this awesome innovation :+1:
And I have a question:
(s/def ::even-big-or-small (s/and (s/or :very-big #(> % 1000)
:very-small #(< % 1))
even?))
(s/valid? ::even-big-or-small 10000)
;; throws CompilerException java.lang.IllegalArgumentException: Argument must be an integer: [:very-big 10000]
what am I doing wrong?
2016:05:25 20:07:21 dryewo just noticed, works fine the other way around:
2016:05:25 20:13:39 anmonteiro even? gets [:very-big 1000] so we are interested in the second element
2016:05:25 20:13:51 anmonteiro that’s the explanation, but I’m also unsure if it’s expected
2016:05:25 20:14:47 dryewo yes, I also got this idea of what’s happening, but is it the correct behavior?
2016:05:25 20:15:13 dryewo shouldn’t s/and be order agnostic?
2016:05:25 20:16:07 anmonteiro I’ve read somewhere that it’s not, by design
2016:05:25 20:16:13 anmonteiro but I can’t recall the reason
2016:05:25 20:18:08 dryewo I’m reading the guide: http://clojure.org/guides/spec
maybe I should first read it to the end before asking more questions 🙂
2016:05:25 20:26:40 dryewo thanks, there is even a section about it in the end of the guide, I’ll look into it
2016:05:25 20:35:09 kenny Is there a pattern for whether you spec functions or use spec for validation (pre/post-conditions)? There doesn't seem to be a reason for pre/post-conditions if you spec a function.
2016:05:25 20:39:51 sveri I dont understand that, I have the code in the following snippet, and it fails with:
(remove-autoinc-columns [{:foo "some_t"}])
ExceptionInfo Call to #'de.sveri.clospcrud.s-play/remove-autoinc-columns did not conform to spec:
At: [:args] val: ([{:foo "some_t"}]) fails predicate: (cat :cols :de.sveri.clospcrud.s-play/columns), Extra input
:clojure.spec/args ([{:foo "some_t"}])
clojure.core/ex-info (core.clj:4617)
2016:05:25 21:10:27 anmonteiro @sveri: you need to wrap ::column in s/spec
2016:05:25 22:35:17 anmonteiro simpler than I imagined
2016:05:25 22:35:18 alexmiller I've left off :fn because I'm lazy but you could also verify some additional things
2016:05:25 22:36:04 alexmiller like :f / :transducer or :colls / :seq are the valid combos and even things like whether the :ret cardinality is minimum of the input colls cardinality
2016:05:25 22:37:09 alexmiller but that's enough to detect things like (map inc 100)
2016:05:25 22:37:47 alexmiller which is the kind of thing that normally leads to IllegalArgumentException Don't know how to create ISeq from: java.lang.Long
2016:05:25 22:38:27 alexmiller which, if it's occurring in a nested context can be pretty confusing
2016:05:25 22:40:20 tyler Anyone given the test namespace a try yet? I’m trying to figure out how check-var works. It appears to be hanging half the time I try it and passing tests the other half.
2016:05:25 22:45:18 arohner @alexmiller: is it possible to check that f’s input matches the coll?
2016:05:25 22:45:43 arohner right now I don’t see a way to do that
2016:05:25 22:46:08 arohner because there doesnt’ seem to be a way to ‘compare’ specs
2016:05:25 22:46:24 alexmiller you would have to use a custom predicate to do so
2016:05:25 22:47:17 alexmiller the :args spec could be (s/and <current> #(custom predicate on the args))
2016:05:25 22:48:04 alexmiller I guess you might need to describe :f with an fspec to get the right parts to work with
2016:05:25 23:11:21 hiredman do you have the test.check library on the classpath?
2016:05:25 23:34:22 tyler Yup gen/generate works
2016:05:25 23:46:57 stuarthalloway @tyler: there is a bug there, I hit it too
2016:05:25 23:47:56 stuarthalloway I think it is fixed on master
2016:05:25 23:52:56 tyler Ah awesome thanks
2016:05:25 23:53:35 richhickey yah
2016:05:25 23:57:15 eraserhd @alexmiller An example wrt errors: (spec/explain (spec/tuple number?) []) gives an error about #(clojure.core/= (clojure.core/count %) 2).
2016:05:25 23:57:31 eraserhd s/2/1/
2016:05:25 23:59:52 richhickey @eraserhd: "val: [] fails spec: _ predicate: (clojure.core/= (clojure.core/count %) 1)” says the collection should have count == 1
2016:05:26 00:00:31 richhickey tuple is not a parameterized coll (see coll-of for that), (tuple number?) says vector of exactly one number
2016:05:26 00:01:18 richhickey tuple is for short tuples of heterogeneous types each of which will be listed
2016:05:26 00:01:36 eraserhd @richhickey Oh hi! My comment is about the message, not that it should pass…
2016:05:26 00:01:41 richhickey e.g. (tuple string? number?)
2016:05:26 00:01:52 eraserhd I was saying that it reveals implementation detail of spec/tuple.
2016:05:26 00:02:39 richhickey vs what?
2016:05:26 00:04:09 eraserhd I was reading the rationale document as saying that we want to capture the predicates the user supplied, so I was hoping for something like: val: [] fails spec: (tuple number?).
2016:05:26 00:04:25 richhickey that doesn’t tell you much
2016:05:26 00:04:38 eraserhd Hrmm, this is true.
2016:05:26 00:05:49 richhickey and the interior preds can be arbitrarily complex, will walk down
2016:05:26 00:06:07 richhickey much better than “you failed the top"
2016:05:26 00:07:45 eraserhd Hrmm. So my (clearly not thought through) expectation would be that count would fail at tuple, but sub-types would fail more specifically. That’s not an idea I can make coherent, though.
2016:05:26 00:10:52 eraserhd (Anyhoo, super excited about clojure.spec’s new ideas, cf prismatic)
2016:05:26 00:13:52 richhickey have fun!
2016:05:26 00:38:15 shaunxcode is it possible create a spec for a map like {[300 200 54] :status/open} e.g. map keyed by vector of integers
2016:05:26 00:42:36 tyler @shaunxcode yes I believe so using map-of
2016:05:26 06:41:58 borkdude Should I include test.check as an explicit dependency when using spec?
I got: FileNotFoundException Could not locate clojure/test/check/generators__init.class or clojure/test/check/generators.clj on classpath. clojure.lang.RT.load (RT.java:456)
2016:05:26 06:42:48 borkdude when instrumenting the adder example from the guide https://clojure.org/guides/spec (cc @alexmiller )
2016:05:26 07:15:01 sveri @borkdude: Yes, you need it for most cases if you want to use spec in your tests. There was only one function that works without test.check (I think it was spec/gen, but not sure). I have read it somewhere in the official docs in the last days.
2016:05:26 07:18:56 sveri Anyone else got an idea why this snippet fails?
2016:05:26 07:19:26 sveri @anmonteiro: wrapping it in (s/spec... doesnt fix it unfortunately
which is strange, cause all I do next is to put it into the argument definition of my function
2016:05:26 09:07:26 sveri Can someone reproduce this?
2016:05:26 09:52:14 dominicm Is there space in spec, for a specification to take parameters via lexical scope? This may be out of scope for spec.
For example, on http request, I take a snapshot of my database, and I want to check if a value is in that database. But I want to have my predicates registered via s/def so that I can take advantage of :via and such.
Use case is checking if an email is taken during user registration.
2016:05:26 12:10:30 anmonteiro Library authors might want to add specs to their existing projects but also keep supporting earlier versions of Clojure
2016:05:26 12:10:57 anmonteiro Is this a use case that has been thought about, or isn’t there any interest in supporting such use case?
2016:05:26 12:12:11 anmonteiro or are we just supposed to solve it through versioning?
2016:05:26 12:32:40 alexmiller Specs could be provided in an adjunct library too
2016:05:26 12:34:42 alexmiller Ultimately, things must move forward to get the benefits of new features as we did with records, protocols, transducers, reader conditionals, etc etc
2016:05:26 12:34:57 anmonteiro Agreed wrt. 2nd point
2016:05:26 12:35:27 anmonteiro and yea, the optional library add-on could also be a possibility for initial compatibility, hadn’t thought about that
2016:05:26 12:35:28 anmonteiro thanks
2016:05:26 12:35:47 bronsa @anmonteiro: having spec in both clojure.core + an external library could cause namespace conflicts
2016:05:26 12:36:29 alexmiller They can also be in a separate namespace in your library jar that's just not loaded (unless you're on 1.9)
2016:05:26 12:36:52 anmonteiro @bronsa: I understand, but I didn’t mean that.
2016:05:26 15:07:45 alexmiller @dominicm: you could register (at runtime) an s/def that partialed/closed-over something known at runtime like a database snapshot. This implies that you are using spec to do runtime data validation. Whether or not this is a good idea, I'm not sure. :)
2016:05:26 15:14:41 dominicm @alexmiller: Yeah, I did wonder if dynamic vars might be a solution, and a bad idea.
I'm definitely thinking in the domain of variable user-input, as opposed to contracts during development.
I like the idea of spec as a ubiquitous format that the clojure community uses to describe data. Some data just can't be controlled (user input). Maybe this is way out of scope for spec though, and you guys think that should be a different library altogether. There's definitely a lot of overlap in that space (email still needs to be a string, over 5 chars, match a regex, before it is db checked).
Nothing says a validation library can't register things for you of course. But I try to keep core at the top, not other libraries.
2016:05:26 16:01:56 gtrak The registry itself is pretty simple, I think a 3rd-party lib could sub out the behavior you want but use registered specs along the way.
2016:05:26 16:03:53 dominicm @gtrak: Yeah, that's what I was thinking about. But if it is in scope for spec, that would be pretty cool.
2016:05:26 16:06:01 gtrak my understanding so far is that spec itself is mostly concerned with the lib-sharing/transmission/self-describing-edn use-cases, but there are a lot of developer-convenience type cases we'd also want to use it for (overlap with Schema), but it can't integrate all of them.
2016:05:26 16:15:25 gtrak For instance, the first thing I wrote was a multimethod type->spec coercer, which is a special case extension of the built 'conform', I wouldn't expect that to get into clojure.spec, but I might expect it to be available as a lib, since it's so easy to write.
2016:05:26 16:25:30 gtrak the one thing I'd request from clojure.spec itself was a way to provide a better error message than what s/conformer creates, which in this case is too generic, a failure in the pred: (fn [from] (coerce from to)). It's possible I can do so by dropping down a level in abstraction but haven't gotten that far.
2016:05:26 16:50:28 dominicm @gtrak: oh I agree! I had wondered if that would be possible! Shame you beat me to it.
But I feel like being unable to lexically scope a spec in some way hinders me a little. Maybe I'm looking at it wrong, or maybe I can come up with a clever roundabout way to do it and lib it up.
2016:05:26 16:55:42 gtrak I think I overheard that specs are serializable somewhere, so lexical scope could really mess that up.
2016:05:26 17:00:12 dominicm
I probably shouldn't refer to scope. What I want is a solution to the problem of runtime values effecting the running of my predicates, without relying on state.
Am I making sense?
2016:05:26 18:19:12 gtrak I don't see how it's possible to do that without state, maybe a dynamic/threadlocal registry, but then the state's in the registry.
2016:05:26 18:19:43 gtrak or you create a one-off spec every time
2016:05:26 18:20:49 gtrak or every consumer of the registry has to let you tunnel things into specs
2016:05:26 20:52:58 sveri @alexmiller: I just found a possible bug in the following snippet:
2016:05:26 20:54:08 sveri Running lein test for this code will run into a never ending loop when it tries to test leiningen.s-play-test
2016:05:26 20:54:39 sveri I would expect an error message here
2016:05:26 20:55:21 sveri The cause is this line (s/fdef foo :args (s/cat)) with the incomplete (s/cat) expression.
2016:05:26 21:19:50 alexmiller Feel free to file a jira for it
2016:05:27 19:57:11 alexmiller I still expect there to be a bit more to come on that page about generators and testing
2016:05:27 19:57:22 alexmiller but seemed better to get something out there sooner
2016:05:27 19:58:50 michaeldrogalis @alexmiller Was there any discussion with respect to predicate failures being recognizable from keys, tuple, etc? I can dig up the Gist link if you need for reference
2016:05:27 19:59:26 alexmiller there have been so many things coming in I don't honestly remember if I talked about it with Rich - I have it in my "things to talk about" file
2016:05:27 20:41:37 alexmiller @sveri I have a patch to fix your bug, should go in the next alpha
2016:05:27 21:12:41 sveri @alexmiller: Great, looking forward to testing it
2016:05:27 21:13:00 alexmiller patch is on the ticket if you want to build your own version of clojure
2016:05:27 21:13:16 alexmiller I added a test to the tests for it too
2016:05:27 21:14:48 sveri Yea, I just saw it, I have not setup the clojure build chain, so I just wait for it, no urgency on my side. Thanks for the quick fix
2016:05:28 01:27:56 cfleming @alexmiller: You mentioned you have a spec for spec. Any interest in showing that? It would help with adding support in Cursive.
2016:05:28 01:30:25 alexmiller It's almost certainly pretty buggy so it's not something I really want to publish yet
2016:05:28 01:30:38 cfleming Ok.
2016:05:28 01:31:46 alexmiller I have turned on instrumentation with it and had it actually find bugs while I was working though :)
2016:05:28 01:32:23 cfleming Yeah, I’m planning to merge in the validation code I demoed at the conj, having real-time validation in Cursive for specs would be nice.
2016:05:28 01:32:31 alexmiller I can send it to you privately if you want
2016:05:28 01:33:20 cfleming And using specs to help with completion is probably nice too, although I need to work with it more to see what’s possible.
2016:05:28 01:33:36 cfleming Ok, thanks. I’m not in a big rush, I won’t get to it for a week or two probably.
2016:05:28 01:33:50 cfleming But I’d be interested to see it.
2016:05:28 01:35:25 sekao anyone have an example of using fdef with variadic args?
2016:05:28 02:04:45 sekao ahh i guess it’s something like (s/cat :args (s/* string?))
2016:05:28 11:42:16 alexmiller @sekao: yep although you don't even need the s/cat there just '(s/* string?)' would be sufficient
2016:05:28 11:53:14 francesco how could I write a spec for, say, clojure.core/conj that enforces that in (conj coll x)x satisfies the same spec as the elements of coll? Is this kind of reasoning within the intended scope of clojure.spec?
2016:05:28 11:59:16 alexmiller That's not a true constraint of conj :)
2016:05:28 11:59:36 francesco totally agree on that.
2016:05:28 12:00:14 alexmiller But you could spec something like that with the :fn spec in fdef which relates the args and ret
2016:05:28 12:01:04 alexmiller It's passed a map with the conformed value of both
2016:05:28 12:49:09 sveri A bit more feedback. I constantly run into errors like this:
Most of the times the problem was that I was missing a (s/spec ::columns) expression around the spec definition. Still, everytime I see this error it is so far away from what the problem actually is, that I need time to remember this.
Worse is, that at first I look and search if I did something wrong while transforming my schema to spec.
Maybe a somewhat better error message is possible?
2016:05:28 12:49:14 sveri @alexmiller:
2016:05:28 14:35:18 alexmiller Not sure what a generically better error would be
2016:05:28 16:50:38 alexmiller Interesting - file a jira if you want so we don't lose it over the weekend
2016:05:28 17:26:58 sveri @alexmiller: For me it would be something like: "Maybe you forgot a spec around a regex? Try (s/spec ...)" But, I dont know how generic that error is and if it would fit everytime?
2016:05:28 18:16:27 sekao is there a way to exclude certain vars from being tested by run-all-tests? i have some side-effecting functions that will cause some havoc if run in generative tests 😃
2016:05:28 18:39:28 arohner I’m not entirely clear on when generative testing happens. In some cases (side-effects), I’d like the ability to check conform but not do generative testing
2016:05:28 18:54:18 sekao for now i copied the code into my project and added an if statement that checks for :no-check in the var’s metadata
2016:05:28 19:33:55 bbrinck @arohner: My understanding is that generative testing only happens if you invoke run-all-tests. If you just wanted to test specific functions in your test suite, you could do something like:
2016:05:28 19:38:11 arohner that ran generative testing on a normal call
2016:05:28 19:38:19 arohner which is something I’d like control over
2016:05:28 19:38:54 bbrinck I may be misunderstanding, but I don't think it did. I think it just checked that the spec matched the invocation: (foo #(when (even? %) 42))
2016:05:28 19:39:14 arohner 42 should pass there. The generative example passed zero
2016:05:28 19:40:13 arohner hrm nvm
2016:05:28 19:40:25 bbrinck I think this is just a confusing error message. The value passed is not 0
2016:05:28 19:40:48 bbrinck it's #(when (even? %) 42), which returns nil, which violates the spec
2016:05:28 19:45:11 bbrinck Hm, yeah, you're right, I was reading that wrong. I still don't think (from playing with other examples) that the generative testing occurs unless you do one of the testing calls. I think this example might just have a bug. I could be misunderstanding though.
2016:05:28 19:46:49 bbrinck (I'm trying to try it myself, but on my version of clojure, the fdef is not working for some reason)
2016:05:28 19:47:41 arohner which testing call did I run there?
2016:05:28 19:53:00 bbrinck Hm, you're absolutely right. I did not expect that.
2016:05:28 19:55:07 bbrinck @arohner: Yep, you're correct - there was no testing call, but it's doing some generation for the args for the inner function.
2016:05:28 19:59:17 bbrinck @arohner: interestingly, that does not seem to occur in all cases:
2016:05:28 19:59:38 arohner yeah, I think fspec is the culprit
2016:05:28 20:01:52 bbrinck ah, yes, the docs do mention that now that I read them a little more carefully 😄 . Well, TIL
2016:05:28 20:05:21 bbrinck Also, FWIW, that example won't quite work on 1.9.0-alpha3. fspec seems to require a ret now
2016:05:28 20:08:31 bbrinck nvm, i realized it was a copy/paste fail on my part (didn't grab the full fspec above)
2016:05:28 20:18:39 sveri Hi, did anyone try to generate a boolean spec? Like this: (s/def ::required #(instance? Boolean %)). Calling (gen/sample (s/gen ::required)) fails for me with: Exception Unable to construct gen....
This becomes problematic if ::required is part of a nested spec, how would I provide a generator for that?
2016:05:28 20:28:55 seancorfield That specific case is covered in the (updated) documentation @sveri
2016:05:28 20:30:38 sveri @seancorfield: Ah, thats nice, I just solved it by defining boolean as true / false: (s/def ::boolean #{true false}) for which the generator works too
2016:05:28 20:31:05 seancorfield "There are some common cases that currently don’t have standard predicates, but good generators exist. It’s likely there will be changes in this area in the future but for now you might find these useful:
(defn boolean? [x] (instance? Boolean x))
(s/def ::boolean (s/with-gen boolean? #(gen/boolean)))
(defn uuid? [x] (instance? java.util.UUID x))
(s/def ::uuid (s/with-gen uuid? #(gen/uuid)))"
2016:05:28 20:31:33 seancorfield (That didn't paste well -- sorry, on my phone)
2016:05:28 20:35:32 sveri Yea, no problem, I get it, thank you 🙂
2016:05:28 21:00:27 borkdude Just finished reading the guide. Good stuff.
2016:05:28 21:12:49 borkdude Why is (doc subs) printing Spec on the last line. Bug?
user=> (doc subs)
-------------------------
clojure.core/subs
([s start] [s start end])
Returns the substring of s beginning at start inclusive, and ending
at end (defaults to length of string), exclusive.
Spec
nil
2016:05:28 21:33:51 borkdude What about multi-arity functions like subs, how to write spec for both cases?
2016:05:28 21:41:48 borkdude I'm not sure if I should wrap with s/spec, both seem to work:
(s/fdef clojure.core/subs :args (s/or :two-args (s/spec (s/cat :s string? :start integer?)) :three-args (s/spec (s/cat :s string? :start integer? :end integer?))))
2016:05:29 01:11:22 alexmiller @borkdude: there is new code in doc to find and print specs, but it shouldn't print that if there are none so that's a bug
2016:05:29 01:11:55 alexmiller @borkdude: for multi-arity, you can cover all options in a single spec via regex
2016:05:29 01:14:24 alexmiller you can just do (s/fdef clojure.core/subs :args (s/cat :s string? :start integer? :end (s/? integer?)) :ret string?)
2016:05:29 01:18:42 alexmiller you can even add a :fn #(clojure.string/includes? (-> % :args :s) (:ret %)) to verify it returns a string included in the original
2016:05:29 01:19:14 alexmiller I didn't test any of that but it should be pretty close
2016:05:29 01:31:14 sekao could we get a way to exclude vars from run-all-tests, like ^:no-check in core.typed? i jerry-rigged a solution earlier today but it required modifying the function directly. i have lots of side-effecting functions that shouldnt be running generative in tests.
2016:05:29 01:35:01 alexmiller it's best to file a jira for anything so we don't lose it, esp over the long weekend
2016:05:29 01:36:11 sekao ok will do
2016:05:29 01:36:30 alexmiller thx!
2016:05:29 01:36:46 alexmiller that way we can track that stuff and you can see what's happening with it too
2016:05:29 01:46:08 sekao ah i should’ve looked there first, i’ll just comment on CLJ-1936 as it sounds similar
2016:05:29 21:00:53 sveri Hey, I have again a problem I dont understand:
2016:05:29 21:02:42 sveri When I copy the part from args: ({:name "QpJ50qrS1m24V", :type :varchar, :unique true, :required false, :autoinc true, :null true, :pk true, :max-length 14, :fk-name "Tp9tG8hwUXK0"})and run the spec validation on it it succeeds.
What bothers me, is that the last line in the message wraps the columns in an extra list: :clojure.spec/args (({:name "QpJ50qrS1m24V" ... I am not sure what that means.
2016:05:30 08:29:00 cfleming When s/spec is used to create a new sequential context, I can’t find a way to capture the sequential thing itself.
2016:05:30 08:30:45 cfleming Here, there seems to be no way to capture the [2.0 :teaspoon] vector object if I also want to match its contents.
2016:05:30 11:17:27 alexmiller @cfleming: you are matching its contents via the cat. If you want to receive the vector as the conformed value you could use coll-of to do a different kind of match or use a conformer to transform the matched result into any arbitrary structure
2016:05:30 11:20:46 alexmiller Probably the latter is what you want
2016:05:30 17:33:44 arohner is there a way to update a map spec? I’d like to say “this fn takes a foo map, and returns a foo map with an extra key added on”. It’d be nice to say (s/def bar (conj foo ::extra-key)), without specifying the keys of the second map explicitly
2016:05:30 18:01:01 arohner arg. repl development overwrites instrumentation
2016:05:30 20:00:10 moxaj Question: why is the second argument (`retag`) necessary in a multi-spec expression? In the guide, it is the same as the dispatch function, and I don't see any use case where providing a different function would be beneficial. And if it's the same, naming it again is redundant, as it can be retrieved via the public field dispatchFn.
2016:05:30 20:46:26 seancorfield As an experiment, I’ve added an optional spec namespace to clojure.java.jdbc: https://github.com/clojure/java.jdbc/blob/master/src/main/clojure/clojure/java/jdbc/spec.clj — feedback welcome (the tests attempt to require that ns and instrument clojure.java.jdbc when running tests, so under 1.9.0 the specs are actually checked for all calls in the tests)
2016:05:30 21:12:42 potetm Is there any mechanism that will allow you to include possible exceptions as part of a spec?
2016:05:30 22:26:41 kovasb How can I spec a function that takes an atom(x), where x conforms to a spec?
2016:05:30 22:28:46 kovasb thinking something like atom-of instead of coll-of
2016:05:31 00:00:52 kgzm How would one use spec with records, when records use unqualified keywords and spec mandates the use of namespaced keywords?
2016:05:31 00:36:40 arohner @kovasb: AFAIK, that doesn’t exist currently, but it should be possible to write from cribbing coll-of
2016:05:31 00:46:26 arohner @potetm: ATM, no
2016:05:31 01:39:36 cfleming @alexmiller: Ok thanks, I don’t fully understand the second option but I’ll investigate it.
2016:05:31 01:44:52 potetm I guess the obvious followup (maybe for #C06E3HYPR?) is: Are there plans to support exceptions in specs?
2016:05:31 01:56:38 seancorfield Given that the spec describe data structures, and function inputs/output — what would it mean to "support exceptions" in specs?
2016:05:31 01:57:49 potetm Yeah that's fair.
2016:05:31 01:58:51 potetm My first pass idea was just to "list the possible exceptions". But I'm totally open to the idea that that's just a bad idea.
2016:05:31 01:59:38 potetm I haven't thought it through or anything. Just noticed that I was hitting problems during generative testing because I couldn't declare that, say, arithmetic overflow, was an okay exception.
2016:05:31 02:00:12 cfleming It seems like something you might want to specify.
2016:05:31 02:00:34 cfleming It’s a potential output from a function, more or less.
2016:05:31 02:00:52 seancorfield I can see pros and cons. Certainly in (unit) tests it can be valuable to say "given these inputs, I expect the following exception"...
2016:05:31 02:01:24 potetm cfleming: Yeah that was my thought as well.
2016:05:31 02:01:53 seancorfield But is that a failure to conform to its spec? It depends on whether the spec is considered to be "valid for all inputs that are valid" (and therefore "undefined" for invalid inputs)...
2016:05:31 02:02:36 seancorfield I just went through this exercise with java.jdbc because there are lots of ways to generate exceptions from those functions… but I’m not sure how to specify that...
2016:05:31 02:03:24 seancorfield I’m not even sure if it makes sense to try to specify that? What if the DB is down, or the table you ask for doesn’t exist or there’s a syntax error in your SQL…? How would you codify that in clojure.spec?
2016:05:31 02:03:53 seancorfield How could you even list all possible exceptions?
2016:05:31 02:04:20 potetm Yeah it's possible that that would be overly restrictive.
2016:05:31 02:04:21 seancorfield (each JDBC driver throws its own types — do you just say "could throw Exception"…?)
2016:05:31 02:05:42 seancorfield It’s a hard problem, either way. clojure.spec definitely has gaps when you have stateful functions (generative testing on java.jdbc functions isn’t possible in general — most garbage input will produce an exception, even if it is in the right "form").
2016:05:31 02:05:58 potetm I dunno, perhaps not since it's not exactly stopping you from doing anything at the end of the day.
2016:05:31 02:11:10 cfleming @seancorfield: Yeah, spec is mostly not that useful for me either, for similar reasons.
2016:05:31 02:11:50 mfikes So Sean and Nolen both seem to have created a my.lib.main-ns.spec namespace to hold specs. Wonder if that will become a de facto place to put them for libraries.
2016:05:31 02:12:51 seancorfield I took that approach so that you didn’t have to load the spec ns — so you could use the code with Clojure < 1.9.0
2016:05:31 02:13:09 seancorfield Although the build system shot me in the foot since it compiles all namespaces(!).
2016:05:31 02:13:55 seancorfield Consequently, the build plain ol’ fails on Clojure < 1.9.0. Not sure how to approach that (make the entire namespace conditional? Ugh!).
2016:05:31 02:15:07 mfikes Hmm… need a good solution to that problem. Every lib will face it.
2016:05:31 02:15:39 potetm Spec actually does this for test.check.
2016:05:31 02:15:45 potetm It's non-trivial.
2016:05:31 02:15:50 cfleming I’m just cutting and pasting clojure.spec.* and using Clojure 1.7.
2016:05:31 02:16:09 mfikes Yeah, the dynaload. Hmm
2016:05:31 02:16:16 seancorfield @potetm: But you can’t have an optional namespace that uses spec and get it through the build system.
2016:05:31 02:16:32 seancorfield I already deal with conditional loading in the java.jdbc tests.
2016:05:31 02:17:01 seancorfield Locally, I can run lein test-all and pass tests on every Clojure version from 1.4 to 1.9 — and only on 1.9 does it use the spec.
2016:05:31 02:18:04 seancorfield But the contrib build system tries to compile the namespaces in the project… So maybe there’s some Maven incantation I can use to exclude the namespace from compilation (or whatever Maven is trying to do with it).
2016:05:31 02:19:24 potetm Right so this is all in the build tool. Yeah I got nuthin....
2016:05:31 02:21:02 potetm Well except for this problem, that tool seems pretty slick 🙂 I've never seen anything like it before.
2016:05:31 02:21:23 mfikes Maybe someone can make a hacked-up stub do-nothing compatibility lib in the clojure.spec and cljs.spec namespaces that consumers on 1.8 can use, just so things can be loaded. Ugh.
2016:05:31 02:21:25 seancorfield @potetm: Have you looked at core.typed or Prismatic Schema at all?
2016:05:31 02:22:20 potetm No I mean the version matrix. All the JDKs all the clojure versions. A really nice idea for open tooling. (I've always had control over JDK and clj versions.)
2016:05:31 02:22:32 seancorfield Tomorrow I’ll talk to @alexmiller about the build system and see if we can figure out something ...
2016:05:31 02:51:34 seancorfield Now I feel dirty for having to learn that much about Maven 😞
2016:05:31 02:59:24 seancorfield Apparently if I want to get really "clever" I could probably create a conditional profile based on the Clojure version and have it still run compile if we’re using 1.9.0… Ugh! <profiles> ...
2016:05:31 12:22:58 andrewhr @seancorfield: lein's {:aot :all} doesn't kick in for included dependencies, did it? Maybe that's the reason for mavencompile's strategy
2016:05:31 14:40:27 akiel @anmonteiro You had an example of clojure.test integration were you used spec-is. I can’t find spec-is in the sources. I’m also interested in a proper clojure.test integration of clojure.spec.test/check-var. clojure.spec.test/run-tests only works in the REPL for me. I need it in leiningen.
2016:05:31 14:41:54 anmonteiro @akiel: spec-is was something I wrote:
2016:05:31 17:33:13 seancorfield @andrewhr: The pom.xml file indicates the compile is just a "sanity check" so I’m taking that as "optional"...
2016:05:31 19:05:55 ghadi Is there anything in between s/keys and s/map-of for heterogenous maps --- maps with keywords and structured values as keys?
2016:05:31 20:34:55 seancorfield As I spec’d java.jdbc I found myself wanting a way to constrain the values of the optional keys in the optional opts argument. I suspect the only way is (s/and (s/keys …) #(some custom predicate)) ?
2016:05:31 20:35:27 seancorfield (I’d have to take a look at what s/keys conforms values to)
2016:05:31 20:40:16 kenny Are you guys tending to write your fdefs in a separate ns or in the same ns above/below where the function is written?
2016:05:31 20:41:35 sekao i’ve been putting them above my functions so far
2016:05:31 20:45:44 kenny Yeah that is great for readability but it seems like it kinda pollutes your ns.
2016:05:31 21:53:56 zane How would one spec a map with non-keyword keys and heterogenous values (the specs of which are dependent on the keys)?
2016:05:31 21:55:08 zane That is, how would you spec ::person from the guide if ::first-name::last-name etc were strings?
2016:05:31 22:13:08 zane Or, to put it another way, is there a version of clojure.spec/keys that works with non-keyword keys?
2016:05:31 22:18:40 seancorfield @zane: I suspect you’d need to say (s/map-of string? ::s/any) and then have some custom predicate on the conformed value.
2016:05:31 22:19:00 zane Right, okay.
2016:05:31 22:19:11 zane As I'd feared.
2016:05:31 22:20:31 seancorfield @kenny: I think overall I’d prefer data structure specs in a separate ns and fdef alongside the functions themselves but I won’t have a solid feel for that until we’ve used it a bit more heavily.
2016:05:31 22:20:58 seancorfield In clojure.java.jdbc, I put all the specs in a separate ns so users on Clojure < 1.9.0 could still use the library.
2016:05:31 22:33:26 kenny Makes sense.
2016:05:31 22:58:46 zane @seancorfield: If I wanted to try to recover the the features of clojure.spec/keys but for maps with non-keyword keys would you recommend I implement clojure.spec/Spec myself?
2016:05:31 23:05:36 seancorfield I suspect that will be a lot of work (but I haven’t looked at it). Since the push is for namespaced keywords — but unnamespaced keywords are also supported — I guess I would have to question your desire to define an API based on maps with non-keyword keys?
2016:05:31 23:05:58 seancorfield By which I mean, what specifically is it that you’re trying to spec out here that isn’t a "regular" Clojure map?
2016:05:31 23:06:29 seancorfield (and, perhaps therefore, spec "at large" is not designed for your use case?)
2016:05:31 23:10:39 seancorfield My sense with clojure.spec is that it’s opinionated deliberately to encourage a particular style of API specification… "idiomatically Clojurey"… The comments in particular about namespaced keywords being "tragically underutilized" and that namespace-qualified keywords is "a practice we’d like to see grow"...
2016:06:01 00:03:38 kenny How are you guys spec'ing recursive functions where intermediate return values have a different spec than the final return value? Are you just adding an or to the :ret or are you writing your function in such a way that that isn't possible (e.g. bundling the recursion in the function by creating an anonymous fn inside your function)?
2016:06:01 00:19:29 seancorfield @kenny: Can you give an example of such a scenario?
2016:06:01 00:20:20 seancorfield (I generally try to avoid functions that can return multiple types although I’ll often have functions that return something-or-nil)
2016:06:01 00:20:26 kenny
(defn new-matrix-nd
"Returns a new general matrix of the given shape. Shape can be any sequence of
integer dimension sizes (including 0 dimensions)."
[dims]
(if-let [dims (seq dims)]
(vec (repeat (first dims) (new-matrix-nd (next dims))))
0.0))
2016:06:01 00:20:33 kenny
(s/fdef new-matrix-nd
:args (s/cat :dims (s/nilable coll?))
;; ret is not a matrix because new-matrix-nd is a recursive fn
:ret (s/or :m vector? :n number?))
2016:06:01 00:20:53 kenny Ideally the spec should be
2016:06:01 00:20:56 seancorfield so (new-matrix-nd []) produces 0.0?
2016:06:01 00:21:02 kenny Yes
2016:06:01 00:21:27 kenny But the final return value is always a matrix
2016:06:01 00:21:50 seancorfield Unless you accidentally call it with nil or an empty sequence of dimensions...
2016:06:01 00:22:55 seancorfield So a zero-dimensional "matrix" is a scalar… hmm… that’s an interesting one...
2016:06:01 00:23:10 kenny Right
2016:06:01 00:24:27 kenny I can wrap up the recursion inside the function
2016:06:01 00:24:51 kenny That would probably be cleaner to the outside world
2016:06:01 00:27:50 seancorfield I’m trying to think how specs are going to be much use to you here tho’… since any call site is going to get "vector or scalar" as a result...
2016:06:01 00:28:20 kenny Hmm.. Yeah it might not make sense.
2016:06:01 00:29:58 seancorfield even your :args spec doesn’t buy you much here: nil or an arbitrary collection — you probably want (s/coll-of integer? []) to give you more checking — and I’d be tempted to disallow nil and require an empty sequence of dimensions… unless you have a really good reason for allowing nil there?
2016:06:01 00:31:02 seancorfield Maybe the Clojure/core folks need to give a bit more guidance in the Rationale as to how they expect spec to be used in the real world? /cc @alexmiller
2016:06:01 00:32:01 seancorfield I’d imagined it as a set of specifications for some pretty high-level parts of your code, around your domain objects, or possibly as the definition of a library API… but I’m not sure about the latter yet...
2016:06:01 00:32:12 kenny nil is just the terminating value. It seems wrapping the recursion inside the fn may solve this.
2016:06:01 00:33:07 seancorfield At work we’re looking at spec to write a specification of our domain model and some of the high-level business logic that operates on that. I don’t know how far "down" the call tree we’ll go...
2016:06:01 00:36:53 seancorfield The Rationale says "the supply chain isn’t burdened with correctness proof. Instead we check at the edges and run tests" … "Invariably, people will try to use a specification system to detail implementation decisions, but they do so to their detriment."
2016:06:01 00:38:57 kenny This cleans up the spec...
(defn new-matrix-nd
"Returns a new general matrix of the given shape. Shape can be any sequence of
integer dimension sizes (including 0 dimensions)."
[dims]
(letfn [(new-matrix-nd' [dims]
(if-let [dims (seq dims)]
(vec (repeat (first dims) (new-matrix-nd' (next dims))))
0.0))]
(new-matrix-nd' dims)))
2016:06:01 00:50:19 seancorfield So coll-of is the spec for a (possibly nil) collection — sequence of values — and * is a regex to match zero or more items and returns a vector.
2016:06:01 00:51:08 kenny Was coll-of designed such that a possibly nil collection of values is valid?
2016:06:01 00:52:06 kenny Not sure how that can be an artifact as nil is inherently false so it would seem to be by design?
2016:06:01 00:52:41 seancorfield
boot.user=> (source s/coll-checker)
(defn coll-checker
"returns a predicate function that checks *coll-check-limit* items in a collection with pred"
[pred]
(let [check? #(valid? pred %)]
(fn [coll]
(c/or (nil? coll)
(c/and
(coll? coll)
(every? check? (take *coll-check-limit* coll)))))))
nil
^ this is what coll-of calls and it has a very specific check for nil
2016:06:01 00:53:00 kenny Hmm. Wonder why
2016:06:01 00:53:44 seancorfield Because nil-punning is idiomatic and encouraged?
2016:06:01 00:54:02 kenny I thought the idea was to be explicit about nil with nilable
2016:06:01 00:54:06 seancorfield (that’s a question for Clojure/core really but…)
2016:06:01 00:55:03 kenny Maybe we can get some input tomorrow on this.
2016:06:01 00:57:06 kenny I suppose it does make sense coming from the Java static typing world because you could pass null in place of any argument. We don't need to adhere to that though.
2016:06:01 01:05:32 seancorfield But any function that accepts a (general) collection is almost certainly going to accept nil I’d say…?
2016:06:01 01:06:05 kenny Why make any assumptions?
2016:06:01 01:06:53 seancorfield As noted above/elsewhere, clojure.spec is deliberately opinionated 🙂
2016:06:01 01:07:06 seancorfield (not saying I agree, just observing)
2016:06:01 01:11:55 seancorfield Very different idioms. If null isn’t treated as falsey, then you have a very different style of programming.
2016:06:01 01:13:31 seancorfield For example, in some languages you’d have a Maybe monad and you would chain operations using monadic functions. Scala has Option[T] and flatMap, as I recall. Haskell has Maybe t and fmap (yes?). But in Clojure you’d chain such operations using some-> or some->> and they’d just be regular functions.
2016:06:01 01:14:26 seancorfield Or you use if-let / when-let
2016:06:01 01:15:32 seancorfield I was chatting with someone the other day and saying that I almost never hit a NPE in Clojure these days. About the only time I still do is in Java interop code (usually calling into the String class for something!).
2016:06:01 01:15:46 kenny True (though we can follow that style of programming is Clojure e.g. cats). Anyways, I suppose that is a tangent and doesn't really pertain to this situation.
2016:06:01 01:16:37 kenny Yeah you're totally right. I can't recall the last NPE I got.
2016:06:01 01:17:40 seancorfield (the irony of this conversation today is that someone reported an error against java.jdbc that a particular erroneous call to one of the functions produces a NPE — for invalid arguments)
2016:06:01 01:18:22 seancorfield The latest version of java.jdbc accepts those calls without error tho’...
2016:06:01 01:20:59 kenny Anyways, the primary reason I don't want nil to exist in this particular library is there is no such idea in the math world.
2016:06:01 02:52:40 andrewhr My feelings about clojure.spec are pretty similar to yours @seancorfield... most for library public APIs (think pedestal, with namespaces keywords and data-based DSLs) or for general domain definition. Following the guidelines presented on Clojure Applied, I imagine using specs at domain edges and possibly Records only as a implementation detail of that domain (as suggested in the book).
2016:06:01 02:55:38 andrewhr I still need more though (or some good rule of thumbs) on the usage of namespaced keywords per se as a design tool. When to use a fully qualified - and by definition more coupled to implementation - vs just simple namespaces aka :foo/bar. And when it will be fine to use unqualified ones as envisioned by the Clojure/core style
2016:06:01 02:55:56 seancorfield Records require non-namespaced keys tho’ so I wonder how that sits with specs? (and I don’t think clojure.spec actually supports records?)
2016:06:01 02:57:57 andrewhr Yeah, as far as I got it does not. Considering this style of "Records as an implementation detail", that makes sense no? I just following through the documentation snippet you shared
2016:06:01 03:00:54 cfleming I’m interested to know how spec handles records, too.
2016:06:01 03:01:05 cfleming I guess you treat the fields as non-namespaced keys?
2016:06:01 03:12:30 mikethompson I'm considering spec from an Information Model point of view.
Let's say I wanted to create a spec for car. It might have keys like colour and year. That part is easy. But what about manufacturer where I don't want containment, and instead I want a reference - an id? I could say the spec for manufacturer was integer?, but how should I be more specific and say it is a reference to another spec.
My mind is wandering forward and imagining a tool which can read the central registry and produce a nice SVG Information Model. But it can't do that unless it knows that the manufacturer in car is more than an integer? ... it is actually a reference to another entity.
2016:06:01 03:26:05 seancorfield @mikethompson: Sure you can have ::manufacturer-id that specifies actual valid manufacturer IDs (or whatever you needed) and for testing you could specify a generator that produced (a small subset of) manufacturer IDs.
2016:06:01 03:57:45 zane > I guess I would have to question your desire to define an API based on maps with non-keyword keys?
@seancorfield: Oh, believe me: If we were in control of this data we would use keyword keys. We're trying to enforce / check our assumptions about data we don't control.
2016:06:01 04:08:06 mikethompson @seancorfield: Yep, understand that. I'm just wondering about how that SVG Information Model Diagram tool could work. By looking in the central registry, how does it know that ::manufacturer-id is a reference to ::manufacturer. I can't see how that can be captured.
2016:06:01 04:09:28 mikethompson Almost seems like metadata on ::manufacturer-id
2016:06:01 04:09:51 seancorfield @mikethompson: maybe :manufacturer/id and :manufacturer/spec?
2016:06:01 04:10:12 seancorfield Conventions are needed somewhere, somehow.
2016:06:01 04:10:21 mikethompson Ahh. Metadata via naming convention 🙂
2016:06:01 04:11:05 seancorfield @zane: If the keys are strings when you get them from "the outside" then maybe keywordize them for use "inside"?
2016:06:01 04:11:11 mikethompson Anyway, all good. I was just wondering if I was missing something. Thanks
2016:06:01 04:13:00 seancorfield @mikethompson: No idea what Clojure/core might recommend here. I was trying to follow the Datomic lead there but I haven't looked at how referential specifications might work yet... Definitely something we'll run into at work as we move forward with this...
2016:06:01 04:16:43 seancorfield @zane: If you keywordize them "as-is" (maybe putting a standard namespace qualification on them too?), then you can validate with the full force of spec? At work, we're considering using a naming strategy with java.jdbc to create namespaced keys from the DB rather than just column names, so we can move straight to spec. Not sure how that will work out yet but we're planning a spike along those lines next week.
2016:06:01 05:16:14 josh.freckleton I'm still a bit new and wanting to bring in some typing. I'm not familiar with the respective libs, is spec effectively replacing schema and typed.clojure?
2016:06:01 05:21:21 seancorfield Hi @josh.freckleton -- have you read the Rationale for spec on the http://clojure.org site?
2016:06:01 05:21:40 seancorfield (it tries to clarify the spec vs type system position)
2016:06:01 05:24:48 josh.freckleton @seancorfield: I have, I'm just trying to clear things up a bit, and I guess i'm wondering where I should invest my time 🙂
2016:06:01 05:26:21 seancorfield spec is different to both core.typed and Schema...
2016:06:01 05:29:28 josh.freckleton @seancorfield: oh, can I ask another question while we're talking about this? in schema or spec, could I have a case (switch) according to types, or defmethods on different types? Would there be a prefered method?
2016:06:01 05:29:37 seancorfield core.typed is intended to provide a type checking system based on static analysis of the code with annotations. Schema is a runtime validation system.
2016:06:01 05:30:09 seancorfield Not sure what you're asking... I think I'd have to see an example of what you're trying to do...?
2016:06:01 05:32:28 josh.freckleton (ahh, your typed vs. schema example makes sense. i still need to digest this)
So say I have a few different "types", and I want to map a fn over them which is specific to their type, I've just started to learn about defmethods, and of course there are switch statements (`case`) switching on the type of the object under consideration. If I, say, had a list of things that could be all different types, what would be the best way in to switch on their types?
2016:06:01 05:33:04 seancorfield Sorry, I still don't understand...
2016:06:01 05:33:45 josh.freckleton my fault, I'll be more specific...
2016:06:01 05:39:09 josh.freckleton I'm working with the free monad + interpreter pattern trying to build some code that takes a data model, and can interpret it in various ways. The important part is that the data model, essentially the Free Monad instance, is a recursive "type" where each "node" of this data model can be different types. Currently, I signify their "type" as a key :type in a mapping, and I (case (:type obj... to decide what to do with each "node". For example, one node can be a "User", and another can be "Owned Books", and another can be "Email", "Password", etc. As I interpret this data model, I will want to treat Users differently from Emails, and Passwords, etc.
The 3 options I know of are to 1. defmethod according to different, real types, 2. to case over their type if spec or schema allows me to switch on types, or 3. what I have now with a map, and a "`:type`" key. Maybe I'm completely misguided in how I'm trying to solve this though, and I'm completely open to you annihilating my idea if it's far out-field!
Does that make more sense?
2016:06:01 05:40:41 seancorfield So you want polymorphic dispatch on the :type key in the hash map?
2016:06:01 05:41:37 seancorfield defmulti / defmethod would work for that. Or use a protocol with multiple defrecord implementations.
2016:06:01 05:42:46 seancorfield (the latter would be actual types and would no longer need the :type key)
2016:06:01 05:43:43 seancorfield defmulti would use a dynamic dispatch based on aspects of the data structure, in your case the :type key.
2016:06:01 05:44:05 seancorfield So, basically, defmulti is a sort of case statement dispatch 🙂
2016:06:01 05:47:12 josh.freckleton Ok, I'm glad that way would work, thx. And would their be an idiomatic way to do it without declaring polymorphic methods, like:
(case (spec/type obj) :a "a" :b "b"), or
case (schema/type obj) :a "a" :b "b")
I haven't seen anything in the docs that fits that ability to check the type of an obj, but I maybe I've missed it?
2016:06:01 05:47:48 seancorfield Yeah, I think you're misunderstanding what spec and Schema are about.
2016:06:01 05:48:33 seancorfield Why wouldn't you just do (case (:type obj) :a "a" :b "b") at that point? The :type key gives you what you need.
2016:06:01 05:50:55 josh.freckleton That's probably why I couldn't ask it clearly 😒
I was just curious if that was an option, it seems similarish to me of Haskell's matching on different types. I think that's the "feel" I was going for, but again I could be wrong since I'm new the functional world.
2016:06:01 05:52:42 josh.freckleton for example:
f Nothing = foo
f (Just x) = bar
You've been a big help sean, helping me prune my search tree of what I need to study, haha. Thank you so much, and thanks for staying up late to help us noobs out!
2016:06:01 05:55:43 seancorfield In Clojure, Maybe would more likely just be nil or not-`nil` and you wouldn't pattern match, just (defn f [x] (if x 'bar 'foo))
2016:06:01 05:56:56 josh.freckleton oh sure, and there's also a maybe monad, I'm thinking for matching on many (10-100) custom types.
2016:06:01 05:57:48 seancorfield Right, so if you want polymorphic dispatch on a single argument (type), you probably want a protocol and defrecord.
2016:06:01 05:58:05 seancorfield If you want more ad hoc polymorphism, defmulti is probably your tool
2016:06:01 05:58:45 seancorfield Clojure is very different from things like Haskell, since there's no extant type system, even tho' Clojure is strongly typed (at runtime).
2016:06:01 05:58:53 seancorfield So the idioms are very different.
2016:06:01 05:59:31 seancorfield Clojure's polymorphism is very different from OOP languages as well.
2016:06:01 06:01:26 josh.freckleton K, I had been considering these different options, and with your suggestion I'll zoom in on defrecord/`defmulti`, that helps me cut out tomorrows work 🙂
2016:06:01 06:01:45 josh.freckleton Thank you!!!
2016:06:01 06:05:21 seancorfield Have fun!
2016:06:01 06:24:29 ikitommi @mikethompson: @seancorfield was thinking also for doing registry visualization to learn things. Did a schema->graphviz tools some time ago, which helps a lot. Are you already working on this?
2016:06:01 06:25:49 mikethompson @ikitommi: thought experiment for me. So, no, not working on it.
2016:06:01 06:55:31 seancorfield I'm not a very visual person so it's a "no" on my end too.
2016:06:01 14:23:22 nwjsmith I'm having difficulty spec-ing a map. Would like to re-use spec for the keys and values. In particular, I'm trying to spec this bit of Datomic Pull: map-spec = { ((attr-name | limit-expr) (pattern | recursion-limit))+ }. I've already got specs for ::attr-name, ::limit-expr, ::pattern and ::recursion-limit, but I can't figure out how to re-use them in the spec for ::map-spec
2016:06:01 14:23:50 nwjsmith I thought s/map-of would be the ticket, but it works with predicates not specs.
2016:06:01 14:41:06 manutter51 you want s/keys basically
2016:06:01 14:44:50 nwjsmith AFAICT s/keys is too strict, e.g I can't see how to specify that all keys conform to (or :attr-name ::attr-name :limit-expr ::limit-expr). It will only let me specify particular keys.
2016:06:01 14:45:29 nwjsmith I think what I'm looking for is something half-way between s/keys and s/map-of.
2016:06:01 15:14:12 zane @seancorfield: Those suggestions are great. Thanks! Full support for maps with keys that aren't keywords would still make sense to me, though. There are plenty of domains where what you're modeling with data really does require keys of other types.
2016:06:01 15:51:03 seancorfield @zane: I suspect maps with "other types" of keys are more likely to be homogeneous since you wouldn’t be able to enumerate all the keys (i.e., to list which are required and which are optional).
2016:06:01 15:51:52 seancorfield We have some maps from strings to hash maps, and some from longs to strings etc. But those don’t have specific required / optional keys — they’re "lookup tables".
2016:06:01 15:52:24 seancorfield (it’s an interesting area of discussion tho’)
2016:06:01 15:52:31 zane That makes sense, and matches my experience.
2016:06:01 15:56:56 nwjsmith @zane reading through your conversation w/ @seancorfield it looks like you and I have the same issue. Have you looked into implementing clojure.spec/Spec?
2016:06:01 15:57:12 zane @nwjsmith: Only superficially.
2016:06:01 16:00:12 zane @seancorfield: I'm realizing that one problem with the keywordize-before-running-specs approach is that any resulting error messages will refer to the keyword names rather than the string ones. That limits their utility significantly (makes them less useful to the calling client, for example).
2016:06:01 16:02:23 zane Maybe validating JSON data via spec just isn't an intended use case?
2016:06:01 16:04:55 seancorfield How are you getting the JSON data? When we get JSON from a web service etc, we have clj-http keywordize the result, and we do the same with Cheshire as well. Basically, if we ingest JSON in any manner, we always convert to keywords at the boundary and validate afterward.
2016:06:01 16:05:28 zane (Aside: I'm learning a lot from your responses! Thanks!)
2016:06:01 16:06:45 zane @seancorfield: Right, right. But our intent was to then return any error messages explain-data produces from speccing the request to the calling client.
2016:06:01 16:07:30 zane explain-data's output is going to refer to the keywordized request, which could be confusing to the client if they're, say, unfamiliar with Clojure.
2016:06:01 16:07:58 seancorfield Hmm, I’m not convinced explain-data is a good format for clients since it refers to the internals of your specs … and what about localization of messages, and error codes for lookup in a reference manual for your API?
2016:06:01 16:15:26 zane Hmm. I suppose I didn't notice anything about specs that made me think they were fundamentally internally-facing. I'm curious what your thinking is there.
2016:06:01 16:17:08 zane I had imagined that error codes could be generated via a transformation of the explain-data output. (Localization isn't necessary in this particular case since this is API is only used internally.)
2016:06:01 16:40:12 seancorfield If you’re going to transform the explain-data output, wouldn’t you then want to convert the keywords (including the spec names) to strings anyway?
2016:06:01 16:41:24 seancorfield Taking this example from the Guide:
2016:06:01 16:46:00 seancorfield You’d want to convert :name and :id and whatever was in :via since those are keywords labeling parts of specs and paths through things — so why not convert the actual keys as well?
2016:06:01 16:47:49 seancorfield (having said that, we haven’t gotten as far as trying this sort of thing to generate responses to clients — we currently have custom validation… and that validation checks the types of values passed in as well as the structure so I’m not sure how much we could delegate to spec… but it’s an interesting option)
2016:06:01 16:56:19 zane Yes, you could totally convert the keywords back to strings, but you wouldn't have to if spec had better support for non-keyword keys in the first place. That's my whole point.
2016:06:01 16:56:52 zane But this is totally something that could be added via an extension library, so I guess I should roll up my sleeves. :relaxed:
2016:06:01 17:01:30 seancorfield (BTW, Alex Miller just replied on clojure-dev that he’s on vacation this week which is why he’s not participating here right now — expect more feedback next week!)
2016:06:01 17:39:22 iwankaramazow Is it possible to dynamically generate specs from some sort of schema?
I have a rest api which exposes a schema like this /contacts/schema (json)
From reading the docs, it wasn't clear if I could generate dynamic specs from the above schema.
2016:06:01 18:20:54 seancorfield @iwankaramazow: Given that clojure.specs API is pretty much all macros, my initial reaction would be "No". I would expect you could generate a Clojure source file from your JSON and use that to define the specs, however.
2016:06:01 18:21:32 seancorfield I’m not sure how much of the underlying spec representation is publicly exposed to support dynamic generation of specs...
2016:06:01 18:22:54 iwankaramazow @seancorfield: Thanks for the response! Feared this, I'll experiment a bit
2016:06:01 18:25:56 arohner I’ll take the contrary stance
2016:06:01 18:26:32 arohner it is possible to build a spec from that, but it requires building some new ‘primitives'
2016:06:01 18:27:03 arohner it is possible to build a fn that validates a map containing strings, and it is possible to build a test.check generator for for that schema
2016:06:01 18:27:13 arohner it’ll take a bit of work, but it’s doable
2016:06:01 18:29:57 arohner @seancorfield: spec’s two primitives are predicates and test.check generators. If you can build a predicate for it, you can use it in spec. If you can build a test.check generator, you can use it to generate
2016:06:01 18:30:41 arohner s/keys is big because it generates a bunch of predicates, one for map? and then one for each key/value. there’s nothing special about it
2016:06:01 18:32:40 iwankaramazow @arohner: sounds logical, I'll brew me some algorithm. Thanks for the input
2016:06:01 18:43:44 seancorfield @arohner: What about dynamically (programmatically) registering those freshly built specs? Is enough of the machinery exposed for that?
2016:06:01 18:44:05 arohner yes
2016:06:01 18:44:44 seancorfield (and, yes, I know it’s all feasible since even private functions are accessible and spec is all built in Clojure — but my "No" was meant to indicate whether it is a realistic goal to attempt and I still maintain the answer is negative there)
2016:06:01 18:45:11 arohner Sure, we have the tools to rewrite spec. I didn’t mean to go that far
2016:06:01 18:46:08 arohner My point is just that 1) any predicate is usable in spec 2) s/keys uses no special machinery 3) you can build a predicate to validate maps of strings 4) you can build a test.check generator to generate maps of strings
2016:06:01 18:46:44 arohner s/keys is the way it is because Rich is being opinionated, not that other map predicates shouldn’t exist
2016:06:01 18:57:04 seancorfield Some of the machinery you’d need is private in clojure.spec (e.g., res) and some of the macros that you’d need to replicate as functions are a lot of code. So, yeah, "it’ll take a bit of work" is certainly true.
2016:06:01 19:13:32 arohner ok, you’re right that you’d lose the per-field error messages
2016:06:01 19:16:21 arohner because currently, a predicate would look like (defn struct? [m] (and (map? m) (contains? m “type”) …)
2016:06:01 19:18:06 arohner actually
2016:06:01 19:18:08 arohner one sec
2016:06:01 19:20:09 arohner
(s/conform (s/and map? (fn [m] (contains? m "type"))) {"type" "struct”})
2016:06:01 19:20:50 arohner s/keys just builds up a set of #(contains? % <key>) predicates. So you just (s/and map? <chain-of-key-checking-preds>)
2016:06:01 19:21:43 arohner hrm, so that gets you key checking, but doesn’t conform values yet
2016:06:01 19:24:55 arohner you could use more ands to check values, but ideally we could reuse the pass-specs-to-check-values thing
2016:06:01 19:31:03 seancorfield ☝️:skin-tone-2: the sign of a Clojure developer with too much time on his hands… 😆 It’s easy to get drawn into the "Hmm, that’s an interesting problem!" rabbit-hole!
2016:06:01 21:03:28 mario @kenny: As far as I understand, the former will allow any seq, while the latter will only allow vectors. So depends on what you want.
2016:06:01 21:05:13 kenny The vec at the end is just the value s/conform conforms to.
2016:06:01 21:06:36 mario I see. Then I don’t know 🙂
2016:06:01 21:17:35 jebberjeb @kenny: why are you using the s/cat there? Do you really want to spec that it's a collection with only one other nested collection (of numbers)?
2016:06:01 21:24:17 kenny @jebberjeb: Yes.
2016:06:01 21:33:53 stathissideris is there any way to say that I want 2 optional elements at the end of a cat but they have to be both present or not at all?
2016:06:01 21:34:13 stathissideris a bit like that, but is e2 is there, e3 should be there as well
2016:06:01 21:34:27 jebberjeb @kenny @mario That second argument to s/coll-of is only used when working with the generator for that spec, I believe. This succeeds but probably isn't a good idea: (s/conform (s/? (s/coll-of number? '())) [[1 2 3]])
2016:06:01 21:38:58 stathissideris @jebberjeb: that looks like it could work, let me try!
2016:06:01 21:39:52 stathissideris works, many thanks 🙂
2016:06:01 21:41:53 jebberjeb @stathissideris: yw
2016:06:01 22:31:31 arohner stathissideris: use s/& around the cat to pass in a pred
2016:06:01 22:32:28 stathissideris @arohner: thanks! that's an option too, but I think I'm happy with @jebberjeb 's solution
2016:06:01 22:33:31 arohner ah yes, I missed the ‘at the end’ part, so you can use an optional cat
2016:06:01 22:33:40 arohner in general, & is used to add extra constraints to a seq
2016:06:01 23:03:22 kenny What is the best way to spec multiple arity functions where the args in one overload aren't necessarily related to the args in other overloads. Example:
(defn triangular-matrix
"Returns a triangular matrix created from `coll`. `coll` is a 1D vector where each element will be used to create the
triangular matrix. `upper?` is set to true to create an upper triangular matrix, false for a lower triangular matrix.
`diagonal` is a 1D vector of elements on the diagonal. `off-diagonal` is a 1D vector of upper or lower matrix elements."
([coll upper?]
;;impl
)
([diagonal off-diagonal upper?]
;;impl
))
2016:06:01 23:09:54 kenny Yeah that makes sense to me
2016:06:01 23:11:22 kenny Why is the common pattern to use cat instead of tuple for spec'ing args?
2016:06:01 23:33:08 stathissideris spec looks amazing so far, the only thing missing is coercion
2016:06:01 23:33:23 stathissideris this is the only point where I think schema is a bit more useful
2016:06:01 23:33:37 stathissideris but still, spec is much more expressive
2016:06:01 23:50:25 kenny @arohner: The only thing slightly annoying about that method is needed to specify keys for alt. The keys in this case are pretty arbitrary.
2016:06:01 23:52:46 stathissideris I have a map that looks like this, but if the key is say :x I have a special spec for that value only: (s/map-of (s/or :k keyword? :k string?) ::s/any)
2016:06:01 23:52:56 stathissideris how would I express that?
2016:06:02 00:00:10 stathissideris but as I expected, you lose conform destructuring and explain can't explain the problem anymore
2016:06:02 00:00:36 stathissideris ...so different solutions would be welcome
2016:06:02 00:01:25 george.w.singer Clojurescript is failing to compile my file if my fdefs are located above their corresponding defns. Is this the way things are intended?
2016:06:02 00:01:33 george.w.singer I.E., you can't spec your function before you define it?
2016:06:02 00:02:55 kenny @george.w.singer: Yes. The var is not defined yet. You could declare the var if you must write the spec above the function. I have just been writing my specs below my function definitions.
2016:06:02 00:03:39 george.w.singer Ok, thanks.
2016:06:02 00:05:13 george.w.singer Macro sugar needs to be added to spec IMO
2016:06:02 00:05:15 george.w.singer Something like
2016:06:02 00:57:10 kenny Problem exists in 1.9.0-alpha4.
2016:06:02 02:05:05 seancorfield Looks like primitive hinting encodes the argument and return type to be encoded in the underlying type (if you add ^double as a return type hint, the cast is to IFn$DD, for example).
2016:06:02 02:51:34 seancorfield Very surprised to see that calling an instrumented function can actually load and call clojure.test.check to do generative testing as part of the conformance test, without explicitly calling clojure.spec.test/run-all-tests etc.
2016:06:02 09:25:04 stathissideris is there any way to make s/cat specs generate a vector?
2016:06:02 09:57:50 slipset What @stathissideris said :)
2016:06:02 20:38:00 surreal.analysis This is a pretty minor note, but this appears to be one of the only channels with _ separation instead of -. Any chance an admin could change that?
2016:06:02 20:39:33 arohner and as we all know, clojure names should prefer - to _
2016:06:02 20:40:35 seancorfield I see no stinkin’ underscores here...
2016:06:03 11:16:18 jcf I'm having some trouble spec'ing a fn that walks maps allowing you to hyphenate keys etc. I wonder if someone can point out my mistake as I'm having a hard time decrypting the error message…
(defn walk-map
"Recursively apply a function to all map entries. When map is nil returns an
empty map."
[f m]
(if m
(walk/postwalk (fn [x] (if (map? x) (into {} (map f x)) x)) m)
{}))
(s/fdef walk-map
:args (s/cat :f (s/fspec :args (s/tuple ::s/any ::s/any) :ret ::kv)
:m ::maybe-any-map)
:ret ::any-map)
(defn hyphenate-keys
"Recursively transforms all map keys from underscored strings to hyphenated
keywords."
[m]
(walk-map (fn [[k v]] [(hyphenated-keyword k) v]) m))
(s/fdef hyphenate-keys
:args (s/cat :m ::maybe-any-map)
:ret ::any-map)
2016:06:03 11:16:27 jcf The error:
ERROR in (t-hyphenate-keys) (core.clj:4631)
expected: (= (sut/hyphenate-keys {:a 1, "a" 2}) {:a 2})
actual: clojure.lang.ExceptionInfo: Call to #'example.common/walk-map did not conform to spec:
In: [0] val: ({}) fails at: [:args :f] predicate: (apply fn), nth not supported on this type: PersistentArrayMap
:clojure.spec/args (#object[example.common$hyphenate_keys$fn__20568 0x754a38a0 "
2016:06:03 11:17:08 jcf I've tried a s/cat to capture ::s/any but I'm sure I'm missing something.
2016:06:03 11:18:24 jcf With the s/cat that I'd expect I need:
ERROR in (t-walk-map) (core.clj:4631)
expected: (= (sut/walk-map (fn [[k v]] [(name k) (if (number? v) (inc v) v)]) {:a 1, :b {:c 2, :d {:e 3}}}) {"a" 2, "b" {"c" 3, "d" {"e" 4}}})
actual: clojure.lang.ExceptionInfo: Call to #'example.common/walk-map did not conform to spec:
In: [0] val: ([[] []]) fails at: [:args :f] predicate: (apply fn), clojure.lang.PersistentVector cannot be cast to clojure.lang.Named
:clojure.spec/args (#object[example.common_test$fn__21912$fn__21926 0x7787a602 "
2016:06:03 12:31:54 manutter51 what is your hyphenated-keyword fn doing? I’m not sure I’m following what’s happening there, but it looks like it’s blowing up in that function, and that’s causing the spec failure
2016:06:03 12:36:58 jcf Not sure if ::s/any is already nilable… 🙂
2016:06:03 12:41:21 jcf In the walk-map test I get a cast exception:
ERROR in (t-walk-map) (core.clj:4631)
expected: (= (sut/walk-map (fn [[k v]] [(name k) (if (number? v) (inc v) v)]) {:a 1, :b {:c 2, :d {:e 3}}}) {"a" 2, "b" {"c" 3, "d" {"e" 4}}})
actual: clojure.lang.ExceptionInfo: Call to #'example.common/walk-map did not conform to spec:
In: [0] val: ([[] []]) fails at: [:args :f] predicate: (apply fn), clojure.lang.PersistentVector cannot be cast to clojure.lang.Named
:clojure.spec/args (#object[example.common_test$fn__21912$fn__21926 0x5ea46e79 "
In the tests that rely on walk-map:
ERROR in (t-underscore-keys) (core.clj:4631)
expected: (= (sut/underscore-keys {"a-b" 1}) {"a_b" 1})
actual: clojure.lang.ExceptionInfo: Call to #'example.common/walk-map did not conform to spec:
In: [0] val: ([[] []]) fails at: [:args :f] predicate: (apply fn)
:clojure.spec/args (#object[example.common$underscore_keys$fn__20550 0x1f36bed9 "
2016:06:03 12:42:12 jcf These error messages aren't intuitive. I've got a feeling this will be another thing newcomers to Clojure really struggle with.
ERROR in (t-underscore-keys) (core.clj:4631)
expected: (= (sut/underscore-keys nil) {})
actual: clojure.lang.ExceptionInfo: Call to #'example.common/walk-map did not conform to spec:
In: [0] val: ([[] []]) fails at: [:args :f] predicate: (apply fn)
:clojure.spec/args (#object[example.common$underscore_keys$fn__20550 0x6dd8e64b "
2016:06:03 12:56:31 jcf I wonder if it's the same problem. I need to make sure I have named keys.
2016:06:03 14:01:49 jcf It was. Because I had enabled instrumentation my functions were being called with vectors, sets, etc. and those weren't supported.
2016:06:03 14:50:41 benzap so i'm converting from schema, and i'm having some issues with respect to preventing clutter within my namespace. maybe i'm not structuring my data correctly
2016:06:03 14:52:00 benzap how would I apply s/def to ::bold, ::underline and ::italic, without s/def? I don't want those in the namespace
2016:06:03 14:52:49 benzap if I did have them in the namespace, i'd prefer to have them called something like ::font-style-underline, but the underlying spec would need to accept :underline as the style key
2016:06:03 14:54:39 benzap Is there any way to do this? I'm confused on whether I should be inlining the spec stuff. I'm sticking to what I did with plumatic.schema by placing all of my schemas in one file called schemas.cljs
2016:06:03 15:20:34 jcf @benzap: I don't think you can. You need to def the keywords in order to enable reuse.
2016:06:03 15:21:33 jcf And I'd think you only want really common schema in a global namespace as spec is embracing namespaced keywords.
2016:06:03 15:22:26 jcf So, :font/bold and :font/underline instead of :schemas/font-style-underline etc.
2016:06:03 15:24:23 jcf I've used specs like :common/any-map because I want nilable maps in a lot of places, but there might be a better way I haven't yet found.
2016:06:03 15:25:01 jcf (And I've done away completely with my shared schema namespaces for now.)
2016:06:03 15:29:38 benzap @jcf: I see, so I guess i'll have to start embracing namespaced keywords as well 🙂
2016:06:03 15:30:26 benzap I already have a font.cljs, would it be common to place spec in-line with the code?
2016:06:03 15:33:15 jcf That's what I'm doing, and what I've seen in all the examples.
2016:06:03 15:34:06 jcf It makes sense keeping everything starting with font in font.cljs; that's where I'd look at least!
2016:06:03 19:08:22 nwjsmith Hrm, is there a way to specify a transform to a spec's conformed data?
2016:06:03 19:09:33 nwjsmith I'm speccing datomic query, which accepts either a list or a map, and I'd like them to conform the same way
2016:06:03 19:32:45 nwjsmith Found it! Looks like clojure.spec/conformer is what I was after.
2016:06:03 21:03:24 hiredman I haven't seen this discussed elsewhere, but as a warning, protocol functions called in a non-higher order way (where clojure knows the protocol function is being called an creates an optimized call site for it) don't properly check args and returns against specs if instrumented
2016:06:04 15:48:57 nwjsmith I'd like to ship a library without requiring test.check, and have the generators dynamically loaded as they are with clojure.spec, but I can't figure out how to do that. Do I wrap all calls to clojure.spec.gen functions in clojure.spec.gen/delay?
2016:06:04 15:50:40 nwjsmith (the library is essentially a bunch of specs, some of which require custom generators)
2016:06:04 18:40:51 hiredman that is kind of interesting to think about, if you are defining custom specs with generators in your source, then test.check needs to be a regular dependency not a test scoped dependency
2016:06:04 18:41:20 hiredman a delay delays runtime code from running, not compile time stuff
2016:06:04 18:41:24 hiredman like var resolution
2016:06:04 18:42:48 hiredman you likely need some combination of something like dynaload from clojure.spec.gen and the delay there and maybe clojure.core/delay
2016:06:04 18:43:36 bbloom or just put your generators in another namespace?
2016:06:04 18:45:18 hiredman that would still have issues with any kind of tooling that loads all namespaces (in the absence of test.check)
2016:06:04 18:45:53 bbloom what kind of tooling does that? seems kinda like a bad idea 😛
2016:06:04 18:47:30 hiredman I was thinking of stuff like lein check, but that only loads stuff from the current project
2016:06:04 18:49:29 bbloom on another track…. 1st order predicate expressions and regular languages are closed under intersection — would be cool if somebody wrote an ‘intersect’ function for spec, could be useful for static analysis
2016:06:04 18:52:39 arohner @bbloom: elaborate?
2016:06:04 18:55:20 bbloom @arohner: well you could just write an abstract interpreter that walks the clojure AST and instead of an environment of symbols -> values, have an environment of symbols -> specs
2016:06:04 18:55:29 arohner oh, yeah
2016:06:04 18:55:31 bbloom each time you use something, you call intersect on it
2016:06:04 18:55:37 bbloom if the result is void, you have a "type error"
2016:06:04 18:55:52 arohner yeah, just wasn’t sure what you meant by intersect
2016:06:04 18:55:56 arohner I have about half of that written
2016:06:04 18:56:25 arohner currently reimplementing the regex part of spec to handle it
2016:06:04 18:56:58 bbloom if each spec defines a set, intersect = set intersection
2016:06:04 18:57:37 bbloom intersecting two predicates is just and-ing them, but you could do better for some predciates, cat, repeat, alt, etc
2016:06:04 18:58:58 arohner yeah, it’s a lot more work than that 🙂
2016:06:04 18:59:30 bbloom certainly not trivial, but should be at least straightforward, no?
2016:06:04 18:59:36 arohner yeah
2016:06:04 18:59:49 arohner parsing specs and the regex stuff is straightforward, but a ton of code
2016:06:04 19:00:04 arohner same w/ flowing predicates through a fn body
2016:06:04 19:01:20 bbloom yeah, i have a clojure interpreter that i’ve copy/pasted and modified 4 or 5 times for code walking purposes - someday i’ll make it a library
2016:06:04 19:03:36 arohner I’m already walking the tools.analyzer output, so it’d be a good amount of work to swap
2016:06:04 19:04:12 bbloom not suggesting that you switch to my thing 😛 just showing that it can be much less code than tools analyzer, which does all the analysis all at once
2016:06:04 19:26:58 bbloom anyway @arohner: excited to see what you come up with 🙂
2016:06:04 19:27:24 arohner yeah, me too. Just need to finish the regex matching crap
2016:06:04 21:04:45 gfredericks the first big question about clojure.spec that I'm going to try to dive into is how I can get functionality similar to https://github.com/gfredericks/schema-bijections
2016:06:04 21:05:26 gfredericks so I suspect (ha!) I'll be reading a bunch of the clojure.spec source soon
2016:06:04 21:20:54 gfredericks is it weird that s/tuple takes preds instead of specs?
2016:06:04 22:11:56 benzap I thought preds were specs
2016:06:04 22:12:54 benzap I have a tuple '(s/def ::color (s/tuple ::byterange ::byterange ::byterange ::byterange))' that seems to work @gfredericks
2016:06:04 22:19:00 benzap as of now, I have successfully converted my project from plumatic.schema to cljs.spec!
2016:06:04 22:19:19 benzap still have a lot to learn about it, but it sure is nice
2016:06:04 23:19:09 gfredericks benzap: yeah it does seem to work, but that means the docs are misleading I think: preds are specs but specs aren't preds
2016:06:05 00:08:24 alexmiller the terminology drifted a bit during dev
2016:06:05 02:55:49 adambros Is there anything planned in terms of having predicates/specs with custom error messages/data?
I’m speaking from a validation perspective, because the alternative is to wrap explain-data
2016:06:05 02:56:40 adambros it also seems that for related reasons, I would be writing a conform-or-explain! function that throws with explain-data when the conform fails
2016:06:05 03:02:16 adambros While I’m asking, it would be a useful feature to be able to somehow get the path at which a predicate is being run. This one is coming up as I’m trying to secure a datomic transaction wrapper by looking up if a user has access to a certain :db/id that’s potentially nested within a map.
2016:06:05 03:25:59 seancorfield @adambros: which alpha are you using? I thought they'd added a path result in explain in one of the later alpha builds.
2016:06:05 03:27:58 adambros i’m referring to accessing the path from within the predicate
2016:06:05 03:28:44 adambros might be bad idea (TM) but its something that would ameliorate my problem
2016:06:05 03:30:55 seancorfield But each predicate only knows what it is called on -- it can't know the whole path, since a predicate is a general function.
2016:06:05 03:32:22 seancorfield If your predicate is string? for example...
2016:06:05 03:32:30 adambros it could if there was a dynamic var *path* or something.
I think i could solve this another way so its fine
2016:06:05 03:33:08 seancorfield Dynamic vars are evil 😆
2016:06:05 03:33:15 adambros *evil*!
2016:06:05 03:33:28 seancorfield 🙂
2016:06:05 03:33:49 adambros im more interested in my other questions, so i probably shouldn’t have asked about it
2016:06:05 03:37:59 seancorfield Ok, what are the other questions? I think I missed those...
2016:06:05 03:38:53 adambros custom error message for predicates/specs
something like (s/def :title (s/either string? {:message “invalid title"}))
2016:06:05 03:39:31 adambros where either’s predicate either succeeds or puts the 2nd argument in explain-data’s result
2016:06:05 03:40:19 adambros and the other was about a potential builtin for validation, conform-or-explain! that throws if it cant conform
2016:06:05 03:41:07 seancorfield I get the impression from the discussions here that explain-data is indeed the preferred basis for any custom error messages. I think there is a JIRA ticket for something like conform-or-explain tho'... let me go look...
2016:06:05 03:46:43 seancorfield Hmm, no, maybe it was just a discussion on the clojure-dev mailing list.
2016:06:05 03:47:26 adambros seems like it’d be a simple wrapper, with an extra arity for a function to call with the data when it fails
2016:06:05 03:47:56 adambros (defn conform-or-explain [spec x & [on-fail]] ...)
2016:06:05 03:48:42 adambros by default it’d throw the error in an ex-info
2016:06:05 03:51:30 seancorfield Ah, I remember now, it was a valid or explain wrapper that was discussed -- for use in :pre assertions.
2016:06:05 03:55:04 seancorfield But, yeah, I think the wrapper is so easy to write -- and some folks would want to throw explain-data and some would throw explain as a string so I don't think there's a one size fits all here.
2016:06:05 03:56:19 adambros fair enough, sounds like I just need to have some sort of spec namespace that declares these wrappers
2016:06:05 03:58:05 seancorfield One of the great things about Clojure: it's easy to write an adapter layer to make the supplied libraries match each specific application's needs 🙂
2016:06:05 03:58:51 seancorfield We have a bunch of wrappers at World Singles. We have several of our own string functions in a namespace that supplements clojure.string.
2016:06:05 04:07:15 seancorfield We also have some collection functions, a bunch of date stuff, wrappers around system level stuff. Wrappers around JDBC stuff (and, heck, I maintain clojure.java.jdbc!).
2016:06:05 13:30:35 benzap anyone know of a good way to define a spec for an atom holding a piece of structured data?
2016:06:05 13:31:49 benzap seems like I would need to deref in order to perform the spec, so a lot of wrapping even for simple checks
2016:06:05 14:01:55 nwjsmith @benzap: I think someone asked this a few days ago and the answer was "There isn't because atoms aren't 'data'". IIRC there was also a suggestion to take a look at coll-of, which might have some bits you could crib to implement an atom-of. I might look into this today.
2016:06:05 15:09:41 alexmiller personally, I think you should work with state by
a) defining your problem in terms of data
b) writing nearly every function as a pure function from data to data
c) isolating the use of stateful entities to as small number of functions as possible
d) focusing on writing great specs for b in terms of a
2016:06:05 15:10:42 alexmiller not every function has to have a spec
2016:06:05 18:50:40 gfredericks has anybody started a clojure.spec utility library yet because I keep thinking about doing that
2016:06:05 18:51:00 gfredericks should probably call it spook
2016:06:05 19:00:21 alexmiller you may want to check your urban dictionary before you venture down that path
2016:06:05 19:00:58 gfredericks urban dictionary has all the best words
2016:06:05 19:01:34 gfredericks spock
2016:06:05 19:01:46 alexmiller totally on a tangent, the urban dictionary was hiring for a full stack clojure position recently
2016:06:05 19:02:02 alexmiller spock is much better :)
2016:06:05 19:02:22 alexmiller although I think there is a scala (or groovy?) testing library called that already
2016:06:05 19:10:08 bronsa just realized 'foo' is valid clojure
2016:06:05 19:10:27 gfredericks I only wish the syntax highlighters were more often aware of that
2016:06:05 19:10:40 gfredericks ...because I use that all the time
2016:06:05 19:34:18 mfikes Replete (standalone ClojureScript iOS app) has an update in the App Store with support for cljs.spec.
2016:06:05 20:16:45 gfredericks śpéć
2016:06:05 20:55:42 stathissideris is there any way to make s/cat specs generate a vector?
2016:06:05 20:59:12 gfredericks stathissideris: both s/coll-of and s/tuple can generate vectors -- are either of those usable for you?
2016:06:05 21:00:33 stathissideris @gfredericks: tuple would work for my validation, but ideally I would like to keep the destructuring that s/cat provides
2016:06:05 21:01:13 gfredericks what you want is a way to gen/fmap the default generator
2016:06:05 21:01:35 alexmiller use with-gen to override the generator with fmap
2016:06:05 21:01:48 gfredericks alexmiller: but how do you get the default generator to pass to fmap?
2016:06:05 21:02:39 alexmiller well you could call s/gen on the original spec but I guess you may end up repeating yourself
2016:06:05 21:02:52 gfredericks that was the hack I was about to suggest
2016:06:05 21:03:12 gfredericks we can put a helper for this into shpec
2016:06:05 21:04:15 stathissideris thank you both! I'm not very well-versed with test.check, but I'm saving this part of the log so that I can go do some reading. It will hopefully make more sense after that 🙂
2016:06:05 21:05:50 gfredericks and <definition> being there twice is the repetition alex mentioned
2016:06:05 21:07:15 stathissideris great thanks, I'll give it a go. In my case <definition> is not too long, so it could be ok
2016:06:05 21:08:08 gfredericks could even try factoring it out to a let and see if that works
2016:06:05 21:08:56 stathissideris ...or even a macro is that doesn't work
2016:06:05 21:11:41 gfredericks alexmiller: the default generator for s/and is necessarily pretty hacky; should there be some documentation somewhere suggesting an override?
2016:06:05 21:12:14 gfredericks somehow overriding test.check's normal error message for such-that would be a fancy approach
2016:06:05 21:17:36 alexmiller I think there is some desire to have a better (programmatic) way to have a conversation about filtering with such-that than just boolean yes/no
2016:06:05 21:18:03 alexmiller but I'm sure Rich would have more to say about that
2016:06:05 21:20:04 gfredericks that sounds sooper fancy
2016:06:05 21:21:46 gfredericks alexmiller: do you know if recursive specs is the only reason for the no-arg-function-returning-a-generator pattern?
2016:06:05 21:22:17 alexmiller Stu worked on that but I assume the dynamic loading is another aspect
2016:06:05 21:22:45 gfredericks oh that sounds plausible
2016:06:06 12:08:53 manderson Curious if there has been any thought given to adding a doc string parameter to clojure.spec/def like there is in clojure.core/def? I understand the spec should be self-explanatory, but it's nice to be able to capture additional information/description about the spec. I've found myself adding comments that would be nice to have as doc strings that could then also be parsed programatically.
2016:06:06 12:09:40 alexmiller yes, there has been thought about it
2016:06:06 12:11:35 alexmiller not sure if anyone noticed, but you can now call doc with a registered spec keyword to get it's spec definition. that's a place that docstring could show up.
2016:06:06 12:11:56 alexmiller but it's something Rich is still thinking about afaik
2016:06:06 12:14:02 manderson cool, thanks Alex. I think the built in doc string could be really useful.
2016:06:06 13:53:00 mike_ananev Hello all! I learn clojurescript. Today I discovered that clojurescript doesn't throw Exceptions in code above like Clojure. Can clojure.spec help in such situations? I use reagent and want prevent state updating from using inappropriate function.
2016:06:06 14:02:55 nwjsmith @mike1452: Hey Mike, welcome to ClojureScript! clojure.spec can definitely help you here:
2016:06:06 18:03:58 angusiguess Is there a nice way to spec the following?
2016:06:06 18:04:21 angusiguess "I want a map of arbitrary keywords whose values must conform to some predicate"
2016:06:06 18:05:55 nwjsmith map-of
2016:06:06 18:06:15 angusiguess @nwjsmith: Beauty, thank you 🙂
2016:06:06 18:06:17 nwjsmith np
2016:06:06 22:42:07 wilkerlucio with clojure.spec I'm finding myself wanting to use some kind long namespaces for my keywords, they are just getting to be a borden to write, for example (s/keys :req [:my-app.some-ns.clock/at, :my-app.some-ns.clock/namespace]), the namespace my-app.some-ns.clock doesn't actually exists, it's just helpful to avoid name clashing, I'm writing those in the my-app.some-ns
2016:06:06 22:42:53 wilkerlucio I was thinking that would be useful to be able to alias those namespaces so they can be used in a simpler way, for example:
2016:06:06 22:44:26 wilkerlucio it's possible to alias like this with the currently clojure features?
2016:06:06 22:45:46 noonian I believe if you require a namespace like (ns my-ns (:require [my-app.some-ns.clock :as clock])) you can use ::clock/foo and it will resolve to :my-app.some-ns.clock.
2016:06:06 22:46:11 wilkerlucio @noonian: yes, but that would require me to create a file with that namespace, that's what I would like to avoid
2016:06:06 22:47:07 noonian Ah got it, sorry I didn’t read your message carefully enough.
2016:06:06 22:48:46 wilkerlucio thanks @jr, I had saw that one, and I agree that will help, but only on map cases... when you are setting it as keywords for lists (on the s/keys for example) you can't use that, so we would still have to repeat the typing, I believe aliasing is a more general solution here that could be very useful with long namespaces
2016:06:06 22:48:50 hiredman you don't actually need to create a file now
2016:06:06 23:19:51 wilkerlucio @hiredman: thanks, I'm going to try that soon, do you think that can work on cljs as well?
2016:06:06 23:21:48 hiredman cljs is a different kettle of fish, I am not sure
2016:06:06 23:31:36 wilkerlucio yeah, I have to fix some compilation issues here and then I'll try it out
2016:06:07 02:33:17 alexmiller @wilkerlucio: in addition to the new namespaced support in CLJ-1910 and CLJ-1919 we are considering changes related to aliasing of non-existent namespaces too
2016:06:07 02:34:19 wilkerlucio @alexmiller: cool, that would be nice, specially for cljs where I can't do the trick mentioned by @hiredman
2016:06:07 02:37:52 alexmiller I did some recon on it a few weeks ago and we talked about it briefly again today
2016:06:07 02:39:36 wilkerlucio I'm happy to hear that this is being considered, I think with clojure.spec the usage of fully namespaced keywords is going to have a great increase in usage
2016:06:07 14:27:06 alexmiller I don't think that test is very good, but that may also be true. will be looking at it today
2016:06:07 14:31:40 seancorfield The use case for clojure.spec is different to Schema tho', right?
2016:06:07 14:33:15 seancorfield You might have conform in several places for destructuring and some calls to valid? but you're not going to have everything instrumented in production code.
2016:06:07 14:33:56 seancorfield Whereas folks do have Schema enabled in production don't they? Hence the focus on performance there.
2016:06:07 14:34:11 gfredericks schema aims to be used similarly though
2016:06:07 14:34:35 seancorfield Ah, so not in production code then?
2016:06:07 14:34:41 gfredericks when you decorate functions with schemas they don't run by default
2016:06:07 14:34:53 gfredericks and similar to spec you can turn them all on
2016:06:07 14:34:54 gfredericks for testing
2016:06:07 14:35:04 gfredericks and you can call them explicitly to validate in/out in production
2016:06:07 14:35:27 gfredericks so maybe they focus on performance primarily for that last case?
2016:06:07 14:44:23 alexmiller we still want spec to have good performance, even when not used in production :)
2016:06:07 15:09:31 bronsa @seancorfield: we disable schema in prod, performance hit is way too much
2016:06:07 15:19:53 wilkerlucio @mishadoff: like alexmiller said, you can create a new one, if you still need that key on the current namespace, make up a new namespace, eg: (s/def :other-ns.anything/name ::specific-name) and then (s/keys :req-un [:other-ns.anything/name])
2016:06:07 15:21:18 mishadoff @wilkerlucio: thanks, for now I’ve decided to use same names for keys and specs
2016:06:07 15:22:23 wilkerlucio @mishadoff: cool, that is the preferred way I believe 🙂
2016:06:07 15:23:33 mishadoff @wilkerlucio: yes, but sometimes you have maps with unqualified keys from third-party libs
2016:06:07 15:24:48 wilkerlucio that's true, maybe with the clojure.spec people will start using more fully qualified keywords from now one, and possibly with specs already coming from the library itself, will be awesome 🙂
2016:06:07 15:53:11 gfredericks is :opt in s/keys just for documentation purposes?
2016:06:07 15:53:29 gfredericks I can't figure out how else it affects anything
2016:06:07 16:09:20 angusiguess @gfredericks: does it work with generators?
2016:06:07 17:41:32 alexmiller notably for spec is the new unform which lets you conform ... backwards
2016:06:07 17:42:05 alexmiller also in addition to the vast quantity of new predicates in core, there are now specs for long, double, and instant ranges in spec
2016:06:07 17:42:15 alexmiller and everything gens yay
2016:06:07 17:45:46 gfredericks yay
2016:06:07 17:46:41 seancorfield That’s an awesome new release! Thank you (to everyone involved)!
2016:06:07 17:52:24 angusiguess yessssss
2016:06:07 18:10:06 brabster hey folks, I have a question on map-of in spec that I'd appreciate some help with
2016:06:07 18:10:32 brabster hopefully this is a good forum, even if it turns out to just be me being dumb!
2016:06:07 18:11:13 brabster if I use s/or in the value spec in a map-of, I lose the qualifier I get when I use it on its own
2016:06:07 18:12:07 brabster eg. as per docs (s/def ::thing (s/or :name string? :id integer?)) conforms eg. (s/conform ::thing "bob") to [:name "bob"]
2016:06:07 18:12:33 brabster but... (s/def ::map-of-kw-to-thing (s/map-of keyword? ::thing)) doesn't behave the same was under conform
2016:06:07 18:13:21 brabster (s/conform ::map-of-kw-to-thing {:foo "bob"}) gives {:foo "bob"}, not {:foo [:name "bob"]} as I was expecting
2016:06:07 18:14:26 brabster Am I missing something? I expected ::thing to behave the same way regardless of where it was used
2016:06:07 18:16:07 brabster oh sorry I see a message in the history that I missed from @alexmiller: Rich said above "currently conform doesn't flow into coll-of, so the value is never conformed, only checked" which I think is likely related to this - sorry!
2016:06:07 18:20:31 brabster is there somewhere I can go to see whether this is planned or to ask for it?
2016:06:07 18:22:45 gfredericks queue the warnings from all your favorite libraries that clojure.core/boolean? is being replaced by their local copy
2016:06:07 18:25:06 pheuter yep.
2016:06:07 18:26:07 alexmiller hopefully all benign if the work we did in 1.7 was successful :)
2016:06:07 18:29:42 gfredericks what sort of work?
2016:06:07 18:32:01 alexmiller it's open but it's very narrow in scope iirc - only when defining a new fn of the same name in terms of the old function of the same name, right?
2016:06:07 18:32:34 alexmiller the general case of just overlapping the name was resolved afaik
2016:06:07 18:33:02 gfredericks ah
2016:06:07 18:33:31 alexmiller prior issues came up when we added "update"
2016:06:07 18:33:48 bronsa yeah, I don't actually think that's a bug
2016:06:07 18:33:50 alexmiller we have since added other things that were in general use without issue (although I can't remember an example now)
2016:06:07 18:34:19 bronsa the fact that def declares the var at analysis time is the only way to write recursive functions with defn
2016:06:07 18:34:39 hiredman mmm, I guess I am note sure what version of clojure the guy was running when I w as helping him debug this
2016:06:07 18:34:44 alexmiller @hiredman: I'm not sure if your final example in the comments is actually the same thing or not, not sure?
2016:06:07 18:34:56 bronsa FWIW I couldn't reproduce that example
2016:06:07 18:35:33 gfredericks should it be possible to write a function that verifies that none of the keyword-references used in any specs have typos?
2016:06:07 18:35:56 gfredericks using keywords instead of vars makes the whole thing feel a lot more typo-sensitive
2016:06:07 18:36:27 hiredman yeah, I didn't look at the history to see the current state of the bug, just spent a while trying to figure out why this guys defmulti was resulting in an unbound var, and after way too long I remember that bug, and he said changing the name of his function fixed it, but that is kind of loose, who knows what is happening on the other end of irc
2016:06:07 18:36:29 gfredericks by "typos" I mean a keyword that is supposed to refer to a spec but no spec has been registered
2016:06:07 18:36:52 alexmiller that's not a typo
2016:06:07 18:36:56 alexmiller necessarily
2016:06:07 18:37:09 alexmiller as long as it's registered by the time you use it, it's fine
2016:06:07 18:37:19 gfredericks right, so I mean "never registered"
2016:06:07 18:37:27 alexmiller well never is a long time
2016:06:07 18:37:45 gfredericks :)
2016:06:07 18:37:57 gfredericks I could have a test in a codebase that looks at this for example
2016:06:07 18:38:10 gfredericks so it would have the responsibility to ensure that all pertinent code is loaded already
2016:06:07 18:38:43 pheuter So I’m trying to play around with the new clojure.spec generators, and I keep getting this error: Var clojure.test.check.generators/large-integer is not on the classpath. I definitely have test.check as a dep and i can access various vars under the clojure.test.check.generators namespace, but large-integer isn’t one of them.
2016:06:07 18:39:03 gfredericks pheuter: did you add test.check magically later?
2016:06:07 18:39:11 gfredericks that's when I get that error
2016:06:07 18:39:14 pheuter no, we had it as a dep for a while
2016:06:07 18:39:25 gfredericks pheuter: is it an older version? large-integer is new
2016:06:07 18:39:34 alexmiller are you using test.check 0.9.0 ?
2016:06:07 18:39:39 pheuter yeah, 0.9.0
2016:06:07 18:39:50 alexmiller do you have an older version also on the classpath?
2016:06:07 18:39:52 pheuter let me clear deps and try to re fetch
2016:06:07 18:39:56 alexmiller lein deps :tree
2016:06:07 18:40:43 gfredericks yeah do tree ↑ and look for conflicts
2016:06:07 18:40:54 gfredericks (at the top of the output)
2016:06:07 18:42:52 pheuter good call, will take a look and make some exclusions, thanks!
2016:06:07 18:50:18 alexmiller @brabster: I asked Rich, he said the point of that was that conform samples (does not check every value) for perf reasons and that is unlikely to change. we might make the docs better around it though
2016:06:07 18:53:41 brabster @alexmiller: thanks for the info, would suggest mentioning it with an example in the docs as that's not the behaviour I'd expect from map-of - working around it at the moment but would have been a nice multi-method dispatch off the conformed inputs, looks ugly in comparison now. Will share when pushed up to github, maybe I'm just misusing the functionality!
2016:06:07 18:55:33 alexmiller I making some updates to the guide for alpha5 right now - I'll try to add a note about that
2016:06:07 18:56:20 brabster btw spec is great imho - really neat and well thought through after a few hours of using it
2016:06:07 19:06:41 gfredericks alexmiller: the thing that makes me uneasy is when I have to type a fully qualified keyword referencing a spec
2016:06:07 19:06:55 gfredericks but where it's actually defined elsewhere using ::
2016:06:07 19:07:13 gfredericks or regardless actually
2016:06:07 19:07:37 gfredericks with var namespaces the compiler will catch renamings that were only partially done
2016:06:07 19:07:47 gfredericks with specs it just looks like something isn't defined yet
2016:06:07 19:58:54 pheuter any ideas on how to use generators, possibly custom, to generate Datomic facts? More specifically, I’d like to generate legitimate temp ids, but it’s not clear to me how to express that as a predicate.
2016:06:07 19:59:34 gfredericks temp ids are a custom datomic type aren't they?
2016:06:07 19:59:44 pheuter a thought is to use a set predicate that’s populated with a bunch of tempids
2016:06:07 19:59:44 gfredericks so the predicate would be #(instance? ThatType %)
2016:06:07 20:00:19 gfredericks what's the API to create a tmp id? it's a 0-arg function and it returns a unique object each time?
2016:06:07 20:00:24 pheuter even simpler, they can be verified as longs! the problem is being able to actually transact them against a test db
2016:06:07 20:00:32 pheuter (tempid part)
2016:06:07 20:00:39 gfredericks long?
2016:06:07 20:00:42 gfredericks longs?
2016:06:07 20:00:51 gfredericks that can't be right...
2016:06:07 20:01:04 gfredericks (tempid part) returns a long?
2016:06:07 20:01:31 pheuter well not exactly, it returns a :db/id reader value, but you can pull out an :idx long from it
2016:06:07 20:01:50 pheuter the problem is getting the generate to generate values in sequential order
2016:06:07 20:01:58 gfredericks sequential?
2016:06:07 20:02:01 gfredericks why does that matter?
2016:06:07 20:02:34 pheuter perhaps that part doesn’t matter, that’s just my interpretation of the error I get: db.error/not-a-partition Entity id -1000441 is not in a valid partition
2016:06:07 20:02:42 pheuter where -1000441 is a sample temp id
2016:06:07 20:02:53 gfredericks are you passing a valid partition to tempid when you call it?
2016:06:07 20:03:12 gfredericks oh um
2016:06:07 20:03:19 pheuter so how would i call it?
2016:06:07 20:03:24 gfredericks you just generate a random long and construct the thing that way rather than calling tempid?
2016:06:07 20:04:01 pheuter right, since -1 < n < -100000 is reserved for tempids in the user partition
2016:06:07 20:04:23 gfredericks but -1000441 is out of range, does that mean you're accidentally generating out of range numbers?
2016:06:07 20:04:36 pheuter sorry, -1000000 is the actual bound
2016:06:07 20:04:45 gfredericks still out of range
2016:06:07 20:04:50 pheuter you’re right, let me double check
2016:06:07 20:05:05 pheuter i’ll feel very silly if that’s the case 😕
2016:06:07 20:05:41 gfredericks feeling silly is what programming is all about
2016:06:07 20:08:21 pheuter :db.error/not-a-partition Entity id -3427 is not in a valid partition ¯\(ツ)/¯
2016:06:07 20:10:16 pheuter ah, it may be how i’m transacting that value, since it expects it in a #db/id reader value
2016:06:07 20:10:24 pheuter at least when using the map form
2016:06:07 20:20:36 zane For anyone following along, the approach we're gonna try is using clojure.test.check.generators/fmap in conjunction with datomic.api/tempid and clojure.test.check.generators/choose.
2016:06:07 20:24:58 gfredericks zane: I assume you are also pheuter; now that I think about it I suspect you'll have to be careful about uniqueness?
2016:06:07 20:25:20 pheuter yes
2016:06:07 20:25:26 gfredericks you can use gen/vector-distinct to get a collection of distinct things
2016:06:07 20:25:29 pheuter that was my follow-up :relaxed:
2016:06:07 20:25:33 pheuter oh awesome
2016:06:07 20:26:42 zane Good observation!
2016:06:07 20:27:05 zane Yes, @pheuter and I are colleagues.
2016:06:07 20:35:27 pheuter Hm, looks like gen/vector-distinct is not one of the lazy combinators in clojure.spec.gen
2016:06:07 20:36:14 gfredericks you can use it directly from clojure.test.check.generators/vector-distinct
2016:06:07 20:36:33 gfredericks wrap it in a function if you have to
2016:06:07 20:36:38 pheuter yeah, makes sense
2016:06:07 20:36:41 pheuter thanks
2016:06:08 04:13:27 shaunxcode in the cognicast espisode rich mentions a new syntax for default namespaces for keywords in a map - do we know what that looks like yet?
2016:06:08 12:13:21 moxaj @alexmiller It's a shame that conform does not (and probably won't) flow into map-of and coll-of😞. I have a use case where performance is mostly irrelevant and inner values should be conformed. It seems the functionality is there, but instead i'll have to resort to some error-prone, hairy tree transformation. Would you perhaps consider a dynamic var or an optional argument to map-of and coll-of to enable such transformations (at the expense of performance)?
2016:06:08 12:41:00 alexmiller @moxaj there actually is 'coll-check-limit' now
2016:06:08 12:42:14 alexmiller Defaults to 100 but you could set it higher
2016:06:08 12:42:45 alexmiller I'm not sure whether that changes what flows though
2016:06:08 12:49:28 moxaj @alexmiller It does not, currently. Just to be clear, only the size restriction is settled on, the flowing stuff might be subject to change?
2016:06:08 13:06:11 alexmiller Let's just say I'm not the one making the decision :)
2016:06:08 13:59:16 richhickey a conforming coll spec would not have the same name - either you are exhaustive/conform or neither, not going to be a dynamic magic thing
2016:06:08 14:26:53 dominicm Is there a way to specify in spec, that a key is the focus of my validation, (or where the path of the failure would be), but take other values from the map?
Useful for having two fields that match for example.
2016:06:08 14:32:29 alexmiller you can write a custom predicate that validates anything you like
2016:06:08 14:32:50 alexmiller like #(= (:foo %) (:bar %))
2016:06:08 14:32:56 alexmiller not sure if I'm answering your question though
2016:06:08 14:55:50 wilkerlucio @dominicm: I guess you want something like (s/and (s/keys :req [::password ::password-confirmation]) #(= (::password %) (::password-confirmation %)))
2016:06:08 14:57:28 dominicm @alexmiller @wilkerlucio Do these methods set the paths? So that path would/could be set to [::password-confirmation]
2016:06:08 14:58:06 wilkerlucio @dominicm: sorry, I don't understand what you mean by set the paths, can you please clarify?
2016:06:08 14:58:54 dominicm @wilkerlucio: In the failure, you get a path when you do a key.
2016:06:08 14:59:20 wilkerlucio well, you always have to be composing, so it may depend where the failure is
2016:06:08 14:59:29 wilkerlucio if it's a key with a problem, it will point directly to it
2016:06:08 15:00:17 wilkerlucio if it's on the last checker (the match one) it will point to the map and say that the fail comes from the map and point the predicate out
2016:06:08 15:00:55 wilkerlucio does that answer your question?
2016:06:08 15:02:30 wilkerlucio @dominicm: I think is a good idea to try the (explain) on the REPL so you see if the explanations of the errors met your needs
2016:06:08 15:03:48 dominicm @wilkerlucio: Yeah. It answers my question. I'm trying to figure out how I can that the predicate after failure, actually comes from ::password-confirmation when doing explain.
To result in:
;; => => In: [:foo.core/b] val: "foo!" fails spec...
2016:06:08 15:59:22 wilkerlucio @dominicm: not sure if there is an easy way to do that, you might get some insight by checking the sources for (s/keys)
2016:06:08 16:00:31 dominicm @wilkerlucio: Looks like it works on reification. Fun fun fun.
2016:06:08 16:15:01 brabster is there any way to tell what spec was used to conform a value?
2016:06:08 16:15:41 alexmiller no?
2016:06:08 16:17:21 brabster I was looking for a way to define a function to transform conformed output
2016:06:08 16:18:32 brabster so if I had a way of extracting the keyword that defined the spec I could use a multimethod to do that - probably misthinking it
2016:06:08 16:20:07 brabster sure I can find another way, thought it was worth asking anyway. thanks @alexmiller?
2016:06:08 16:24:24 nwjsmith Although I'm not sure that's the best way to go about it, I just wanted my Datomic queries to conform the same whether in map or list form
2016:06:08 16:25:03 alexmiller I think you're treading on impl details there that can change without warning, fyi
2016:06:08 16:29:12 nwjsmith 😓
2016:06:08 16:37:19 nwjsmith I'm going to try the multi-spec route to see if I can get similar results. Maybe I don't want conformed map and list queries to be the same.
2016:06:08 16:42:57 brabster @nwjsmith: thanks for offering up the suggestion anyway 🙂
2016:06:08 18:17:01 borkdude We're having a meetup about spec and are wondering about the output in this snippet:
2016:06:08 18:28:29 fenton having issues just getting specs going... lein repl complains about spec not being defined
2016:06:08 18:34:21 nwjsmith @fenton have you tried lein clean?
2016:06:08 18:34:48 nwjsmith What is the output of lein deps :tree?
2016:06:08 18:36:11 fenton thank you...i had it over-ridden in ~/.lein/profiles.clj ! doh!
2016:06:08 19:02:29 richhickey @borkdude: what are you wondering?
2016:06:08 19:03:21 richhickey the fact that it reports the error at :baz ?
2016:06:08 19:05:42 richhickey that’s because :cat2 has eaten the ints, then failed its secondary pred. Currently & can’t know that its preds won’t be made happy by some future input, so it’s still working on the & and then can’t eat :baz
2016:06:08 19:09:09 richhickey currently regex ops can communicate that they can succeed w/o more input, but not that they are ‘full’, which is what would be required for & to bail out
2016:06:08 19:22:21 richhickey that particular case could probably be handled though
2016:06:08 19:36:17 dryewo what does (s/cat :a :b) mean? It compiles, but matches only empty lists
2016:06:08 19:36:53 dryewo to me it does not seem a valid spec, it should complain
2016:06:08 19:37:11 gfredericks yeah the :b seems like the potentially invalid part
2016:06:08 19:37:33 gfredericks well though
2016:06:08 19:37:36 dryewo I’m asking because I had smth like (s/cat :url ::url) and I forgot to define ::url above
2016:06:08 19:37:42 gfredericks keywords are predicates
2016:06:08 19:37:48 gfredericks so it's probably being treated that way
2016:06:08 19:38:04 gfredericks but arguably it would be worth not treating them that way since you can't used namespaced keywords as predicates anyhow
2016:06:08 19:38:43 gfredericks though if it matches empty lists then I'm probably mistaken about the treating-it-like-a-predicate part
2016:06:08 19:38:44 dryewo (s/valid? :a {:a 1}) throws an exception
2016:06:08 19:46:08 ikitommi could there be spec-enabled versions of fn and defn?
2016:06:08 19:46:11 borkdude @richhickey: thanks!
2016:06:08 19:47:47 ikitommi writing a function (defn age-plus1 [{:keys [::age]}] (inc age)) already defines something about it’s inputs, the specs could be extracted from the source?
2016:06:08 20:00:11 richhickey @borkdude: next alpha will do this ^
2016:06:08 20:00:15 richhickey thanks for the report
2016:06:08 20:11:27 angusiguess @ikitommi: If I understand correctly, the reason for fdef and explicit instrumentation is to prevent default instrumentation of functions.
2016:06:08 20:39:22 stathissideris @richhickey: In the cognicast you mentioned a bit about the possibility of generating specs because they're data. It's not obvious to me how I would be able to do that, could you point me the right direction?
2016:06:08 20:42:02 richhickey they are data as code is data, you’d generate them as you would any other code
2016:06:08 20:42:19 stathissideris oh that's what you meant! thanks 🙂
2016:06:08 20:43:17 richhickey call s/form on a spec for an example
2016:06:08 20:44:30 stathissideris that's very useful... I'm playing with a (simple) heuristic that would produce specs from example data
2016:06:08 20:44:49 stathissideris hard to get right, but maybe I can come up with something that's good enough
2016:06:08 20:46:01 richhickey specs for spec will help, e.g. the spec for s/cat:
2016:06:08 20:46:54 stathissideris are those somewhere in the code already, or still unreleased?
2016:06:08 20:47:13 richhickey not yet released
2016:06:08 20:47:34 stathissideris but that's the 2-stage generate you mentioned in the cognicast
2016:06:08 20:47:35 richhickey I have to go through them with @alexmiller still
2016:06:08 20:48:06 richhickey right, that’s for testing mostly. Working from example data is interesting, but not possible in the general case
2016:06:08 20:48:20 richhickey testing and parsing I should say
2016:06:08 20:48:43 richhickey although now with unform, the toolchains get interesting
2016:06:08 20:49:27 stathissideris I'm inspired by F#'s type providers... if they can do it to a useful extent, I think the idea has some mileage
2016:06:08 20:50:40 richhickey if not regex-requiring sequences, not so bad. keysets mostly and leaf scalar types
2016:06:08 20:51:24 richhickey still you won’t be able to tell required from optional, etc
2016:06:08 20:51:55 stathissideris yeah figuring out regexes would take some crazy sequence alignment algorithms (as in biology)
2016:06:08 20:52:45 stathissideris well, if a key appears in only half the maps, we can assume it's optional... I'm thinking this could be a cool way to inspect data
2016:06:08 20:55:17 richhickey sounds like fun
2016:06:08 20:56:59 stathissideris we'll see how far I can take it! Imagine applying this to functions while exercising an existing codebase to figure out signatures.
2016:06:08 21:01:20 nwjsmith As I understand from the docs multi-spec is meant for tagged maps, but would be appropriate for specs that apply to either maps or sequences?
2016:06:08 21:02:59 nwjsmith What do I lose by dropping the re-tagging in multi-spec?
2016:06:08 21:04:26 richhickey you can do any discrimination you can with a multimethod - tag keys in maps is just an example. Also note that retag could be polymorphic
2016:06:08 21:05:01 sparkofreason Saw an earlier comment that instrumenting protocol implementations doesn't always result in the spec being validated. Will this be supported at some point? Or does it even make sense, i.e. would it be "better" to spec a map with function specs instead of using protocols?
2016:06:08 21:05:01 richhickey multi-spec is for data-dependent specs, generally
2016:06:08 21:06:42 richhickey spec + protocols tbd. Certainly protocol behind API fn is good practice and supported (spec the API), but many people expose their protocols directly
2016:06:08 21:26:50 dryewo is there a way to disallow non-mentioned keys?
(s/valid? (s/keys :opt-un [::foo ::bar]) {:foo 1 :baz 2})
2016:06:08 21:44:05 nwjsmith An and with a map-of spec and your keys spec will do it I think
2016:06:08 21:45:35 dryewo map-of is for homogenous maps
2016:06:08 21:46:31 nwjsmith Yes, but you can use map-of's kpred to ensure the keys are the ones you accept
2016:06:08 21:47:15 dryewo ah, you mean put a set there?
2016:06:08 21:47:37 nwjsmith yes
2016:06:08 21:48:40 dryewo indeed, it works
2016:06:08 21:48:44 dryewo thanks
2016:06:08 21:49:44 richhickey you can use s/and with s/keys as well
2016:06:08 21:56:25 richhickey map-of samples, will not test every key, so a set not a good predicate
2016:06:08 21:56:45 dryewo but keys will
2016:06:08 21:57:18 seancorfield A use case I can think of is where you might want to limit keys on a map you expose as a data structure outside your code boundary: for example something you convert to JSON to return from a web service or when you insert! into a SQL database (using clojure.java.jdbc for example, where additional columns would lead to an exception). But I’d question whether those are places where you should rely on something like clojure.spec to catch the error (instead of a more explicit approach).
2016:06:08 21:58:04 seancorfield We have select-keys for that sort of scenario, right?
2016:06:08 21:59:43 richhickey yeah
2016:06:08 22:00:59 richhickey @dryewo: keys is open, so will let other keys through, and map-of won’t necessarily catch them, it just samples for type consistency, not particular values
2016:06:08 22:01:26 richhickey you could use #(every? #{:foo :bar} (keys %)) as the second pred
2016:06:08 22:05:26 dryewo I tried this: (s/valid? (s/map-of #{:foo :bar} nil) {:foo 1 :baz 2}) and figured that map-of limits possible keys to those that satisfy the set. But you say it’s something that can change in the future?
2016:06:08 22:06:38 dryewo the solution with every? looks quite ultimate
2016:06:08 22:06:48 dryewo I found *coll-check-limit* in the sources, that explains
2016:06:08 22:07:03 richhickey neither coll-of nor map-of do exhaustive checking, nor conforming. We will make the docs clearer
2016:06:08 22:07:23 richhickey they are ok for very large collections
2016:06:08 22:07:34 dryewo thanks, that makes sense
2016:06:08 22:07:55 ghadi i don't want closed sets of keys without a compelling example
2016:06:08 22:08:21 ghadi haven't seen one yet
2016:06:08 22:08:24 richhickey enough people seem to want exhaustive conforming version so we’ll likely have both
2016:06:08 22:08:50 richhickey @ghadi: well, that’s why open is how it is, I agree 🙂
2016:06:08 22:08:55 gfredericks the only thing that comes to mind for me is the principle of being conservative about the data you return from an API
2016:06:08 22:09:20 richhickey you just engender brittle code on the other end
2016:06:08 22:09:39 gfredericks oh if the closedness is publicized then yeah
2016:06:08 22:09:57 gfredericks I was just imagining using it for checking that you aren't accidentally leaking extra stuff
2016:06:08 22:10:08 richhickey well as @seancorfield said, you can call select-keys
2016:06:08 22:10:14 gfredericks but I suppose specs are intended to be publicized
2016:06:08 22:10:17 richhickey right
2016:06:08 22:10:41 gfredericks cool, I'm on board
2016:06:08 22:11:27 richhickey there’s a bit in the podcast interview about change, while maybe not made clear yet, this kind of thing is an important part of the design
2016:06:08 22:12:01 gfredericks because it means adding new keys is backwards compatible?
2016:06:08 22:13:02 richhickey there are separate semantics for accept/return, but they align with co-contravariance, and the keyset and regex system will let us say when things accept/return compatible things
2016:06:08 22:13:52 gfredericks how will predicate equality work in that case? check that the forms are the same?
2016:06:08 22:14:05 richhickey i.e. :req less on what you accept is compatible, :req (essentially guaranteeing) more on return compatible, etc
2016:06:08 22:15:03 richhickey regexes can be tested to see that they new accepts everything old does (accept) and old accepts everything new does (return)
2016:06:08 22:15:47 richhickey base predicates presumed immutable, yes forms
2016:06:08 22:16:40 richhickey what I hope people find is that at the fine granularity of spec, things rarely change, and if the do they must compatibly
2016:06:08 22:18:23 richhickey and it could get fancier (generative testing to do the non-spec preds)
2016:06:08 22:18:24 gfredericks richhickey: I asked on the ML about tactics for moving libraries forward; in the podcast you mentioned something about adding numbers at the end of function names, do you imagine that being a sort of last resort?
2016:06:08 22:18:46 gfredericks stu also seemed to mention creating entirely new namespaces, so I'm curious how common you imagine each of those tactics being
2016:06:08 22:20:19 richhickey how often do you change the sigs of public fns? I think for algorithmic things rarely and for informational things only in spec-compatible ways. It’s the breaking change that;s the last resort, the numbering of fn names is just being honest with your users
2016:06:08 22:22:51 richhickey The problem with libraries is that a lib has 50 fns and when you add 2 new ones it gets a different ‘version’ and no one knows what’s different. What they actually depend upon are the individual fns that they call, not the lib. Stable specs will show the true granularity of change (usually additive)
2016:06:08 22:24:27 richhickey I’d much rather deal with foo-2 (which btw can co-exist with old foo) than ‘version 2’ of foo.
2016:06:08 22:24:42 richhickey I can move to foo-2 when I please
2016:06:08 22:24:46 gfredericks yeah, I definitely appreciate that
2016:06:08 22:25:29 gfredericks there are a handful of minor warts in the test.check API that feel hard to improve without breaking
2016:06:08 22:25:39 gfredericks for some value of "breaking"
2016:06:08 22:26:16 gfredericks at least if "the API is surprising and hard to sort through" is part of the problem, as then adding a foo-2 might maintain those downsides
2016:06:08 22:26:45 gfredericks so I suppose I'm having to accept not being able to "fix" things in that way
2016:06:08 22:27:36 richhickey for big changes you can also do test.check2
2016:06:08 22:27:49 gfredericks as the lib name, or namespace name?
2016:06:08 22:28:11 richhickey lib/ns same difference
2016:06:08 22:28:41 richhickey then people can move when they please, specs will be in different nses, so they haven’t changed
2016:06:08 22:31:08 richhickey @gfredericks: I would like to be able to convey explanation data through such-that somehow
2016:06:08 22:31:57 gfredericks richhickey: alex mentioned something about having a fancier such-that to make the generator for s/and more robust, do you know if that's a different issue from what you just mentioned?
2016:06:08 22:32:01 gfredericks (just so I can keep track)
2016:06:08 22:32:06 richhickey maybe a (compatible!) extra arg which is a fn to call on failing val to get exception text)
2016:06:08 22:33:00 richhickey not sure if that’s what @alexmiller was talking about
2016:06:08 22:33:17 gfredericks what he said exactly was: "I think there is some desire to have a better (programmatic) way to have a conversation about filtering with such-that than just boolean yes/no"
2016:06:08 22:34:09 richhickey yes, so right now the pred fails and test.check can’t say anything useful because the pred is boolean
2016:06:08 22:34:27 ghadi re: open/closed keysets -- I haven't had the need to ensure a closed set, but when working with an API not under my control, it is useful to detect added information
2016:06:08 22:34:37 richhickey but spec knows what the conditions were
2016:06:08 22:35:13 gfredericks richhickey: okay so he was talking about the same thing? I interpreted his statement to imply some super-fancy way of giving such-that hints about other things to try
2016:06:08 22:35:14 richhickey @ghadi: yes, that’s different, and has been requested - (keys-not-in spec map)
2016:06:08 22:36:21 richhickey @gfredericks: nope, just better failure reporting, because the failure to gen could be in a nested spec we’d like to at least tell the user which one
2016:06:08 22:44:26 ghadi could something along the lines of keys-not-in work at any path? use case is detecting a fully compatible change made on an external API
2016:06:08 22:46:34 gfredericks richhickey: after thinking about it a bit, I'm imagining a function that's expected to return a throwable, that way you can customize the error text and the ex-data
2016:06:08 22:47:53 gfredericks I guess that means the stack trace starts in a weird place though, which I was trying to avoid :/
2016:06:08 22:54:26 kendall.buchanan Big thanks to everyone who contributed to clojure.spec. Spent two days building an internal library on it, and it’s, so, so, so, so, so, so… enjoyable.
2016:06:08 23:46:16 bbloom @ghadi: another use case i just ran into in a (non-clojure) project: warning on configuration errors
2016:06:08 23:47:18 bbloom had a config.toml file and somebody had a rogue [Section] and a Key = true line was getting put in to the wrong section, so the config setting wasn’t taking hold
2016:06:08 23:47:32 bbloom i used a feature of our toml parser to print all the unrecognized keys to the logs
2016:06:09 02:53:04 arohner @bbloom: @ghadi yes, figwheel uses that idea to great effect in config file validation
2016:06:09 02:53:53 arohner “unrecognized key :foo under :bar, you probably want it in :baz
2016:06:09 07:02:13 olivergeorge Hello. My CLJS apps use a lot of string keys. Primarily for performance reasons, transforming JSON has a cost. I went looking for the equivalent of the :strs feature of map destructuring, perhaps something like :req-strs and :opt-strs could be added to s/keys.
2016:06:09 07:02:27 olivergeorge For now the workaround seems to be using a conformer...
2016:06:09 07:02:36 olivergeorge
(def strs->keys
"Turn map with string keys into map of keyword keys. Without this we can't use s/keys."
(s/conformer #(reduce-kv (fn [m k v] (assoc m (keyword k) v)) {} %)))
(s/conform
(s/and strs->keys
(s/keys :req-un [::id ::options]))
{"id" :id
"options" [{:id 1} {}]})
2016:06:09 07:03:18 olivergeorge Am I missing anything?
2016:06:09 07:35:21 hiredman it depends on what you are looking to get out of it, I guess
2016:06:09 12:56:49 dominicm @hiredman: You lose thikngs like path lookups doing that. I think that's the key feature that goes missing. Spec has a bit of a balancing act to perform making sure that via/paths are added.
s/keys is a bit complicated in implementation, but I would be thinking that a similar-but-different implementation should surface for string keywords.
2016:06:09 13:35:16 gfredericks I figured it could be handled with a bijection system that doesn't produce a spec on the other side, but that doesn't help if you're avoiding transformations for performance
2016:06:09 14:42:00 rickmoynihan Hey... I've been very pleased with clojure.spec, which looks absolutely fantastic, and a nice step up from plumatic.schema - and I am fully intending to play with it properly (I've only really had time to read the announcement, listen to Rich on the cognicast, and looked at a few examples)...
Anyway, one thing I'd really like to do is understand how this fits with JSON schema... Now, clojure.spec is clearly far superior to JSON schema, but it's not used by the javascript community etc... So I was wondering if anyone has thought about how one might be able to build a JSON schema from a clojure.spec. As spec is way more expressive, I'm guessing that to do this you'd have to stick to a subset of spec (and I guess you could probably use clojure.spec to specify that subset)...
The basic idea would be to write a super-spec in spec; and output JSON schemas to interoperate with other tooling/libraries in other languages etc... e.g. swagger clients etc.
If this is possible from within clojure you'd then be able reuse that subset of spec (which generated JSON schemas), and layer ontop the additional expressivity and specs you'd want clojure side.
2016:06:09 15:38:11 gfredericks rickmoynihan: the library I linked to just above has a lot to do with that problem, but it works with plumatic/schema. I've been pondering how best to translate it to clojure.spec
2016:06:09 15:38:59 gfredericks the idea is it uses a set of transformations to turn an internal schema (the nice one) into a different schema (the coarser json-style schema in this case), with functions generated to transform data structures between the two
2016:06:09 15:45:36 gfredericks it's still a bit half-baked, but I used it a bunch on a previous project
2016:06:09 15:46:51 gfredericks in particular it doesn't do a good job of distinguishing between bijections and non-bijections (coercions, projections, whatever you might call them); I'm thinking ou could do that interestingly using test.check generators
2016:06:09 15:49:06 gfredericks e.g., if my JSON API can accept {"unitCount":42} as well as {"unitCount":42.0}, you might specify that in the schema, but say that the first one is preferred, so that the transformation function always uses that, but could also have a generator that would take {"unitCount" 42} and generate sometimes {"unitCount" 42} and other times {"unitCount" 42.0M}
2016:06:09 15:49:23 gfredericks so it would give you a way to test flexible APIs in a robust way
2016:06:09 15:51:03 gfredericks unrelated: if I'm reading the clojure.spec source correctly, it ought to be safe to add specs to test.check (and to clojure.core) without the danger of an infinite loop during instrumentation -- does that sound right?
2016:06:09 15:51:17 rickmoynihan does spec support coercion? I though rich said it wasn't part of its job
2016:06:09 15:51:33 gfredericks specs can refer to themselves via the keyword
2016:06:09 15:53:14 rickmoynihan yeah but I didn't think spec could coerce a value from "foo" into :foo - or can it?
2016:06:09 15:53:28 gfredericks oh geez sorry
2016:06:09 15:53:33 gfredericks I misread "coercion" as "recursion"
2016:06:09 15:53:39 rickmoynihan lol 🙂
2016:06:09 15:53:53 gfredericks (and thus thought your question was unrelated to what I was previously talking about)
2016:06:09 15:54:07 gfredericks okay right afaik it doesn't support coercion
2016:06:09 15:54:30 gfredericks I'm not sure if the "external spec" idea would work or not
2016:06:09 15:54:48 gfredericks e.g., with json you'd want to have string keys, which isn't natural
2016:06:09 15:54:50 rickmoynihan tbh I'm not too worried about the coercion case...
I'd just like to write clojure.specs (even in a limited subset of spec) and output JSON schemas from them
2016:06:09 15:55:10 gfredericks I think that by itself would probably be easier, if you only support a subset of schemas
2016:06:09 15:55:34 rickmoynihan well I'm sure that would be possible too
2016:06:09 15:56:43 rickmoynihan it doesn't need to be perfect - I'd just like to define a service in clojure, and emit a JSON schema that's good enough to catch errors between service boundaries - where client services might be implemented in javascript/ruby etc... And ideally also target swagger, to give client developers access to some swagger tooling / swagger-docs on the other side. (Though that's just an extra nice to have)
2016:06:09 16:06:39 gfredericks I'm trying to write (for fun) a variant of defn where each arg can have a spec and you can have multiple bodies with the same arity as long as the specs are different
2016:06:09 16:06:55 gfredericks i.e., you can overload on spec, not just arity
2016:06:09 16:10:10 rickmoynihan interesting idea
2016:06:09 16:10:39 rickmoynihan I really like multispec
2016:06:09 16:11:09 rickmoynihan I'm guessing you just generate those when the arities are the same?
2016:06:09 16:11:36 gfredericks I was going to start with just one body that takes varargs and parses them
2016:06:09 16:15:00 gfredericks incidentally I have to parse arglists for this and spec is really good at that :D
2016:06:09 16:15:54 ikitommi @rickmoynihan: there is a open issue in ring-swagger to add support for spec, haven't had to to investigate yet. (Currently supports the Plumatic Schema). Could split the lib into separate submodules for the different sources models.
2016:06:09 16:17:25 rickmoynihan ikitommi: does ring-swagger include use a separate library for JSON schema; or bake its own one?
I'm no swagger expert - but I'd assumed that Swagger included all of JSON schema...
2016:06:09 16:18:04 rickmoynihan I just think it'd be most useful to target JSON schema directly rather than drag in swagger/ring etc too
2016:06:09 16:23:41 rickmoynihan We're in the fortunate situation of not having a lot of legacy plumatic/schemas - mainly because we only started using schemas a month or two before spec's announcement... so I'm not too concerned about plumatic support for legacy reasons...
But I'm curious whether anyone think's there's a future for plumatic/schema beyond just supporting legacy... i.e. does it have uses that clojure.spec can't support so well?
2016:06:09 16:25:23 gfredericks there's nothing I used it for that I would keep using it or
2016:06:09 16:25:25 gfredericks s/or/for/
2016:06:09 16:26:03 gfredericks even if there's something minor it could do better I feel like there's more leverage to be had using The One True Tool
2016:06:09 16:26:57 gfredericks I'll make a clojure.spec utility library if I have to though, for specialized stuff that clojure.spec decides not to support directly
2016:06:09 16:27:20 gfredericks e.g., maybe for the super succinct map syntax plumatic/schema has for defining map schemas
2016:06:09 16:27:40 rickmoynihan the only things I think plumatic/schema has over clojure.spec right now are:
1. it's perhaps a little closer to json schema - and possibly less work to bridge into json schema
2. no automated coercions
2016:06:09 16:28:37 wilkerlucio I'm wondering here about sort of generic specs, I'm thinking on the channel case, I can spec the return of a function to return a channel, maybe would be nice to be able to annotate also the expected value that will come from the channel, do you people have any thoughts on how to deal with stuff like this?
2016:06:09 16:45:18 richhickey @gfredericks: having the fn return data which you throw is better - (s/keys :opt-un [:ex-message :ex-data])?
2016:06:09 17:34:25 ikitommi @rickmoynihan: the plumatic->json schema is done within the lib (the json-schema ns, based on protocol & supporting multimethod). Swagger (OpenAPI nowadays) only supports only a "pragmatic” subset of JSON Schema as it’s original target was the OO-languages like Java - has client code generators for those. Things like oneOf or anyOf are not supported - the requests to add those are over 2y old now. I think having a separate pure spec<->json schema would be awesome! Having it work with swagger requires some extra work, happy to do/help with that.
2016:06:09 18:30:56 gfredericks richhickey: yeah that's essentially what I was thinking; can you think of any args that should be passed? if we make it a one-arg function that's passed an empty map that would be better for backwards-compatibly adding more things later :) I'm wondering because if there's no need for any args then it's not clear why it even needs to be a function and not just passing the data directly
2016:06:09 18:52:28 gfredericks this fancy defn is fun; I suppose it's a lot like core.match
2016:06:09 18:56:00 gfredericks uh oh I just made it stack overflow
2016:06:09 19:05:53 gfredericks there's a note in there about a stack overflow that I haven't tried to debug, that happens when I add a spec to the defn+spec macro itself
2016:06:09 19:08:04 fxposter hi everyone
I have 2 questions about clojure.spec, which are not that obvious from the beginning:
1. is it okay to use predicates that connect to external resources for validation? ie: I have a tree of "paths" and want to validate that it matches the filesystem or posts on some website? I don't see any techncal problems with that, but maybe there are other solutions for that?
2. for example, I have 2 data structures: new {"application": {"branches": {"release": "1.0", "snapshot": "1.1"}}} and old {"application": {"branches": {"release": "0.9", "snapshot": "1.0”}}} and I want to actually check that the new value is a "valid update" of the old one, based on some constraints (for example: "new value contains at least all branches from the previous one" and "all versions in new value are greater than the same ones in the old one"). is there a way to at least partially express that in clojure.spec and get validation + error reporting?
Thanks
2016:06:09 19:12:16 gfredericks I think the predicates are supposed to be pure functions
2016:06:09 19:12:46 gfredericks I couldn't say where the first place you would run into trouble would be though
2016:06:09 20:28:08 akiel Can someone explain what I found in my snippet?
2016:06:09 20:36:24 benzap does replacing sp/alt with sp/or make it work?
2016:06:09 20:37:05 akiel Yes!
2016:06:09 20:37:32 benzap haha, i've had the same issue
2016:06:09 20:38:01 benzap someone worked out the differences. It has something to do with how sp/alt is used in regex, and sp/or is used otherwise.
2016:06:09 20:38:26 akiel So sp/alt on something which is not a regexp doesn’t work? Ok I see...
2016:06:09 20:38:54 benzap Simply put yes. Someone else on here had a really good explanation, but I didn't completely follow
2016:06:09 20:39:14 benzap He was wondering why sp/alt and sp/or were so similar, and produced the same results in some contexts
2016:06:09 20:39:26 benzap had to do with the regex distinction
2016:06:09 20:40:08 akiel so maybe we need spec for spec - I mean it’s all macros - everything can happen
2016:06:09 20:41:14 benzap haha, i'm curious to see how much spec is used 🙂
2016:06:09 20:41:48 benzap I wonder if they'll use clojure.spec on all of the clojure.core to try and get away from the java stracktraces
2016:06:09 20:42:20 akiel cat and alt says: “returns a regex”, and says: "returns a spec” - so maybe a spec is not a regex 🙂
2016:06:09 20:42:33 benzap ah ok
2016:06:09 20:42:47 akiel type systems would help rant
2016:06:09 20:43:43 benzap I was talking to someone about clojure.spec on freenode#programming, and he said clojure.spec is like contracts in racket
2016:06:09 20:44:14 benzap the idea being that they're supposed to be more powerful, since you can apply predicates
2016:06:09 20:45:02 akiel yes spec is more powerful as a type system, because you can inspect actual values at runtime
2016:06:09 20:45:26 benzap ya, the whole instrumentation is pretty neat
2016:06:09 20:45:45 benzap i've used it, but it's rather slow. I need to start using it per namespace
2016:06:09 20:46:13 akiel just use it only in dev and test
2016:06:09 20:46:39 benzap I have been, but it's even slow then. Maybe 20 times slower
2016:06:09 20:46:43 benzap makes testing rather slow
2016:06:09 20:46:58 benzap It isn't an issue if I run it once in a while, so i'll throw down instrumentation every once in a while
2016:06:09 20:47:40 benzap But I wonder if there's any room for improvement in performance, it would make it very appealing to just leave instrumentation enabled even in production
2016:06:09 20:48:22 benzap There's a reddit post comparing different schema/data validation libraries. No one has commented on it yet, and I wish someone more knowledgable would
2016:06:09 20:48:35 akiel Than I think, you apply spec to inner loop things. I would apply it only to my public API.
2016:06:09 20:48:52 benzap ya, that makes sense
2016:06:09 20:49:07 benzap like, only validate data that comes in from an external source for production?
2016:06:09 20:50:36 akiel In production I would check data coming over wire directly without instrumentation, just in normal code.
2016:06:09 20:51:07 akiel I do this with Schema already.
2016:06:09 20:51:16 benzap I've been using the clojurescript version of clojure.spec, cljs.spec
2016:06:09 20:51:20 benzap oh nice
2016:06:09 20:51:34 benzap yeah, I converted my project i've been working on from schema to spec
2016:06:09 20:51:48 benzap didn't take that long, but it did require quite a bit of refactoring
2016:06:09 20:52:36 benzap I'm still not sure where I should put the specs. I kind of placed them at the beginning of the files, and would s/fdef after each function
2016:06:09 20:53:34 benzap I suppose that's good enough, I don't think i've tested s/fdef when the function hasn't been declared yet, so I don't think I could place them in another file?
2016:06:09 20:53:40 akiel I’m not completely sure either. But I have seen putting s/fdef before each funtion often.
2016:06:09 20:53:48 benzap oh, before?
2016:06:09 20:53:50 benzap hmm
2016:06:09 20:54:03 akiel yes it is possible
2016:06:09 20:54:11 benzap that's good to know
2016:06:09 20:54:18 akiel its like contract first and than the implementation
2016:06:09 20:54:35 benzap that makes sense
2016:06:09 20:54:54 benzap i'll have to consider moving my stuff into a separate folder
2016:06:09 20:55:24 benzap it's hard to get used to, really. I considered it an eye-sore at first
2016:06:09 20:55:37 benzap definitely useful though, I caught a lot of bugs early
2016:06:09 20:56:57 akiel opposite to schema, the annotations are not inline to the function - so the possibilities are broader - like in core.typed I think - never used it
2016:06:09 20:57:49 benzap Yeah, that would explain my confusion. At first, I had a hard time figuring out how to convert from schema
2016:06:09 20:58:00 benzap made a lot more sense once I got started
2016:06:09 20:58:42 benzap when I used schema, I had originally defined the structures in a separate file, sortof like how you would define a jsonschema
2016:06:09 20:58:53 arohner @benzap: it’s probably slower because it’s doing generative testing at runtime. Hopefully that becomes a config setting
2016:06:09 21:00:15 benzap @arohner: that's interesting, I thought it was trying to conform each function with respect to it's defined spec. If it's also throwing in generative testing, I can see that causing some performance issues
2016:06:09 21:01:35 arohner well, fspec does. checking fdef now
2016:06:09 21:02:46 benzap a somewhat related topic, there's a library in clojure called specter, which compiles itself to increase performance
2016:06:09 21:03:19 benzap I wonder if the same concept could be applied to clojure.spec, where you could pre-compile the validator for your functions
2016:06:09 21:19:27 fenton what would a spec look like that does the following: if a map has one key, it should have another key
2016:06:09 21:19:33 bbrinck And in plumatic schema, you can create validators ahead of time for performance.
2016:06:09 21:24:33 fenton oops wrong description of problem. if a key in a map has a certain value, then another key should be present. writing specs can be a challeng...like a whole nother language! 🙂
2016:06:09 21:31:11 angusiguess @fenton That is likely a multispec
2016:06:09 21:31:47 angusiguess Where you write a multimethod looking for the first key, and that dispatches another spec to check for the second.
2016:06:09 21:32:00 fenton @angusiguess: ok, will look into that. thx.
2016:06:09 21:32:09 angusiguess 😅
2016:06:09 21:58:23 arohner could also be a simple s/or
2016:06:09 21:58:50 arohner depends on how complex it needs to be
2016:06:09 22:01:55 danielcompton @rickmoynihan: we use coercions which are extremely handy
2016:06:09 22:03:25 danielcompton If spec (or surrounding tooling) doesn’t support automated coercions, then we’ll need to keep schemas for our boundary interfaces between webapp and RethinkDB
2016:06:09 23:09:50 rickmoynihan danielcompton: Yeah - I've used coercions before too - and I don't dispute their utility at boundaries... I guess you could easily build a specialised coercion library on top of spec though... don't know enough yet how you might do that...
Regardless I much prefer what I've seen of spec to schema; and don't think coercions are enough of a feature on their own to either not use spec, or use schema as well as spec... I'd definitely much prefer something that worked with spec
2016:06:10 02:23:32 sparkofreason Is there a straightforward to define a function spec and reuse with multiple fdef's?
2016:06:10 07:43:37 mpenet @rickmoynihan @danielcompton : same here, also free type hinting, custom explain messages, performance and the list goes on. So far we have no reason to migrate really, we're waiting to see where clj.spec is going, it's probably a bit too early
2016:06:10 13:51:37 ikitommi Besides coercions, my top feature request to spec: helper-fn to create a spec from a vanilla clojure (function) var. It would understand the Clojure destructuring syntax. Something like this:
* the responsibility to extract (and use) the specs would be on the user - I would use these on the web-tier to auto-extract docs & do coercion in the web-api tier with our libs
* would not add new meta-data to vars (arguments are already in :arglists)
* no need to describe the shape of the data twice (both for the function arguments & for it's spec)
* (optionally the :ret and :fn could be read from the Var metadata too)
Thoughts?
2016:06:10 14:01:03 angusiguess I think this is rad, my only real question is why might we rely on spec to do it?
2016:06:10 14:01:34 angusiguess There are pretty good hooks to pull validation information out of a defined spec and an alternate defn like this should be a macro.
2016:06:10 14:02:13 angusiguess I don't work on spec so this is grain of salt stuff, but it seems like the opinion of spec is that people could conceivably bring their own sugar but under the hood there's a common language for validation.
2016:06:10 14:09:14 alexmiller @ikitommi: you could build that from what exists now. since it wouldn't generically apply, I don't think we would do that as part of core or anything. One thing that would help is a spec for destructuring, which I have and which will be released at some point in some form (details TBD still)
2016:06:10 14:16:25 ikitommi @alexmiller: spec for destucturing sounds cool. Did a dummy version of the extractor, will play more with it. Are there any caveats in playing with :arglists?
2016:06:10 14:18:25 alexmiller there are a few cases where people have abused it a bit in what was put in it (data.generators is one that comes to mind) but generally should be fine
2016:06:10 14:19:32 alexmiller I think I would also consider allowing overrides via lookup in the registry - s/fdef registers stuff there under the fn symbol and those can be obtained via s/fn-specs
2016:06:10 14:19:51 alexmiller so you have an existing registry for overrides of things you couldn't build automatically
2016:06:10 14:21:08 alexmiller also note that CLJ-1919 will add a new syntax for namespaced key destructuring.
2016:06:10 14:21:55 alexmiller your example there is not syntactically correct btw - the keys of :or should always be unqualified symbols (matching the bindings that are created). there are some bugs in this area in current Clojure that will be fixed in CLJ-1919.
2016:06:10 14:22:20 alexmiller so that is, what you have there probably works now, but by accident not intent, and will change
2016:06:10 14:24:13 ikitommi uh, copy-paste error in the code. But thanks! will check out your pointers.
2016:06:10 15:10:16 wilkerlucio what's the correct way to express to an fspec that a function takes no arguments?
2016:06:10 16:13:36 gfredericks (s/cat) I'd guess
2016:06:10 16:13:54 gfredericks maybe #{()} would work too
2016:06:10 16:16:46 gfredericks alexmiller: there's no reason not to add specs to test.check is there?
2016:06:10 16:17:09 gfredericks assuming it accounts for older clojures
2016:06:10 16:18:52 gfredericks I guess this question is a superset of seancorfield's question on the ML, but also about test.check in particular since it's used in clojure.spec
2016:06:10 16:24:04 seancorfield I haven't moved forward with that since the first cut. Want to see more discussion on the ML first.
2016:06:10 16:24:52 seancorfield (sorry if I don't follow up for a few hours -- doors closing en route for a cat show!)
2016:06:10 16:28:36 alexmiller None other than that it then requires Clojure 1.9
2016:06:10 16:28:53 alexmiller Which requires test.check
2016:06:10 16:46:41 gfredericks okay, cool; I'll probably do a separate .specs namespace like sean did
2016:06:10 17:11:55 jcf Anyone tried writing specs for stateful objects like database connections, or a Datomic database?
2016:06:10 17:14:51 jcf I can't think of a nice way to specify a function takes a db, and some other args. To generate a db I need a database connection that isn't available when I define my specs.
2016:06:10 17:16:13 jcf Imagine a trivial example like this:
2016:06:10 18:05:31 gfredericks seancorfield: were you thinking of having a conditional require in the .jdbc namespace? otherwise you'd have the problem of up-to-date users having to opt-in to the specs
2016:06:10 18:05:50 gfredericks when there are use cases for the specs besides explicit testing, e.g. clojure.repl/doc
2016:06:10 19:34:02 jcf Is there a way to merge two s/keys specs?
2016:06:10 19:34:17 jcf I must be missing something. Back to the manual!
2016:06:10 19:35:30 donaldball I’ve been idly thinking about the problem of db args as well, though in the sql context. The problem seems the same for datomic and jdbc though: a spec saying the db arg is e.g. a jdbc connection isn’t sufficient. You really want a spec that says this value is a jdbc connection to a database with at least a certain schema and maybe even a certain set of entities.
2016:06:10 19:35:42 jcf Oh wait, just me being stupid apparently.
2016:06:10 19:36:43 jcf @donaldball: I'm using this at the mo:
2016:06:10 19:37:19 jcf Don't love it if I'm honest. Creating a new connection for every test is pretty inefficient, but it works.
2016:06:10 19:38:19 jcf If you've not used Component or Datomic that's probably meaningless.
2016:06:10 21:02:31 gfredericks oh woah
2016:06:10 21:04:29 gfredericks I hadn't thought about this generator setup encouraging people to use stateful resources in their generators
2016:06:10 21:12:35 dominicm @gfredericks: I'd say that the encouragement and bias towards namespace level hoisting (with s/def) takes away our ability to lexically scope and generally pass around explicit arguments.
Diving too deeply into that statement takes you to mount vs component.
2016:06:10 21:15:12 gfredericks dominicm: the way I've used generators in the past is purely data-driven, so there's not even component-like stuff until the test starts running; but if you have a spec that's explicitly for a stateful thing, then you can't write a generator for it that way
2016:06:10 21:16:01 gfredericks my gut would be to try to keep doing the data-driven thing, and so not use generators for specs that describe stateful things
2016:06:10 21:16:11 gfredericks not sure how easy that is with datomic
2016:06:10 21:16:51 gfredericks a datomic entity is a map of attributes, is that right?
2016:06:10 21:17:03 gfredericks if that's the case you could make most of your specs just expect maps
2016:06:10 21:17:17 gfredericks and could do low-level testing with maps instead of entities
2016:06:10 21:24:17 dominicm @gfredericks: It's definitely difficult to manage with generators. The discussion (in my opinion) transcends just testing. Doing explicit s/explain-data on a runtime database is also a use-case.
2016:06:10 21:24:57 gfredericks that should work with map specs though I would think?
2016:06:10 21:29:27 dominicm (s/explain-data {:email "
The unique email checker needs a database to make it's check. If I make it part of the map data (assoc m ::db (d/db conn)), then when I check, the returned data gives me an incorrect path to the error position.
2016:06:10 21:30:46 dominicm I have just got a macro working, which would take the error generated, and readjust the path for you. But it's still somewhat unnatural.
2016:06:10 21:32:09 gfredericks so your spec is making database queries?
2016:06:10 21:33:04 ghadi that is a bad idea
2016:06:10 21:36:12 gfredericks I think specs should be pure functions
2016:06:10 21:43:31 ghadi absolutely
2016:06:10 21:44:58 gfredericks something that's not a pure function can be a plain ole test :)
2016:06:10 22:14:46 eggsyntax I need to write a script to generate some Datomic seed data, and I'm experimenting with using spec to do so. Two questions:
1) One part of the seed data I need to write is the key/val that'll generate a temporary db/id. If I were hand writing the seed data, it would look like {:db/id #db/id[:db.part/user -1015948] ... }. Can anyone help me understand how I would go about creating a spec for that?
2) ideally, it'd be nice to just spec that part of the schema, and then somehow use that to generate the seed data, but I'm not quite sure how go about that. Any hints?
2016:06:10 22:15:45 eggsyntax In other words, how can I spec a tagged literal like that?
2016:06:10 22:35:15 eggsyntax (or at least that's the spec for the tagged literal itself. I'm not quite sure how to get from that to a spec that'll produce seed data as above. I guess I'll need to write a generator for it, but it's gonna take some experimentation for sure.
2016:06:10 22:37:38 hiredman using spec just to generate data seems super weird, why wouldn't you use the generators from test.check or https://github.com/clojure/data.generators directly?
2016:06:10 22:39:45 eggsyntax Well, if I can figure out the relationship between a spec for the schema and a spec for the seed data, I can use the spec to do validation as well as generating seed data, and maybe find other uses for it as well.
2016:06:10 22:40:37 eggsyntax And it's also partly a clojure.spec learning exercise for myself 🙂
2016:06:11 11:05:32 borkdude Does spec offer anything in relation to coercing like Schema does?
2016:06:11 11:36:38 alexmiller Not really. It does have conformers but their intent is a bit different.
2016:06:11 11:38:49 alexmiller @rymndhng: this is the intended behavior - map-of samples its values for performance reasons so doesn't conform everything. This been a common question though and something will probably be added for it.
2016:06:12 08:02:31 patrkris Hi Everyone. How do you name aggregate stuff with namespaced keywords? Say I have a map that would look like this without namespaced keywords
The keys at the first level might be namespaced like :customer/first-name and :customer/last-name, but how would I name the keys inside the address map?
2016:06:12 17:06:35 danstone Does anyone know of a way to compose key sets? e.g something like (s/merge ::foo ::bar)
edit: and works for validation, but does not yield a working generator...
2016:06:12 17:10:17 t3chnoboy Hi! I’m trying to validate a javascript class and not sure how to make it work:
2016:06:13 00:00:25 t3chnoboy Is there a way to validate core.asyncchan type?
2016:06:13 00:02:57 t3chnoboy I want to define a spec for a function which returns the following map:
{:in-chan (chan)
:out-chan (chan)}
2016:06:13 00:05:34 jfntn @t3chnoboy: you should be able to make a predicate for a ManyToManyChannel instance, or perhaps more generically test that (satisfies? ReadPort in-chan) and (satisfies? WritePort out-chan) also both defined somewhere in core.async...
2016:06:13 00:17:49 jfntn indeed, think you could import ManyToManyChannel and reference it directly though
2016:06:13 00:21:14 jfntn The protocol check is probably a better way to go however
2016:06:13 08:48:00 dominicm @gfredericks: Yep, my specs make db queries. Spec has s/valid? to accomodate runtime validations (not just function checking in development time). I honestly think having two libraries, one for "types" and one for "db connecting checkers" would be wasted effort.
2016:06:13 11:10:17 jimmy hi guys, I want to discuss a bit about spec. Spec is amazing, I can see the workflow that we can try out things in dynamic clojure, then add spec to it later to ready for production. But would spec have an optimization behind to check for example: we define a function with the args which is an integer? then spec will also generate a function with type hint integer beside doing the validation ? Then we can have something that is very flexible at output error and very fast code as well. Just my thought.
2016:06:13 12:10:16 shem is there a way to write a spec that says "this map should contain these keywords, and only once each"?
2016:06:13 12:17:18 stathissideris @shem: but keys can only appear once each in a map... by definition!
2016:06:13 12:21:42 shem ergh, right. was eyeing generator output that spat several instances of the map. need more coffee
2016:06:13 12:42:48 jimmy hi guys, how do we define a spec that can validate both namespaced key and non namespace key, for example : user/first-name and first-name one of those would be valid.
2016:06:13 15:27:26 settinghead has anyone tried using core.async with clojure.spec? i ended up having a lot of functions that returns a channel in my code (`(defn [] (go …))`). i’d like to spec these functions beyond checking its type is ManyToManyChannel (i.e. checking what comes out the channel is valid based on a spec). I’m thinking of adding transducers to the returning channels. but is there a better overall approach? should i write my functions this way to begin with?
2016:06:13 15:41:19 jimmy @manderson: I meant
(s/valid? ::map-spec {:user/first-name "joe"})
and (s/valid ::map-spec {:first-name "joe"})
2016:06:13 15:43:12 manderson ::first-name is the same as :user/first-name so either should work with above spec. Is that what you're asking?
2016:06:13 15:43:47 manderson ^ is the same if you are in the same namespace. :: simply appends the current namespace qualification to the keyword
2016:06:13 16:18:22 jimmy @manderson: sorry, I was out. I will check on this, I think I miss understand something here.
2016:06:13 16:27:25 jimmy yeah I did mis understand, it works fine. with user/first-name. thanks
2016:06:13 16:32:48 jimmy @manderson: ah I have another question, if I refer the spec in another ns, it shouldn't work with :user/first-name case as long as I understand ?
2016:06:13 16:38:31 manderson using the definition of :user/map-spec as defined above, it will validate fine for :user/first-name or :first-name. The namespaced keywords reflect the namespace they were defined in if set with ::.
2016:06:13 16:41:31 jimmy ok, I see. in s/keys is there a way that we can validate something like :first-name or :user/first-name, one of those is required.
2016:06:13 17:02:56 manderson hm, good question. this seems to work:
2016:06:13 17:03:47 manderson the keywords in or are just tags...
2016:06:13 19:24:26 danstone Question, when should one specify a property of a function in the :fn of an fdef rather than in a proper test.check property?
2016:06:13 20:48:05 danielcompton I’m pretty sure the answer will be “no”, but is there any way to use spec to define newtype’s, i.e. I have a username and a full-name which are both strings. Can I spec functions to stop passing a username where a full-name is required? For this discussion, assume that both strings have no distinguishing features you could use.
2016:06:13 20:50:03 danstone You might be able to use conform with a custom conformer for 'full-name'. edit: never mind, it won't work - as you say such a conforming fn would never know the difference between the two values
2016:06:14 03:25:26 mfikes It’s too bad (s/describe (s/spec #(< % 3))) has to return something that doesn’t quite look like an anonymous function literal
2016:06:14 03:34:18 mfikes I’m not sure what to make of the fact that
look the same. Perhaps it is OK because they behave the same way.
2016:06:14 05:02:19 jimmy hi guys, how do we use variable inside of (s/def (s/keys)) like this
(def a [::first-name])
(s/def ::user (s/keys :req a))
It's a value in a macro in a macro. And I'm no macro master ...
2016:06:14 05:44:42 jimmy I have an error while trying to generate from clojure.spec, I think this would be the try out such-that limitation in clojure.spec implementation.
(defn str-limit [lower upper] (s/and string? #(<= lower (count %) upper)))
(s/def :project/description (us/str-limit 116 1000)) ;; the gen doesn't work if the lower value is around above 100
(s/def ::project-gen (s/keys :req [:project/description]))
(gen/generate (s/gen ::project-gen))
;; -- error
ExceptionInfo Couldn't satisfy such-that predicate after 100 tries. clojure.core/ex-info (core.clj:4703)
2016:06:14 07:09:59 stathissideris @nxqd: This is because string? generates totally random strings and then checks the second predicate to see if they conform, and gives up after 100 tries. Consider wrapping the spec with s/with-gen and providing your own generator.
2016:06:14 09:07:47 jimmy @stathissideris: thanks !
2016:06:14 11:26:13 skapoor hi guys, am playing with clojure spec and running into an un-expected behavior: (s/valid? #{nil "a" "b"} nil) ;; returns false when it should be true
2016:06:14 11:28:55 minimal nil is not truthy so the set as a predicate is returning a non-truthy value
2016:06:14 11:29:57 minimal
(s/valid? #(contains? #{nil "a"} %) nil)
true
2016:06:14 11:30:47 skapoor @minimal yeah, I tried that and it works.. but it should return true when a set is used in specs too right?
2016:06:14 11:33:34 minimal If you are relying on the value returned from the set then it needs to be truthy to be valid
2016:06:14 11:36:04 skapoor okay, (map #{"a"} ["a" nil]) and (map #{"a" nil} ["a" nil]) both return ("a" nil)
2016:06:14 11:37:46 minimal The not-found value of a set is also nil so it is troublesome if you don’t expicitly check using contains?.
2016:06:14 11:38:56 skapoor i'm using sets in specs to define an enum of values with the possibility of nil.... so it looked like a convenient way. guess, I'll have to either use s/or nil? ... or #(contains?)
2016:06:14 11:39:07 minimal or you can use a defualt value that isn’t nil. (#{} 1) => nil(get #{} 1 :not-found) => :not-found
2016:06:14 11:40:02 minimal Yeah it’s tricky
2016:06:14 11:41:50 skapoor ok thanks. i'm wondering if I should file an issue for this on jira..
2016:06:14 11:42:14 minimal I don’t think it’s an issue
2016:06:14 11:42:33 skapoor only because the contains? behavior is right but when a set is used as a function call it's not.
2016:06:14 11:43:14 minimal Using set as a predicate is a convenience but you are relying on an implicit conversion to boolean based on the value in the set
2016:06:14 11:45:58 gfredericks either choice is surprising
2016:06:14 11:46:11 gfredericks to somebody
2016:06:14 11:46:22 gfredericks depending on whether you expect the set to be treated specially or to be treated like a predicate
2016:06:14 11:46:31 skapoor the implicit conversion to boolean is being done by the internal clojure implementation..
2016:06:14 11:49:49 skapoor @minimal: but I get it, it won't be considered a defect. so I'll just use a different way. thanks!
2016:06:14 15:20:14 alexmiller @mfikes from your question way back on describe - s/form is useful for distinguishing these
2016:06:14 15:23:23 alexmiller @skapoor: you could also wrap s/nilable around the set (s/nilable #{"a"})
2016:06:14 17:39:52 arohner @alexmiller: what’s the rationale behind instrument only checking :args?
2016:06:14 17:53:19 alexmiller shift in perspective - instrument is about checking invocations are correct, not verifying the function itself is correct. (this is similar to how only the :args are checked in macro fdefs). The check and test functions are for verifying functionality of the code.
2016:06:14 18:05:49 bfabry is there still a way to set up your app so every spec is checked on every invocation?
2016:06:14 18:41:35 arohner @alexmiller: this seems to severely hamper spec’ing non-pure functions (i.e. things that are difficult/impossible to write generators for)
2016:06:14 18:42:33 arohner and IMO, validation and generative testing are still too tightly coupled
2016:06:14 18:49:43 alexmiller @bfabry: no, but that was not really the intention of instrument, which is about verifying that callers of a function are calling it correctly. If you want to test the the behavior of your functions are in accordance with your specs, you should use the spec.test functions to test your functions.
2016:06:14 18:51:17 bfabry @alexmiller: sure, and no doubt were we to use spec we would use the spec.test functions. but like @arohner mentioned not all functions are pure, and if I'm going to write all those specs then I might as well get some extra value from them for free by having them always be checked in lower environments where I do not care about performance
2016:06:14 18:51:34 alexmiller @arohner: there are still more things coming that will help with verifying aspects of non-pure functions
2016:06:14 18:53:41 arohner @alexmiller: one nice feature of schema is that you can choose to use validation at e.g. user-input boundaries, in production. Instrument seems more designed for dev-time atm
2016:06:14 18:53:59 alexmiller @arohner spec does not remove the need to write tests for your functions. those tests can use invoke specs to validate as appropriate
2016:06:14 18:54:14 alexmiller @arohner: instrument is only designed for dev time
2016:06:14 18:54:25 alexmiller you should not instrument in production
2016:06:14 18:55:14 alexmiller you can choose to explicitly conform with spec at boundaries if you like
2016:06:14 18:56:10 alexmiller there will be a conform-ex that validates or throws if not, not quite in yet
2016:06:14 19:03:04 bfabry if I explicitly conform, then it will happen in production, when I only want it in staging. it also adds a whole bunch of boilerplate to every single function. I don't really understand the reasoning here, being able to reuse the :ret and :fn specs for extra checking when performance isn't a consideration just seems like an obvious win. and I mean, I definitely can still do that, writing my own macro that wraps all functions or whatever, but it sounds like I won't be the only one
2016:06:14 19:10:26 alexmiller there is an assertion facility still coming as well
2016:06:14 19:11:27 alexmiller the point is that checking ret/fn every time should be redundant to what you have (presumably) already confirmed in testing - that your function works.
2016:06:14 19:16:30 bfabry I'm maybe a bit skeptical that adding generative testing (while definitely awesome) is going to straight away mean I stop writing functions that produce unexpected outputs when they encounter production data. and I'm a big fan of fail fast with a good error message when that does happen
2016:06:14 19:19:44 alexmiller you (will be) able to assert that return (if instrumented at dev time) or choose to explicitly validate it at production time if you want
2016:06:14 19:20:16 alexmiller it's unclear to me if you're talking about dev or prod
2016:06:14 19:20:32 bfabry actually talking about master/staging
2016:06:14 19:21:10 alexmiller fair enough - so you can turn on instrumentation in staging
2016:06:14 19:21:36 alexmiller that will check args on functions
2016:06:14 19:21:48 bfabry on my laptop/travis I run unit and generative tests, in master/staging the application runs "production-like" but with assertions turned on for :args :ret :fn, production the app runs with no assertions <-- this is my ideal scenario
2016:06:14 19:22:05 alexmiller there will be an assertion facility that you can use to (explictly) check ret/fn for instrumented functions
2016:06:14 19:22:55 danstone As the channel is a little bit louder this evening, I thought I'd pose a question I asked yesterday again:
What is the intended usage of :fn in fdef - In the docs it says something like 'relationships between args and the return'. Is the idea here to restrict it to type-y properties (e.g arity 1 of map returning a transducer rather than a seq)?
The reason I ask is it's possible to define many more general properties of functions as part of the spec. As spec gets richer I imagine it may be possible to auto-generate the code for many properties (idempotency is easy if you have spec'd the args)
2016:06:14 19:23:43 alexmiller I think what I'm talking about is close to that, but varies in that ret/fn are not automatically checked but require an explicit assert (which is not active in production)
2016:06:14 19:29:05 bfabry right. and so I'll probably end up writing a macro that wraps every function to add that explicit assert, and I've got a feeling that a whole lot of people will do that. we use plumatic/schema atm which checks arg/return values when validation is turned on, and I'd say the same number of bugs are caught by the return value checking as the arg value checking. aaaanyway, writing the macro is nbd, and maybe it'll turn out I don't actually need it or I'm the only person who does 🙂
2016:06:14 19:41:04 alexmiller one question I have is whether you're getting the same level of generative testing from schema that you can get from spec (that is, whether more of those bugs could/should have been caught earlier)
2016:06:14 19:41:52 alexmiller also keep in mind that the return value often is an input to another function, which can check its args
2016:06:14 19:44:40 bfabry no, we're definitely not. but like I said I'm just a bit skeptical that generative testing will suddenly mean these issues disappear, and it costs me like an extra $5 per month to run extra validation in the staging environment so why not? the return value will likely be the input value to another function, but maybe that function doesn't have a spec yet, because we didn't feel it was worth the time to write yet, or maybe it's too broadly defined etc. If I believed I could perfectly specify every function up front so that bugs were impossible I'd be writing haskell 😆
2016:06:14 19:48:16 alexmiller maybe you should just write your code without the bugs?
2016:06:14 19:48:26 alexmiller 😀
2016:06:14 19:53:51 bfabry haha
2016:06:14 20:49:26 wilkerlucio hey people, does anyone here found/created a generator for cljs to create strings from regexps? I was hoping to use the string-from-regexp from test.chuck but I just realised it's implementation is for CLJ only
2016:06:14 20:56:20 alexmiller I would ask @gfredericks
2016:06:14 21:09:02 wilkerlucio thanks Alex, I opened an issue on test.chuck, he will see it I think 🙂
2016:06:14 21:09:56 wilkerlucio one more thing, given I have my fdef all spec set, I remember seeing a simple command to test that function but I'm not finding it, what's the simplest way to run generative specs on a function?
2016:06:14 22:10:39 ikitommi Is this good? should the specs be immutable by default? If there are name clashes, depending on the import order of namespaces, the specs might mean different thing.
2016:06:14 22:11:15 ikitommi or some hooks for the registry to resolve those clashes.
2016:06:14 22:14:58 wilkerlucio @ikitommi: this is the reason they are encouraging namespaced keywords I think, do you have a situation where the same namespace is loaded on multiple files?
2016:06:14 22:15:03 bfabry functionally the same as multiple (def's isn't it?
2016:06:14 22:20:59 bsima does anyone have examples of using fdef and check-var? I'm having trouble getting it to work in my test suite
2016:06:14 22:23:28 ikitommi @wilkerlucio: true that - the specs must have a namespace, but it doesn’t have to be a clojure namespace. One might have multiple :order/ids in a large system. Should not, but could.
2016:06:14 22:23:31 wilkerlucio @bsima: check this snippet, may help you:
2016:06:14 22:25:47 wilkerlucio @ikitommi: that's some sort of thinking shift, moving from those to fully qualified, there are going to be some nice helpers to deal with longer namespaces in 1.9
2016:06:14 22:26:50 wilkerlucio yes, the problem exists like you said, but it's the same for def as mentioned by @bfabry, I guess it's just about people starting moving towards fully qualified namespaces to avoid name clashes, I believe when it's the norm will be very positive for everyone
2016:06:15 04:44:38 jimmy hi guys, how do we use variable inside s/keys like this
(def a [::first-name])
(s/def ::user (s/keys :req a))
2016:06:15 09:54:49 danstone @nxqd: I don't think we can because s/keys is a macro. Though I'm not sure if this is worth it for keys, as it doesn't capture predicates, I would expect it to be a function. As far as I can tell the only thing macro'ey it does is look at forms like (or ::foo ::bar).
2016:06:15 17:50:25 alexmiller At the moment you can't do this except via eval or something
2016:06:15 17:50:52 alexmiller But there has been talk of adding a fn entry point that would allow it
2016:06:15 18:57:09 seancorfield With check-fn and check-var, is there a way to get a "nice" explanation of the failures, like explain produces?
2016:06:15 18:57:48 seancorfield I'm trying to see how the changes in Alpha 6 affect the testing workflow.
2016:06:15 19:47:01 eggsyntax @nxqd: (somewhat belatedly) you can replace the var with the function that populates it (it can cache its results, if it's an expensive call), if it's a var that it makes sense to populate at macro eval time.
2016:06:15 19:48:47 eggsyntax Unrelated: test.check has a fn (`fmap`) that lets you call an arbitrary fn on the results of a generator. But is there any way to create a generator that just calls an arbitrary fn? I haven't found one.
2016:06:15 19:59:45 eggsyntax The only solution I've found so far is to do (fmap (fn [_] do-what-I-want) some-arbitrary-gen), but it's pretty ugly to put in a generator and then ignore what it generates.
2016:06:15 20:52:38 jannis Folks, is it a good idea to define fdef function/macro specs next to the actual function/macro definition or would you rather define them in separate my-project.specs kind of namespace?
2016:06:15 20:53:40 jannis I'd like to associate most of my functions with specs for automatted random function testing but at the same time I don't want to clutter my code base with specs too much.
2016:06:15 20:53:53 jannis I guess there are no best practices established yet?
2016:06:15 20:54:47 eggsyntax I've been debating that myself, haven't come to any particular conclusion.
2016:06:15 20:55:57 eggsyntax Currently defining them on their own because it's still fairly experimental for us...but as we fully integrate them, I'll definitely consider moving them to live next to what they're speccing.
2016:06:15 21:11:58 seancorfield I think we’ll have data specs in separate namespaces. Not sure yet about function specs. Part of me would like them above function definitions — that seemed natural for when we used Schema and core.typed.
2016:06:15 21:12:45 seancorfield We went back and forth between core.typed and Schema several times before we abandoned them. With spec being in core, I think we’re more likely to stick with it.
2016:06:15 21:12:56 jannis Yeah. It also means you don't have to require the function namespaces and the function spec namespaces when you want to test functions.
2016:06:15 21:18:44 jannis It would feel a little more natural to define the function spec like`:pre` and :post, e.g.`(defn my-fun [arg1 arg2] {:spec {:args ... :ret ... :fn ...}} <body>)` or something like it, although there are good reasons to keep them separate.
2016:06:15 23:22:52 bbrinck @jannis: What do you think about putting the fdefs before the function? IMHO, that's a little clearer. I like having the function specs near the functions since that provides documentation when reading the function. Not sure about data specs, though.
2016:06:15 23:29:18 bbrinck or rather, put each fdef before its associated fn
2016:06:15 23:45:09 bfabry what have I messed up here...?
Clojure 1.9.0-alpha7
Java HotSpot(TM) 64-Bit Server VM 1.8.0_65-b17
Exit: Control+D or (exit) or (quit)
Commands: (user/help)
Docs: (doc function-name-here)
(find-doc "part-of-name-here")
Find by Name: (find-name "part-of-name-here")
Source: (source function-name-here)
Javadoc: (javadoc java-object-or-class-here)
Examples from : [clojuredocs or cdoc]
(user/clojuredocs name-here)
(user/clojuredocs "ns-here" "name-here")
boot.user=> (require '[clojure.spec :as s])
nil
boot.user=> (require '[clojure.spec.test :as t])
nil
boot.user=> (defn foo [x] (inc x))
#'boot.user/foo
boot.user=> (s/fdef foo :args (s/cat :x integer?) :ret integer?)
boot.user/foo
boot.user=> (t/check-var foo)
java.lang.IllegalArgumentException: No :args spec for
2016:06:16 00:35:21 seancorfield (t/check-var #’foo) — it accepts a Var.
2016:06:16 00:36:31 seancorfield Similarly (s/fn-spec #’foo) — although the result of that is less useful to print since it is an object.
2016:06:16 00:37:45 seancorfield You could also do (t/check-fn foo (s/fn-spec #’foo))
2016:06:16 00:38:44 seancorfield check-fn is so you can provide an (s/fspec :args … :ret … :fn …) for an arbitrary function without fdef.
2016:06:16 01:23:50 leifp Hi, all. I hacked up specs for most fns in clojure.core and 1/3rd of the macros (WIP). I don't know much about clojure.spec (of course), though, so it wasn't the good kind of hacking,. Hold your nose if necessary: https://gist.github.com/leifp/abe50082f6baa0063f8b7840e80657af
2016:06:16 09:58:59 jannis Shouldn't ((instrument #'my-fun) :foo) throw an exception with an explanation if my-fun has a spec defined for it with fdef and if the returned value does not conform to the spec of my-fun?
2016:06:16 09:59:15 jannis Instead it just returns :clojure.spec/invalid.
2016:06:16 10:04:09 jannis Oh, that's because that's what my-fun returns in this case. So it doesn't even fail, it just returns its value, even if that doesn't conform to the function spec.
2016:06:16 10:41:39 patrkris I see clojure.spec uses "named arguments" in several places (e.g. (clojure.spec/fdef :args ... :ret ....)). Is there any convention as to when to use named arguments vs. a map? Maybe named arguments are primarily used for macros?
2016:06:16 11:10:10 pithyless Hi, all. Has someone come across the need to have an s/keys that is more strict (does not allow extra keys)?
2016:06:16 11:14:45 jrychter @pithyless: I have. I am guessing Rich thought it isn't a good idea, as in the long term it could hurt expandability and composability. But I still think we should be able to do it in some cases.
2016:06:16 11:22:42 pithyless I'm sure it's been brought up internally; I wonder how likely it would be to get an :only option added to s/keys.
2016:06:16 12:00:42 jrychter As I suspected — "spec embraces the idea of open maps". But there are cases where you don't want open maps. A good example is the current world of lein-cljsbuild, where I regularly pull my hair out because it isn't clear if a particular setting is in the right place. Extra keys do no harm, so people tend to put config options all over the place, and you end up with a terrible mess. Figwheel recently started doing some great work towards fixing this — and this kind of config checking should really be "these keys, and these keys only".
2016:06:16 12:02:38 pithyless I've got some half-baked solution with clojure.set/difference and keys, but I wish it were simpler.
2016:06:16 12:05:22 jrychter I am hoping someone will write a validations library similar to bouncer, based on clojure.spec.
2016:06:16 12:10:14 jannis Things are getting a little meta over here... I'm spec'ing out a data format and a parse function that first uses clojure.spec/conform to validate and normalize the input and then passes it on to a transformation function. For that transformation function the input data format is the output of conform, so I now have a spec for what conform returns as well. 🙂
2016:06:16 12:53:52 pithyless @ghadi: Thanks for linking to the archives. I understand why the default is what it is, my use case falls clearly in the "don't want to accidentally be leaking stuff" camp. But it's a minor thing... clojure.spec is nice 🙂
2016:06:16 13:16:37 dominicm @jrychter: As someone working on this: spec isn't suited to the task.
2016:06:16 13:17:25 jrychter @dominicm: oh 😞 I was hoping that one could build a validation system based on what explain-data returns…
2016:06:16 13:17:55 dominicm That's the easy bit.
2016:06:16 13:19:34 jrychter I currently use bouncer, but I have to work around its problems, and as I migrated from schema to spec, I was hoping I could get rid of it entirely.
2016:06:16 13:19:37 dominicm @jrychter: When you do user registration validation, you need to check email uniqueness. So that requires 2 things:
1. Data to validate, the email
2. Sideband data, the database (connection).
Spec has a funnel for 1. Not one for 2.
2016:06:16 13:20:37 ghadi spec has nothing to do with integrity checks (#2)
2016:06:16 13:21:38 dominicm Your only options for validating in that way, and neither of these are particularly spec-y, are dynamic vars, or the weird macro I started working on, and gave up on because it was a big ball of mud.
2016:06:16 13:21:44 jrychter As for (2) above, I currently use global state encapsulated using mount.
2016:06:16 13:22:06 dominicm @jrychter: You may not have as much trouble.
2016:06:16 13:22:23 dominicm However, I land on the weavejester side of the camp. 😛
2016:06:16 13:22:34 jrychter Meaning, some of my bouncer validations actually use database or even (gasp) network connections.
2016:06:16 13:22:45 dominicm Yep, it's a necessity.
2016:06:16 13:22:55 dominicm @ghadi: What kind of check does spec do?
2016:06:16 13:22:58 jrychter EU VAT numbers are an example.
2016:06:16 13:28:12 dominicm > weavejester side of the camp
I should clarify this means that I use component.
2016:06:16 13:48:10 dominicm @ghadi: I'm mostly interested in the terminology and reading on the subject.
2016:06:16 15:32:15 seancorfield As I recall, the discussion about closed maps ended with consensus that using select-keys was appropriate for "not leaking additional keys", but there wasn't much consensus on validation of closed maps.
2016:06:16 15:33:21 seancorfield I got the impression @richhickey is "considering" adding support for it due to repeated requests from the community but he doesn't consider it a good idea.
2016:06:16 15:37:33 seancorfield I think designing systems to simply ignore extra keys is more robust, but I also accept there are cases where that is difficult (the example I gave is clojure.java.jdbc/insert! where a map is converted to a SQL insert statement without reflection and therefore extra keys become extra columns and will be rejected by the database; the alternative is to introspect the database and ... well, do what? Silently ignore the extra columns? Throw a different exception?)..
2016:06:16 15:39:23 seancorfield And there in lies the issue: the behavior for extra keys is not knowable at that level - it would have to be an application-level decision (and the application has the means to do that reflection and call select-keys if it wants).
2016:06:16 15:41:26 seancorfield Which is why I side with Rich that it's not clojure.spec's job to directly support restricting keys, since it shouldn't be the norm, and you can do it via custom predicates already if you really have one of those odd cases where you really do need to check.
2016:06:16 15:42:22 donaldball Goes back to my point that if you want that level of validation, just specifying that the db arg to insert! is a Connection or a Connectable or whatever is not sufficient; it would need to specify that the database in question has such and such a schema
2016:06:16 15:44:06 seancorfield Right, and that's a whole different thing. java.jdbc provides metadata/reflection APIs but doesn't use them internally (for performance mostly),
2016:06:16 16:46:55 bhauman A strict key set constraint can make a lot sense depending on the situation. This depends on the api of course, but I can't imagine a higher level api, that third parties are going to rail against, not having tighter key set constraints.
2016:06:16 16:48:11 bhauman That being said, you can compose a strict map key set and description with spec fairly handily.
2016:06:16 16:51:55 bhauman Spec is beautiful.
2016:06:16 17:48:56 wilkerlucio @pithyless: if you really want to get just the strict keys from a map, I would suggest using select-keys as people mentioned before, as a plus, I wrote a simple utility that can extract the keys from a keys spec, it's still a naive implementation (it doens't consider req-un or opt-un) but can be starting point if you want to go further on the idea:
2016:06:16 17:52:24 wilkerlucio so I believe if extra keys are harmful on your case, you can use this kind of trick to remove unwanted ones, so we need to force the restriction of the keys, but just eliminate the ones you can't deal with (and only if you can't silently pass it on)
2016:06:16 17:59:48 seancorfield @wilkerlucio: The s/def for ::name and ::email don’t do anything in that example…?
2016:06:16 18:01:10 seancorfield Are you doing something else to check the values of the ::my-aggregate map conform to those specs, by convention (based on the key names)?
2016:06:16 18:01:51 wilkerlucio @seancorfield: the idea here was just to show an example of how to do a select-keys while reusing the specs, you are supposed to call s/valid? yourself before doing the strict-keys on this case
2016:06:16 18:02:44 seancorfield OK, so you’re relying on convention that the (qualified) key names are the same as the specs that apply to their respective values?
2016:06:16 18:03:02 wilkerlucio yes
2016:06:16 18:03:24 wilkerlucio but like I said, this is a naive implementation, if you really wanna rely on it, will need more work there
2016:06:16 18:04:33 seancorfield FYI, instead of (->> (s/form spec) (next) (partition 2) (map vec) (into {})) you could just do (->> (s/form spec) next (apply hash-map))
2016:06:16 18:05:03 wilkerlucio @seancorfield: thanks for that, I'll update the snippet 🙂
2016:06:16 18:11:42 bfabry has anyone come up with a good way of speccing non-keyword maps with known keyval pairs?
2016:06:16 18:30:46 zane Does spec just not play well with tools.namespace?
2016:06:16 18:33:27 bfabry instrument doesn't check :ret or :fn in the latest alpha
2016:06:16 18:33:28 bfabry only :args
2016:06:16 18:33:37 zane Ouch.
2016:06:16 18:33:41 zane Is that going to change?
2016:06:16 18:33:51 zane Did it check :ret and :fn in previous alphas?
2016:06:16 18:33:54 bfabry yes
2016:06:16 18:34:24 zane … Huh.
2016:06:16 18:34:32 bfabry the reasoning is that :ret and :fn are for checking whether the function is correct, which should happen when you're using the functions in clojure.spec.test. :args are for checking the function was invoked correctly
2016:06:16 18:38:15 zane So, we have a function that reads environment variables. That function has a :ret spec on it that validates that required environment variables are set and have valid values.
2016:06:16 18:38:58 zane My understanding now is that instrument will not help me here and I should use s/conform?
2016:06:16 18:40:04 bfabry yeah, s/conform or s/valid? or whatever explicitly. there's also an s/conform-ex coming iirc
2016:06:16 18:41:34 zane Where should I look for info on s/conform-ex?
2016:06:16 18:41:50 bfabry in a subsequent release 😆
2016:06:16 18:43:27 bfabry I'm sure it's coming soon, they've been evolving very rapidly
2016:06:16 18:44:38 zane Understood.
2016:06:16 18:44:53 zane Do you know what s/conform-ex going to do?
2016:06:16 18:45:10 bfabry conform or throw an exception on failure I assume
2016:06:16 18:54:05 leifp As mentioned before, I've (roughly) spec'ed a good chunk of clojure.core. I made that an actual repo in case someone wants to test and/or beautify them: https://github.com/leifp/spec-play
2016:06:16 18:57:40 zane @bfabry: When is the :ret argument to fspec ever used, then?
2016:06:16 18:58:22 bfabry when clojure.test.* functions run generative tests
2016:06:16 18:59:23 zane Got it. So only for generative testing.
2016:06:16 18:59:24 zane Oof.
2016:06:16 19:05:05 zane That seems like a very weird design choice to me.
2016:06:16 19:08:01 bfabry I'm not real sold on it either
2016:06:16 19:08:26 bfabry but I haven't used spec on anything big enough to be confident
2016:06:16 19:18:54 robert-stuttaford so, i'm new to test.check in general. anyone know how i might generate a set of keywords, from a known set of possible keywords?
2016:06:16 19:19:30 robert-stuttaford e.g. i have #{:a :b :c :d :e} and i want generated subsets of same
2016:06:16 19:23:03 bfabry anyway yeah s/exercise generates data that suits a spec, a spec of "sets of these keys" is (s/coll-of #{:keys} #{})
2016:06:16 19:23:38 robert-stuttaford ::types works, but it can't be generated because of how s/and generators work: generate for the first and discard anything that doesn't satisfy the rest of the ands
2016:06:16 19:23:42 bfabry I think you just want (s/coll-of ::type #{})
2016:06:16 19:24:36 robert-stuttaford indeed, thank you
2016:06:16 19:26:26 robert-stuttaford ok. my next question (which is the thing i really want to solve, now that i've softened you up 🙂 ) is how might i write a generator when i'm using s/and on two s/keys specs?
2016:06:16 19:27:09 bfabry I've not actually looked into generators sorry
2016:06:16 19:27:17 robert-stuttaford ah 🙂 worth a try!
2016:06:16 19:27:35 robert-stuttaford it's a super-interesting problem to solve
2016:06:16 19:27:38 robert-stuttaford to me, anyway
2016:06:16 19:47:39 wilkerlucio @robert-stuttaford: just a suggestion, since you are modeling something new, maybe would be better to use the namespaced keys instead of clear ones, with namespaced keys you can for example validate a map keys even if you don't know the aggregate name for it
2016:06:16 20:18:46 bhauman Spit-balling on strict keys just for the fun of it. I would love some feedback.
2016:06:16 20:27:21 leifp bhauman: What are the advantages of this vs. just (s/& (s/keys ...) #(only-these-keys % [:k ...])) ? Or a macro that expands into that.
2016:06:16 20:29:09 bhauman @leifp: the only interesting thing here is the explain data
2016:06:16 20:29:29 bhauman where it points exactly to the key that failed
2016:06:16 21:11:43 leifp bhauman: Hmm... I guess there is no explain equivalent of conformer or with-gen, so I can't really think of another way to do it than reifying Spec. Your impl. looks fine, but it doesn't seem to explain the extra keys if one of the required keys fails its spec.
2016:06:16 21:13:48 bhauman @leifp: I haven't looked at that, must be because of the s/and
2016:06:16 21:14:03 bhauman btw I've iterated on it a bit
2016:06:16 21:16:10 bhauman @leifp: what do you mean by extra keys? you mean in the explain data?
2016:06:16 21:23:32 leifp That output line was expected and not present, I mean.
2016:06:16 21:24:38 bhauman oh yeah it short cuts
2016:06:16 21:27:15 bhauman s/and short cuts
2016:06:16 21:27:29 bhauman so that makes sense
2016:06:16 21:33:47 bhauman I would need to compose over the keys-spec to get that behavior
2016:06:17 07:04:06 olivergeorge Hi Specy Specers. What's the most human friendly way of viewing the output from check-var? I see that reporter-fn can be provided, is there a commonly used one? (I'm using CLJS so perhaps that makes a difference).
2016:06:17 09:27:28 jrychter I find myself writing lots of (s/and string? seq) to specify non-empty strings. Also, I'm missing a predicate for strings of length from n to m (analogous to int-in-range?).
2016:06:17 09:34:58 jannis Is it possible that s/with-gen alters the spec it defines a generator for? I have a simple (s/cat :base ... :children ...) spec that works fine but as soon as I wrap it in (s/with-gen <spec> #(gen/tuple (s/gen ...) (s/gen ...))), data that would previously conform to the spec now becomes invalid.
2016:06:17 09:35:52 jannis Note: I', not generating the data using the generator yet. I'm using hand-written data.
2016:06:17 15:53:48 seancorfield @olivergeorge: there's a discussion about that on the main Clojure mailing list. I'm very interested in the answers to that question.
2016:06:17 16:37:31 leifp @jannis: It looks like it's introducing a new regex context like spec does:
2016:06:17 17:34:35 sparkofreason I'm working on some code that does simulations over state machines. The transition functions tend to have a lot of detailed conditional logic, and as a result test.check doesn't seem to be a great fit for validating the functions (random inputs tend to be ignored or lead to errors, and writing generators to provide valid input is essentially the same as writing the state machine model). When instrument-all checked the return value it was very useful, since I caught a lot of errors of omission, misspelled keywords, etc. I'd like to suggest we have an option to check :ret and :fn for cases like this.
2016:06:17 17:50:14 seancorfield Yeah, whilst I agree in principle with the justification @alexmiller offered as to why :ret and :fn checking was removed in Alpha 6, I also agree that there is potentially a lot of value in having the option to be able to instrument functions in a way that does conform the result at least.
2016:06:17 17:55:34 sparkofreason Thanks, I may use that to hammer out my own version of instrument for now.
2016:06:17 17:56:16 bfabry I think there's gotta be something coming for spec'ing impure functions that will cover this
2016:06:17 17:56:47 bfabry otherwise you could never spec them, really, or the spec would be pointless
2016:06:17 18:05:13 alexmiller @seancorfield: Rich has some ongoing work, I'm not sure what the endpoint will be on this. you might have noticed that explain-out was made public today in master
2016:06:17 18:06:25 alexmiller @bfabry: not every function is a great candidate for generative testing via spec (but the spec may still be useful for docs or other purposes)
2016:06:17 18:07:58 eggsyntax @alexmiller: where was the explanation @seancorfield mentioned of why :ret and :fn checking was removed? I’m curious to read it.
2016:06:17 18:09:19 alexmiller @seancorfield: I think Stu is looking at some testing-related mods too btw
2016:06:17 18:27:24 seancorfield re: explain-out — I already added a comment on that commit thanking him for that 🙂
2016:06:17 18:29:57 seancorfield @alexmiller: I really do appreciate the steady stream of alpha builds so we can all try this stuff out and provide feedback.
2016:06:17 18:30:46 alexmiller I'm sure there will be more :)
2016:06:17 18:34:34 seancorfield Having an option on instrument to use the old version of spec-checking-fn with :ret and :fn conforming would be very nice. I think checking just :args is the right choice for most cases of instrumentation, but I think being able to "fully instrument" certain functions would be very valuable — especially for functions that cannot easily be tested the generative way.
2016:06:17 18:36:05 seancorfield For example, working on java.jdbc’s specs, they can’t reasonably be tested generatively because many of them are side-effecting (updating the database) and writing generators that conformed to the database schema would be … a huge amount of work, if it’s even feasible (e.g., unique key constraints etc?).
2016:06:17 18:36:30 seancorfield So losing the ability to conform the :ret and :fn specs there is kind of a big deal, IMO.
2016:06:17 18:39:35 seancorfield I’ll be interested to see how this all ends up since any given code base is going to have a mix of functions that can reasonably be tested generatively and functions that can’t, so (clojure.spec.test/run-all-tests) needs a way to distinguish those, right?
2016:06:17 18:51:16 seancorfield (mind you, right now I can’t run generative testing on java.jdbc because the system doesn’t know how to generate a java.sql.Connection… which might be an interesting exercise 🙂 )
2016:06:17 18:52:24 seancorfield (and I’d also need a generator for a java.sql.PreparedStatement)
2016:06:17 18:53:11 seancorfield What is the recommendation for stuff like that? How would you even write a generator for some of these Java objects?
2016:06:17 19:34:36 tomc Would anyone be willing to offer some guidance on conventions for attributes shared by entities of different types? For instance, I have "question" and "survey" entities, each of which can have names. Right now I'm using (s/def :entity/name string?) and entities of either type can have an :entity/name attribute along with their type-specific attributes. The alternative of course is to define both :question/name and :survey/name. I haven't seen the :entity/attr pattern elsewhere, and I'm wondering whether there's a reason for that.
2016:06:17 20:03:39 wilkerlucio @tomc: I believe you can share the attribute as long as the semantic is the same, for example, a car name may have a different semantic from a person name, but that will depend on the requirements on your system
2016:06:17 20:09:32 tomc @wilkerlucio: thanks a lot, that's helpful.
2016:06:17 20:12:29 gphilipp I’m trying to generate dates within a range with spec, and I’m stucked with this piece of code which generates #inst whose year is above 9999, making them unrecognized when i try to def them manually afterwards : (gen/sample (s/gen inst?) 60)
2016:06:17 20:13:51 gphilipp ex of data generated: #inst"26138-06-03T15:09:43.670-00:00
2016:06:17 20:18:07 wilkerlucio @gphilipp: you can try something like this:
2016:06:17 20:18:58 wilkerlucio the first number, 100000 is a cap to limit the increment, and the 1465391285642 is the ms for a start date, adjust those to met your needs
2016:06:17 20:19:29 wilkerlucio ah, and this example is for CLJS, please change the Date initialisation if you are using on Clojure
2016:06:17 20:25:39 gphilipp @wilkerlucio: thx, I will try this
2016:06:17 20:28:41 leifp @gphilipp: There is also an clojure.spec/inst-in macro.
2016:06:17 20:28:54 seancorfield The default 100 tests for test.check can take a really long time on some fairly simple looking specs...
2016:06:17 20:29:38 seancorfield …running 50 tests wasn’t too bad but it seems to be taking more than linearly longer to do 5, 10, 25, 50, 100...
2016:06:17 20:31:10 leifp @seancorfield: Yeah, as well as being very dependent on ordering. The caveats about "generate-and-test" style vs. constraint satisfaction are in full effect here.
2016:06:17 20:32:00 seancorfield On the plus side, I figured out how to write generators for stuff like java.sql.Connection etc 🙂
2016:06:17 20:32:51 seancorfield And I also finally figured out how s/keys and the test.check integration hang together… which answered a question I’d asked someone else here before I understood what was going on...
2016:06:17 20:35:15 gphilipp Thanks @leifp, (gen/sample (s/gen (s/inst-in #inst "2016-01-01" #inst "2016-12-31")) 100) did the trick
2016:06:17 20:39:16 leifp @gphilipp: The end date is exclusive, remember.
2016:06:17 20:39:23 gphilipp ah, correct
2016:06:17 20:40:09 gphilipp I still wonder why (gen/sample (s/gen inst?) 60) generates invalid instants.
2016:06:17 20:44:05 leifp See? Generative testing is going to help us avoid the Year 26k Problem.
2016:06:17 21:10:41 seancorfield (and it took me a while to even get so far as to make run-all-tests actually start running… with-gen taking a 0-arity function that returns a generator is very counter-intuitive and I kept getting that wrong 😞 )
2016:06:17 21:57:20 leifp @seancorfield: The (s/* (s/or ...)) combo seems to be the culprit. It would probably be very fast if you could limit it to a max size. I don't know how to do that, though.
2016:06:17 21:59:42 seancorfield I guess I could always add a custom generator around it just to avoid that?
2016:06:17 21:59:50 seancorfield (and, thanks for the pointer on that!)
2016:06:17 22:18:28 leifp @seancorfield: This works: (s/def ::column-spec (s/with-gen (s/cat :spec (s/* (s/alt :kw keyword? :str string?))) #(g/vector (g/one-of [g/string g/keyword]) 0 6))) Still not what I'd call lightning-fast, but you can bound the generation time by decreasing the max. And note that g/ refers to clojure.test.check.generators, the equivalent using clojure.spec.gen didn't work.
2016:06:17 22:20:39 leifp Kind of clunky, though, and you'd need to do that everywhere you have (s/* (s/alt ...)). Maybe if enough people run into performance problems, rep will be made public, or the regex generators will be optimized.
2016:06:17 22:28:12 seancorfield Thanks @leifp
2016:06:17 23:58:39 leongrapenthin Assume I parse with conform. Then I have functions that operate on the value returned by conform. I can't get a spec for the value returned by conform (so that I can spec said functions) - It seems like this could be automated though. Imagine (s/conform-spec ::my-spec) would return the spec of the return value of calling (s/confom ::my-spec 42)
2016:06:18 00:08:39 wilkerlucio I noticed (s/exercice number?) can generate some NaN entries on cljs, is that by design?
2016:06:18 00:10:33 leongrapenthin So conform-spec as described above can apparently be implemented quite easily via (defn conform-spec [s] (s/and (s/conformer #(s/unform s %)) s))
2016:06:18 00:17:21 leongrapenthin But it would probably be more valuable if a spec could give a spec of what its conform* returns
2016:06:18 03:55:32 wildermuthn Is there a changelog for Spec? I’m looking at some of the recent commits to cljs.spec, and see that the docs have been updated. But would be great to know if there was another place to reference to see the updates.
2016:06:18 04:05:42 wildermuthn The more I learn about Spec, the more I think it is going to be regarded one of the essential Clojure features, on par with immutable data. For instance, I found out about conformer, and not ten minutes later had the beginning of a solution that I’ve never had for parsing JSON enums in a sane way. I shared this on #C03S1L9DN, but interested in getting feedback if I’m going at this the wrong way:
2016:06:18 08:37:45 vikeri A question about spec and generators, is it able to generate data from a spec directly or does it generate data and test it against the spec? As an example I suppose that a generator for string will only generate strings, but what if you have some slightly more complicated predicates like strings with the length 3? Will it understand to only generate strings with the length three? I guess this would need some way of inverting an if statement, which sounds difficult. The main question in hence: How ”smart” can the generator be?
2016:06:18 10:21:40 alexmiller Well it's not magic :)
2016:06:18 10:23:31 alexmiller If you have (s/and string? #(= (count %) 3)) the and generator generates based on the first pred, then filters based on the subsequent ones
2016:06:18 10:24:04 alexmiller So it will generate random strings and keep those that are length 3
2016:06:18 10:25:43 alexmiller That's probably a restrictive enough filter that you'll need to supply a custom generator
2016:06:18 10:28:40 alexmiller @wildermuthn: I list incremental changes for each release in the announcement notes in the Clojure mailing list
2016:06:18 10:29:28 alexmiller @wildermuthn: not clear to me what you're asking about with conformers above
2016:06:18 12:20:41 ghadi @alexmiller: s/every has a tiny typo in it causing this:
2016:06:18 14:32:11 nha I was expecting an error on the return value.
2016:06:18 14:33:14 minimal Instrument was changed to only check :args
2016:06:18 14:35:09 nha Ooh so how do I check the return value ? (ie. what would be a simple test here ?)
2016:06:18 14:37:29 nha Also is there a constraint on the order of things ?
Can I s/fdef before defn-ing a function ? Can I s/instrument-all when I want ?
2016:06:18 14:41:04 minimal For now you have to use the functions from the clojure.spec.test namespace like check-var
2016:06:18 14:43:48 nha Thanks I will dig into it. Those can do "normal" testing as well right ? ie. not generative testing
2016:06:18 14:43:51 minimal You can s/fdef before. The docs say the instrument fns are idempotent
2016:06:18 14:43:57 nha Thanks 😄
2016:06:18 14:44:29 nha Aah I read that but assumed it meant it can be called many times
2016:06:18 14:44:38 minimal Yeah
2016:06:18 14:46:16 nha Any reason the :ret has been removed ? Will it come back at some point ?
2016:06:18 15:03:31 nha So it looks like clojure.spec.test is only for generative testing, I still have to use whatever I was using if I want to explicitly give input arguments to the tested function (ex. for things too hard to generate). Correct ?
2016:06:18 15:17:17 gfredericks nha: sounds right
2016:06:18 20:56:07 arohner FYI, I’m reliably reproducing the ’no such var encore/bytes?’ thing that @seancorfield reported the other day
2016:06:18 20:56:27 arohner I don’t understand the cause, but I’m experiencing the same behavior
2016:06:18 21:08:24 seancorfield There's an updated Encore that fixes that. Some interaction with cljx / do / defn in that case - and conflicting with a (new) core function.
2016:06:18 21:09:02 seancorfield But I also eliminated every single dependency conflict and that solved the other, similar problems I had after that.
2016:06:18 21:09:19 arohner yeah 😞
2016:06:18 21:09:21 seancorfield So I'm not sure what the root cause is at this point.
2016:06:18 21:09:25 arohner I’m hoping the clojure bug gets fixed
2016:06:18 21:10:49 arohner is there a way to re-use fdef or fspec on other functions?
2016:06:18 21:10:57 arohner I’d like to say “these two defns have the same fnspec"
2016:06:18 21:14:15 arohner ah, looks like I can just (s/def ::name (s/fspec)) (s/def ::foo ::name)
2016:06:18 21:15:28 bbloom @arohner: if two functions have the same fspec, wouldn’t they just be the same function? (or your fspec contains no interesting properties?)
2016:06:18 21:16:13 arohner @bbloom no, there are lots of things that have the same spec that are interesting
2016:06:18 21:16:19 arohner i.e. simple math
2016:06:18 21:16:29 arohner +, *, - all take two numbers, return a number
2016:06:18 21:16:43 arohner in this case, parsing
2016:06:18 21:16:53 arohner take some input, return a parsed value and the rest of the stream
2016:06:18 21:17:22 bbloom seems like you’d want to reuse a spec for args and result, but a different spec for the property relating args to results
2016:06:18 21:17:46 bbloom there might be a missing abstraction here
2016:06:18 21:18:21 bbloom we’ve got specs for data and specs for functions which contain arg spec, return spec, and relation spec, but we don’t have a spec for args + return (signature?) without relation
2016:06:18 21:18:34 arohner yes, I’d like to get more specific in the future
2016:06:18 21:18:57 arohner and I still have an open challenge out to “correctly” spec clojure.core/map
2016:06:18 21:19:14 arohner I assert it can’t reasonably be done right now
2016:06:18 21:19:20 bbloom correctly = ?
2016:06:18 21:19:43 arohner ‘map takes a fn of type X -> Y, and a seq of X, and returns a seq of Y'
2016:06:18 21:20:22 bbloom can’t do context sensitive parsing
2016:06:18 21:21:42 eggsyntax @arohner: how could you verify that it’s a fn of type X -> Y in a language that doesn’t do type declaration?
2016:06:18 21:22:09 eggsyntax Is it sufficient to hand it an X, designate whatever comes out as Y, and then go from there?
2016:06:18 21:22:36 arohner well, I can spec that f is X->Y now, and spec that coll is coll-of X
2016:06:18 21:22:55 eggsyntax Ah, gotcha.
2016:06:18 21:23:10 eggsyntax Makes total sense; somehow I was assuming f was unspecced.
2016:06:18 21:23:18 arohner but that doesn’t narrow down map’s return spec
2016:06:18 21:23:34 arohner because map’s spec is just seq or whatever
2016:06:18 23:42:58 zpinter hello everyone! wondering if anybody knows of a clean way to define the spec for a map and its keys without having to repeat the list of keys again for the map spec
2016:06:18 23:43:27 zpinter my first attempt was something like this:
2016:06:18 23:44:55 zpinter however, every time I add a new key, I have to update two different places (which I'd ideally like to avoid in this case, since there'd be a lot of keys)
2016:06:18 23:45:47 bbloom zpinter: i’m far from an expert (i haven’t tried spec properly yet) but i think you can just use s/and, right?
2016:06:18 23:45:48 zpinter the recommendation from #C03S1L9DN was to make my own macro, which seems like a good approach, just wondering there was anything built-in
2016:06:18 23:47:15 bbloom oh, nvmd, i think i misunderstood what you were asking for
2016:06:18 23:49:21 bbloom but yeah, i’d guess making your own macro is the way to go - but depending on what you’re doing, the redundancy might make sense in order to allow decomposition later
2016:06:18 23:49:32 bbloom for example, you may want to limit the styles on some thing to only color related styles or only spacing styles or what not
2016:06:18 23:50:21 bbloom so you’d do (s/def ::padding…, (s/def ::padding-left ... and then group those with s/and to form ::padded
2016:06:18 23:50:43 bbloom then you can have (s/def style (s/and ::padded ::colored ::etc-etc-etc
2016:06:18 23:51:22 zpinter hmm... does s/and work with s/keys?
2016:06:18 23:51:49 zpinter I guess you could use s/and to combine multiple (s/keys :opt values....
2016:06:18 23:51:52 bbloom i’d be surprised if it doesnt, but like i said - i haven’t installed the alpha yet to try it 😛
2016:06:18 23:55:43 zpinter will keep investigating, thanks @bbloom
2016:06:18 23:56:07 eggsyntax @zpinter: I'd be interested in seeing what you come up with 🙂
2016:06:19 02:10:29 alexmiller You can compose any kind of spec with and
2016:06:19 02:11:47 alexmiller Note also that s/keys validates all keys in the map regardless of what's in req or opt
2016:06:19 02:12:07 alexmiller So you don't technically have to list them as opt keys
2016:06:19 10:25:11 akiel Has someone thought about specs for ex-data?
2016:06:19 13:57:56 cigitia In clojure.spec, is there a way to associate custom generators with your own functions, the way that clojure.spec.gen/gen-for-pred and gen-builtins associate generators with various clojure.core functions?
2016:06:19 15:00:13 gfredericks cigitia: you can make a spec that has a particular generator associated with it using spec/with-gen
2016:06:19 15:01:56 cigitia @gfredericks: Yes, though that requires either registering the spec returned by with-gen under a keyword, or using with-gen inline every time you use the function.
2016:06:19 15:02:07 cigitia I’m wondering whether it’s possible to have generators when using functions directly—
2016:06:19 15:02:35 cigitia Like how int? has a generator even when you use int? directly as a predicate, due to the way gen-for-pred works.
2016:06:19 15:16:09 gfredericks I think you can say (def int? (spec/with-gen #(instance? Integer %) gen/large-integer))
2016:06:19 15:16:14 gfredericks I might be lying
2016:06:19 15:59:45 alexmiller There is more stuff coming in this area soon
2016:06:19 16:16:18 akiel @gfredericks: is :throws discussed somewhere? I can’t find something about it.
2016:06:19 16:57:02 gfredericks akiel: no I just made it up
2016:06:19 16:57:30 gfredericks I was imagining an API that related to what you were asking about
2016:06:19 17:26:18 akiel @gfredericks: Yes that’s exactly what I’m after. @alexmiller What do you think about exception data specs? As far as I see it, exception data keys are not even documented normally.
2016:06:19 17:33:13 alexmiller We are not going to spec exceptions
2016:06:19 17:34:30 alexmiller You could of course create specs for the ex-info map keys and I can see that possibly useful in functions that receive ex-info data after an exception has happened (to generate user errors or error pages etc)
2016:06:19 17:37:51 akiel Do you think that the shape of the ex-data should be public API of a function or do you are more in line with Joshua Bloch: Effective Java Item 57: Use exceptions only for exceptional conditions were he suggests that exception should not be used for decision making.
2016:06:19 18:20:51 alexmiller Yes
2016:06:19 18:21:07 alexmiller Unless you're abusing it for performance :)
2016:06:19 19:24:53 akiel @alexmiller: sorry for asking further - yes public or yes not-public?
2016:06:19 19:30:48 hiredman not specing execeptions makes a lot of sense
2016:06:19 19:34:16 hiredman 1. spec is not a type system 2. spec is primarily about specifying data and exceptions are not data, they are weird control flow operation in a language 3. specs for functions specify what valid argument data looks like, and given that valid argument data, what a valid result looks like, and that is how the generative stuff works, and in that model there is no place for exceptions
2016:06:19 19:35:58 hiredman given a function F, if F throws E, to ensure correctness you want to look at all callers of F, and ensure that they handle E correctly, which is exactly the opposite of how the generative testing works (if I understand correctly)
2016:06:19 19:36:14 akiel @hiredman: in case API’s don’t use exceptions for control flow I’m with you. But than we don’t need ex-info and ex-data at all and should stop build API’s which put data into exceptions
2016:06:19 19:38:02 akiel s/conform is a good example for an API not using exceptions - it returns ::s/invalid
2016:06:19 19:40:40 bhauman Is there a way to validate that you have defined all namespaced keywords in the your spec definitions? And get a warning if some are missing?
2016:06:19 19:41:21 hiredman write a predicate that checks the spec registry
2016:06:19 19:46:01 bhauman hmmm ... so say compose a macro over spec/keys, capture the key args and and then warn if if the keys are not present in the registry? or were you thinking something else?
2016:06:19 19:47:03 bhauman if you wanted to verify everything with a single call though it seems like you would have to parse the describe of each of the registry members
2016:06:19 19:49:57 bhauman cool yep, but was thinking of a different use case, where you are inspecting the self referential integrity of the registry itself
2016:06:19 19:50:55 hiredman not sure I follow? that s/map-of spec will fail if given a map with namespaced keys that are not speced
2016:06:19 19:51:13 hiredman oh
2016:06:19 19:51:18 hiredman I think I get it now
2016:06:19 19:52:15 hiredman yeah, spec seems to be slightly later binding than clojure is
2016:06:19 19:53:11 bhauman yeah, it's doable, but I think it would be nice to have some help beyond describe
2016:06:19 19:54:19 hiredman you'd like ::foo to throw an error earlier then when you run the spec, basically, like clojure does with vars
2016:06:19 19:55:27 bhauman I'd like to do ( check-missing (clojure.spec/registry))
2016:06:19 19:56:39 bhauman to help me in the repl so that I can see if I forgot something
2016:06:19 19:58:20 hiredman that might not be currently possible, I don't think specs have a way to get all the specs they depend on
2016:06:19 19:58:36 hiredman which might be an interesting enhancement
2016:06:19 19:59:20 bhauman exactly, it would be nice to have a spec walker
2016:06:19 20:03:17 bhauman or at least a corollary to describe that returns the actual internal data
2016:06:20 02:05:06 alexmiller there will likely eventually be a deep-describe, which will require something like that
2016:06:20 13:54:51 alqvist I feel that the usefulness of s/inst-in is hampered by cljs incompability. Any words of wisdom?
2016:06:20 14:43:03 alqvist Or I could just use the latest clojurescript - Never mind
2016:06:20 15:04:32 angelini is there anyway to reproduce the behaviour of alpha-5’s s/instrument without using spec.test? The rational of only checking :args makes sense, but it was really useful when testing code at the REPL to verify that :ret and :fn we’re correct.
2016:06:20 15:32:18 adamfrey when you have a test.check generator that is failing a such-that predicate 100 times, is it possible to print out the failing values, just to make the generator less opaque? Nothing stood out to me in the source code for doing that.
2016:06:20 17:39:05 akiel Are there plans to support some kind of validation context? I think about a spec for JSON Web Tokens which are signed. Such a validation would need the public key as context. Or do I stretch the intended usage patterns here?
2016:06:20 19:41:30 alexmiller @adamfrey nothing explicit but you can gen/sample the s/gen of some simpler part to get an idea. It's usually the first pred in an s/and that is creating things that won't pass the later preds in the s/and
2016:06:20 19:41:49 alexmiller @akiel no plans for that
2016:06:20 19:42:17 alexmiller Prob a sign you are taking things too far :)
2016:06:20 20:10:02 akiel @alexmiller thanks
2016:06:20 20:38:54 alexmiller @cigitia: this channel is for spec, please take to #off-topic
2016:06:20 20:40:57 cigitia My apologies; accidentally sent to wrong Slack team; meant to sent to personal friends
2016:06:21 08:07:44 borkdude makes me wonder about the spec for get-in: (get-in {:a "lol"} nil) ;;=> {:a "lol"}
2016:06:21 08:08:53 borkdude I guess it could be correct if nil is regarded as the empty sequence
2016:06:21 11:55:12 akiel @borkdude: nil is the empty sequence 🙂 I think it should be possible to write a spec for get-in.
2016:06:21 14:33:25 jannis Is there any chance of opts being added to clojure.spec.test/run-(all-)tests? I have a couple of functions that exceed the GC limit if I run 100 tests against them. 25 or 50 is fine, so I'd like to set :num-tests globally instead of having to use check-var for every single function I want to test.
2016:06:21 14:35:10 jannis An alternative idea would be per-`fdef` opts perhaps?
2016:06:21 14:58:25 jannis Or a way to exclude specific functions from run-(all-)tests?
2016:06:21 15:06:29 akiel @jannis why are your functions are so memory hungry? do they have side effects or are the inputs to big? for the latter, I would rather optimize the generators.
2016:06:21 15:07:15 jannis Good point, it's probably the generators generating big inputs. I'll double-check.
2016:06:21 15:09:29 jannis Although... the generator is already generating relatively small data.
2016:06:21 15:10:19 angusiguess such-thats are a good thing to look out for.
2016:06:21 15:12:05 jannis I have overriden almost all generators because my data specs seem to be too complex to use the default generators.
2016:06:21 15:12:23 angusiguess One thing we ran into was the following:
2016:06:21 15:12:43 angusiguess If there's a significant distance between your generator and your validation predicate that generator tends to perform poorly.
2016:06:21 15:13:24 jannis Of course, that makes sense. That's why I overrode most of the generators so that they don't have to be tried a lot to have conforming data generated.
2016:06:21 15:15:24 angusiguess I haven't tried it, but does increasing the heap size help?
2016:06:21 15:16:52 jannis Without knowing much about it, GC limit exceeded to me sounds like it's generating too much data too quickly for GC to catch up. Let me re-run the tests with a heap size of 2GB.
2016:06:21 15:17:58 angusiguess GC Limit exceeded usually means that the ratio of GC pauses to 'useful' computation is quite high.
2016:06:21 15:18:03 angusiguess So more heap space can reduce pauses.
2016:06:21 15:18:17 jannis Ok
2016:06:21 15:23:18 jannis It's also a test performance issue by the way. With 100 tests, that single function test runs for ~5-10 minutes until it errors out with the GC limit. I'm aiming for a fast test suite, so being able to reduce the number of tests globally - or for this function - would help.
2016:06:21 15:56:46 jannis Increasing the heap size avoids it failing after 5-10 minutes. Instead it has now been running for more than half an hour. 😉
2016:06:21 15:58:07 akiel @jannis I have you source code open. But I never used boot before and I’m using cursive - so its not so easy for me to get it running.
2016:06:21 15:58:49 jannis @akiel: Once you have boot installed, all you should need is boot test-once
2016:06:21 16:20:11 akiel @jannis: the generator of ::property-spec is already very slow. Try (s/exercise ::properties-spec 15) and than with 20. The thing is that generated samples become bigger and bigger if you generate more. test.check has some internal size thing. You have to improve the generator.
2016:06:21 16:21:11 jannis @akiel: Cool, thanks for investigating. I'll see what I can do. 🙂
2016:06:21 21:15:20 cigitia Is there any particular reason that the sequence regex ops do not recognize strings as seqables of chars? For instance, ”ab” does not conform to (s/cat :a #(= % \a), :b #(= % \b)), while [\a \b] does.
2016:06:21 21:19:52 cigitia In addition, are there plans for negative-lookahead regex ops, like those from &? For instance, in PEGs / parsing expression grammars, & has a negative counterpart called ! .
In particular, an end-of-sequence regex op would be very useful—it could be called ::end, and it could be equivalent to (! ::any). As far as I can tell, this isn’t currently possible.
2016:06:21 22:25:41 bfabry lol, @cigitia your slack-fu is struggling
2016:06:21 22:25:42 alexmiller @cigitia: you're posting news links again
2016:06:21 22:25:57 cigitia Argh, sorry
2016:06:21 22:26:33 alexmiller @cigitia There are good tools for string regex. Spec is never going to be great for that and there are no plans to use it for that
2016:06:21 22:27:07 alexmiller No plans for negative look ahead ops
2016:06:21 22:27:17 cigitia Okay, thanks
2016:06:21 22:27:38 alexmiller They are generally not needed for the kinds of things we expect you to do with spec regex
2016:06:21 22:32:30 cigitia I had actually been working on a PEG library that could create grammars for generic EDN data, including but not limited to strings, before Spec was announced
2016:06:21 22:34:43 cigitia Once Spec was announced, it seemed like there wasn't much room for such a library anymore, and so I thought about instead making a library that would utilize specs and convert them into parser transducers
2016:06:21 22:35:25 cigitia Maybe there's still room for the first idea, though, then, hm
2016:06:22 02:59:51 olivergeorge I was expecting s/conform to return fully "conformed" data but it doesn't seem to work like that.
2016:06:22 03:00:00 olivergeorge
=> (s/conform (s/coll-of (s/conformer (fn [x] (println "I am conforming" x) (str x))) []) ["1" 2 :3])
I am conforming 1
I am conforming 2
I am conforming :3
["1" 2 :3]
2016:06:22 03:01:04 olivergeorge it uses the conformer in order to check the data but returns the un-conformed data. Seems to be "non-recursive"
2016:06:22 03:02:43 olivergeorge Is that intended behaviour? Perhaps I'm attempting to use it incorrectly - as a data verification & transformation tool.
2016:06:22 03:40:34 alexmiller The problem here is that coll-of samples its contents and doesn't flow the entire contents
2016:06:22 03:40:53 alexmiller There are some things in this area changing in next alpha
2016:06:22 03:57:48 olivergeorge Thanks Alex, I think I see similar behaviour with map-of.
2016:06:22 04:02:11 alexmiller Yes
2016:06:22 04:02:44 alexmiller every and every-kv are new things already in master but not yet released
2016:06:22 04:03:29 alexmiller One of those sets will conform all args, although I'm not sure which
2016:06:22 04:08:31 olivergeorge Perfect. Thanks.
2016:06:22 11:38:53 akiel Is there a way to spec values which will be conveyed over a channel or returned by a promise?
2016:06:22 11:53:39 alexmiller nothing built in yet
2016:06:22 11:54:19 alexmiller ultimately I expect there will be some things provided for core.async
2016:06:22 11:55:09 alexmiller you could supply a map transducer to an async channel that called s/conform or something like that though
2016:06:22 12:09:50 akiel some kind of spec wrapper which takes a spec and validates channel values on the go would be nice
2016:06:22 12:11:11 akiel I only like to be sure that nothing speaks at this conceptionally before spec gets final
2016:06:22 14:02:18 eggsyntax test.check has a fn (`fmap`) that lets you call an arbitrary fn on the results of a generator. But is there any way to create a generator that just calls an arbitrary fn? I haven't found one.
The only solution I've found so far is to do (fmap (fn [_] do-what-I-want) some-arbitrary-gen), but it's pretty ugly to put in a generator and then ignore what it generates.
2016:06:22 14:04:04 eggsyntax (I understand that shouldn't be a typical use, but it'd be good to have an escape hatch for cases where you have an existing source of randomness that you don't want to have to duplicate)
2016:06:22 14:43:53 akiel @eggsyntax: As you said already, it’s not the typical use to implement the root generator yourself. You have to understand that test.check needs to control the generated values in a way to get shrinking work. So I have no answer to your question only the suggestion: Don’t do this! 🙂
2016:06:22 14:44:49 eggsyntax Heh, fair enough.
2016:06:22 14:46:25 eggsyntax Although in this case I'm still gonna do it...again, existing (complex) source of domain-specific randomness.
2016:06:22 14:46:53 eggsyntax Maybe I'll have to figure out how to make a Rose tree from it...
2016:06:22 14:47:19 eggsyntax I realize there's a likely cost in terms of shrinking, though.
2016:06:22 18:07:43 alexmiller @eggsyntax: have you looked at return ?
2016:06:22 19:22:49 eggsyntax @alexmiller: I have, but it looks like that'll only generate a constant value, right? I need something that'll call a fn every time it needs a new one.
2016:06:22 19:39:02 angusiguess test.check does have a no-shrink qualifier too I believe.
2016:06:22 19:39:23 angusiguess If the value doesn't benefit from shrinking (like uuids for ex)
2016:06:22 23:06:16 bostonaholic has anyone come up with a reasonable clojure.spec/def for a database argument?
2016:06:22 23:06:27 bostonaholic all I’m using at the moment is ::s/any
2016:06:22 23:07:29 bostonaholic but obviously that won’t work with generators
2016:06:22 23:08:38 bostonaholic I’m not sure there’s much value here, but interested in hearing what a possible solution might be
2016:06:22 23:40:25 eggsyntax @bostonaholic: what should the DB argument look like?
2016:06:22 23:41:04 bostonaholic well, it’s a datomic db
2016:06:22 23:42:01 eggsyntax Ah, ok, you hadn’t mentioned it was datomic.
2016:06:22 23:42:10 bostonaholic yeah, sorry
2016:06:22 23:42:31 bostonaholic that’s why I’m thinking this isn’t a good idea
2016:06:22 23:44:59 olivergeorge @alexmiller picking up on my question about fully conforming yesterday. I tried every-kv this morning (cljs release) and I see the same behaviour:
with further specs for those keys.
(don't have a datomic-using project right in front of me, I'm improvising on the type, and assuming that it's a record)
2016:06:22 23:50:24 eggsyntax (which could be dead wrong 🙂 )
2016:06:22 23:51:34 bostonaholic you’re right
2016:06:22 23:56:03 alexmiller @olivergeorge its not done yet
2016:06:22 23:58:38 alexmiller @olivergeorge: I believe when it is done map-of will be what you want
2016:06:23 00:01:18 olivergeorge Thanks again Alex. I suspected that might be the case.
2016:06:23 01:08:29 eggsyntax @alexmiller: is there a roadmap online anywhere for what changes are definitely coming? Or even something that discusses some of them?
2016:06:23 01:34:38 alexmiller no
2016:06:23 01:34:54 eggsyntax Oh well 🙂
2016:06:23 01:35:06 alexmiller there are definitely changes around instrument and the c.s.test namespace coming soon
2016:06:23 01:35:15 alexmiller and map-of and coll-of work is under way
2016:06:23 01:35:33 alexmiller there are a bunch of other things on the list but I don’t how that will all pan out
2016:06:23 01:36:14 eggsyntax Excellent! I don’t think I’ve actually said this to y’all before about spec, but thanks 🙂. I’ve been working with it for about a week, and it’s just terrific work.
2016:06:23 01:36:35 eggsyntax Just how powerful it is didn’t even fully sink in until I’d dived into it for a while.
2016:06:23 01:36:41 eggsyntax ^ @alexmiller
2016:06:23 01:39:18 seancorfield If I have (s/keys :req-un [::foo ::bar]), am I correct that something in clojure.spec will expect ::foo and ::bar to exist as specs, for some operations? I was trying to add stuff to clojure.java.jdbc.spec to support generative testing (by writing custom generators for db-spec, java.sql.Connection, and java.sql.PreparedStatement etc) and I started getting errors that the "unqualified" keys in some of my specs didn’t resolve...
2016:06:23 01:40:43 seancorfield That wasn’t obvious to me from the documentation about s/keys and the -un keys in maps — but it makes sense from the p.o.v. of actually trying to generate maps to pass into functions whose arguments are spec’d as s/keys...
2016:06:23 02:31:31 alexmiller if you’re trying to gen keys in a map, then yes you would need them to have a definition
2016:06:23 02:31:49 alexmiller otherwise how would you gen them?
2016:06:23 02:33:01 alexmiller are you getting this while gen’ing or during something else?
2016:06:23 03:01:50 seancorfield During gen’ing. And, yes, it makes sense. It just wasn’t obvious from the docs...
2016:06:23 03:02:37 seancorfield Because I missed this (very important) phrase "These variants specify namespaced keys used to find their specification"
2016:06:23 03:04:13 seancorfield I just didn’t associate the ::first-name etc with the specs given two examples above since they were "unqualified keywords" in my mind.
2016:06:23 03:06:16 seancorfield Not sure what to suggest to make that clearer…? Perhaps repeat the specs in the :unq/person example?
2016:06:23 03:06:39 seancorfield (and now I notice that ::phone is not provided as a spec anywhere on that page)
2016:06:23 03:07:19 seancorfield Obvious in hindsight, of course.
2016:06:23 08:34:34 olivergeorge This is a thought in passing. Often we want confidence that some data is fully realized. It seems like a challenging case. We have realized? as a test and could write a recursive ::realized spec. It is challenging since displaying the data in reporting might "realize" it. So the test modifies the data.
2016:06:23 08:35:11 olivergeorge Perhaps not a clojure.spec domain issue... data is valid, it just doesn't exist yet!
2016:06:23 08:35:51 olivergeorge We've seen cases where lazy and dynamic aspects cause surprises. Right now we're considering a class of problems where delayed evaluation makes debugging harder.
2016:06:23 08:36:30 olivergeorge Just raising in case it's something of interest as spec matures.
2016:06:23 08:48:26 thomasdeutsch how can i spec this structure?
2016:06:23 21:41:06 alexmiller The explain-data will tell you what it’s missing in the form of predicates but that’s not what you’re asking
2016:06:23 21:41:51 alexmiller (btw, as of the next alpha, you’ll be able to rewrite pickup-gt-delivery as (fn [{:bill/keys [delivery pickup]}] (> pickup delivery)) )
2016:06:23 22:07:58 uwo @alexmiller: cool. thanks for the response!
2016:06:24 02:22:56 bsima How do you spec protocol functions? Is it the same as spec-ing a regular function? (I haven't actually tried it yet...)
2016:06:24 02:23:42 bsima I tried to spec a multimethod using fdef a few days ago and it didn't work but I didn't have time to figure out why
2016:06:24 02:37:33 seancorfield @bsima: Read this issue for some information about spec and protocols http://dev.clojure.org/jira/browse/CLJ-1941 (regarding instrumentation)
2016:06:24 02:42:25 bsima mm thanks seancorfield
2016:06:24 15:11:37 jannis How do I best write a generator for :args? It seems like even if I use (gen/cat <sequence-generating generators>) it generates a sequence that is then passed to :args as the first argument, instead of "splicing" it into the macro/function call.
2016:06:24 15:13:37 akiel @jannis Is it not sufficient for you to write individual generators? Do you have many dependencies between your args?
2016:06:24 15:17:21 jannis The args are fairly flexible and somehow what gets generated for name and & forms is not quite what it should be. Perhaps my :args spec is wrong (thinking var args perhaps need special treatment)?
2016:06:24 15:21:21 jannis Even if I just use (s/cat :name ::command-name) and drop the & forms argument in the macro, check-var complains with "Wrong number of args (1) passed to: command/defcommand". Something there is odd.
2016:06:24 15:22:41 akiel Yes (s/cat :name ::command-name) generates only one argument
2016:06:24 15:23:22 akiel varagrs is no special - you have already a s/* in your args
2016:06:24 15:23:42 akiel what is ::command-name?
2016:06:24 15:23:47 jannis But one argument should be fine for a macro that takes exactly one argument.
2016:06:24 15:24:17 jannis It's a symbol? spec
2016:06:24 15:24:59 akiel ah yes you are right - one arg should be possible
2016:06:24 15:25:05 jannis I can paste the entire file once my laptop has rebooted.
2016:06:24 15:30:18 akiel I get the same wrong number of args message with check-var
2016:06:24 15:30:37 akiel but if I convert the macro into a function, it works
2016:06:24 15:30:52 akiel ther may be some issues using char-var with macros
2016:06:24 15:31:02 akiel I never spec’ed macros before
2016:06:24 15:31:11 jannis Yep, I never ran into this with specs for functions.
2016:06:24 15:32:00 jannis Actually, this is a Clojure + ClojureScript macro, which will just pass on the arguments to a function that does the actual work. I could move the spec to that function instead.
2016:06:24 15:32:14 jannis That could work around the problem.
2016:06:24 15:34:21 akiel but speccing macros is an interesting feature on its own - but check-var might simply not work with it
2016:06:24 15:35:18 jannis Could be?
2016:06:24 15:36:50 akiel I simply don’t know. I’ll hit this if a spec my first macro. But I have currently none.
2016:06:24 15:36:55 jannis Ok 🙂
2016:06:24 15:37:40 jannis Moving the spec to the function and turning the var-args into a (s/cat :name ... :forms (s/spec (s/cat :description ...))) subspec structure does the job 🙂
2016:06:24 15:37:46 jannis All the generated inputs are now useful.
2016:06:24 15:40:40 akiel nice 🙂
2016:06:24 16:00:28 kendall.buchanan Question about the direction of namespaced keywords:
2016:06:24 16:01:12 kendall.buchanan With Clojurescript (Om appears to be pushing namespaced keywords) on the front end, specs in the middle, and Datomic on the end, namespaced keywords feel natural.
2016:06:24 16:01:35 kendall.buchanan But if you’re working with a Javascript front-end, that doesn’t support them. And a SQL database on the other. What’s one to do in the middle if you want to leverage specs?
2016:06:24 16:01:49 kendall.buchanan Just :req-un everything?
2016:06:24 16:02:09 kendall.buchanan Or create translation layers across the board?
2016:06:24 16:02:49 akiel :req-un works fine - the only thing you loose is automatic validation of all keys even if not present in req or opt
2016:06:24 16:04:38 kendall.buchanan Makes sense. I’ve hesitated on spec for fear that they can’t live in “the real world” (a non-full-Clojure stack environment).
2016:06:24 16:05:46 akiel even in clojure itself - nobody can change existing apis - so un-namespaced keywords will be still there
2016:06:24 17:49:16 seancorfield @kendall.buchanan: I’m working on making namespaced keyword support in java.jdbc a lot easier. In the upcoming 0.6.2 release you’ll be able to specify a :qualifier "foo" option on calls to get :foo/col-name back from all queries. I need to figure out the final story for what happens if you push :foo/col-name into java.jdbc.
2016:06:24 17:50:31 kendall.buchanan @seancorfield: That’s very cool. I suspect work will need to be done on all ends of the stack to make namespaced keys more tenable for the average project.
2016:06:24 17:56:36 seancorfield Now I look at it, I probably won’t need to do anything for pushing namespaced columns into java.jdbc:
2016:06:24 18:12:23 alexmiller @jannis: check-var (very soon to be just “test” in the next alpha) is not going to work with macros
2016:06:24 18:14:42 alexmiller @kendall.buchanan: we are considering adding something in s/keys that lets you alias unaliased keys to arbitrary specs, but no guarantees
2016:06:24 18:19:23 jannis @alexmiller: Will there be an equivalent for fdef'ed macros?
2016:06:24 18:19:34 alexmiller you can fdef macros now
2016:06:24 18:20:18 alexmiller it’s just that check-var is not going to work to test them, afaik
2016:06:24 18:23:56 akiel Is there a way to compose multiple s/key specs into one big one so that a map has to satisfy them all?
2016:06:24 18:24:38 bfabry @akiel: I wrote a macro that did that because I couldn't find anything. it was kinda painful
2016:06:24 18:26:06 akiel @bfabry: I need it to work with multi-spec because I have a big environment map were I have multiple subsets of varing keysets.
2016:06:24 18:30:41 akiel Ah s/form is nice. Never used it.
2016:06:24 18:34:05 akiel @bfabry: Thanks but your macro will only work with static s/keys specs.
2016:06:24 18:34:23 bfabry yup, I couldn't find a way around that because s/keys is a macro
2016:06:24 18:39:12 alexmiller @akiel you can just do s/and of multiple s/keys
2016:06:24 18:39:40 bfabry @alexmiller: that won't create good generators though
2016:06:24 18:39:59 alexmiller true
2016:06:24 18:41:02 akiel @alexmiller: Ok I only have to wrap all subsequent s/keys into a one-arg function and ignore the conforming value?
2016:06:24 18:48:23 akiel @alexmiller: ok it works
2016:06:24 18:48:24 alexmiller s/and will create a spec, not sure why you need to make it into a function
2016:06:24 18:48:46 akiel No my reading of the doc was just to loose.
2016:06:24 18:48:50 alexmiller the resulting conformed value should be the map
2016:06:24 18:49:32 akiel I thought there would be always a predicate which gets the previous value. Bit this counts only for preds and not for specs.
2016:06:24 18:51:34 alexmiller not sure I understood that
2016:06:24 18:54:46 akiel The example is (s/and even? #(< % 42)) and the doc says: “Successive conformed values propagate through rest of predicates.” So I thought you can have one initial spec and than only predicates which are called with the previous conformed value.
2016:06:24 18:54:50 bfabry @alexmiller: out of interest is there any move to make some of these macros functions? a dynamic s/keys seems like it could be useful
2016:06:24 18:55:53 alexmiller @akiel: yes, although I think maybe you’re reading too much into the choice of “spec” and “predicate” there
2016:06:24 18:56:40 alexmiller and combines specs which can be either predicates or other things
2016:06:24 18:56:47 alexmiller the conformed value flows through them
2016:06:24 18:57:38 akiel @alexmiller Ah ok. I see. The specs get also the previous conformed value and just use it to be able to return the whole thing at the end.
2016:06:24 18:58:50 alexmiller @bfabry: yes, that’s possible. things are macros so that we capture forms for reporting purposes. but there may still be function forms as well (which would fill in under the macro forms)
2016:06:24 18:59:47 bfabry cool cool
2016:06:24 19:19:46 fenton I have a multi-spec. My in data can look like: {:method :search :search-term "blah" :auth "abc123" :gps-lat 3.4 :gps-long 5.3} or {:method :get-products :product-id 12345 :auth "abc123" :gps-lat 3.4 :gps-long 5.3}. My multi-spec already keys off the :method key. My data has some shared keys: :auth, :gps-lat, :gps-long and some keys that are not shared: :search-term and :product-id. Ideally I'd like to create a spec that merges two s/keys specs into a single s/keys spec, however s/cat creates a spec that expects two hashes in a list as opposed to a single hash with merged keys.
2016:06:24 19:21:24 alexmiller I talked through some options for something like this with @luke - maybe he could elaborate on where he ended up
2016:06:24 19:24:32 alexmiller iirc, he ended up with a spec for the common keys and separate specs for each variant
2016:06:24 19:24:57 alexmiller and then specs for each variant that anded common and separate
2016:06:24 19:25:54 alexmiller and then instead of multi-spec, it’s an s/or of the concrete variants
2016:06:24 19:26:05 alexmiller with the downside that it’s a closed system
2016:06:24 19:26:46 alexmiller with multi-spec, you’d have to repeat the keys in each variant
2016:06:24 19:28:40 fenton @alexmiller: okay, i'll look into what it looks like with s/or option and compare that to repeating in a multi-spec scenario. Thanks.
2016:06:24 19:29:46 fenton the closed system you mention doesn't use s/keys?
2016:06:24 19:41:52 alexmiller no it would
2016:06:24 19:43:31 alexmiller common = (s/keys …)
A = (s/and common (s/keys A…))
B = (s/and common (s/keys B…))
all = (s/or :a A :b B)
2016:06:24 19:44:05 alexmiller that doesn’t help you with the conforming question though
2016:06:24 19:45:23 fenton @alexmiller: k i'll give that a try.
2016:06:24 23:52:44 leongrapenthin @fenton @alexmiller I ran into this too, it seems a common scenario with Datomic "polymorphism" aka {:payment/amount 42, :payment/type :payment.type/stripe, :payment.stripe/payment-id 1234} - it would be great to have a merge-keys operation that takes two s/keys and merges their implementations - what do you think?
2016:06:24 23:55:33 alexmiller Talked about it with Rich today
2016:06:24 23:55:41 alexmiller We'll see
2016:06:25 01:28:41 seancorfield Alpha 8 is looking like a great release, based on the commits over the last few days!
2016:06:25 02:14:53 alexmiller Gonna be a big one :)
2016:06:25 02:32:05 seancorfield Any likely ETA @alexmiller ?
2016:06:25 02:46:46 alexmiller Soon
2016:06:25 02:47:22 alexmiller Didn't quite finish everything today as expected so prob early next week
2016:06:25 05:29:52 bbloom rather than complain about having to upgrade fipp for new reader/printer forms, i’ll instead boast: open source contribution opportunities! clojure 1.9 functionality PRs welcome 😉
2016:06:25 18:26:15 fenton @alexmiller: small typo in spec guide. the description around function specs uses an int? predicate instead of integer?
2016:06:25 18:30:04 alexmiller That's correct as is
2016:06:25 18:30:50 fenton oh, hmm, couldn't find an int? predicate
2016:06:25 18:31:05 alexmiller int? is a new predicate in 1.9 that matches any fixed precision integer
2016:06:25 18:31:23 gfredericks so in particular not bigints?
2016:06:25 18:31:27 fenton ah, good to know.
2016:06:25 18:31:29 alexmiller It was briefly called long?
2016:06:25 18:31:47 gfredericks bigints are the only difference between int? and integer??
2016:06:25 18:32:45 bronsa AFAICT yes
2016:06:25 18:33:08 gfredericks interesting
2016:06:25 18:33:39 gfredericks I wonder why that's considered more useful than the other way
2016:06:25 18:34:57 bronsa well there are a number of constructs in clojure that don't work with idxs bigger than Long/MAX_VALUE so I'm guessing int? was provided to cover those use-cases in specs
2016:06:25 18:35:31 bronsa guarding against i.e. (range 0 (inc Long/MAX_VALUE))
2016:06:25 18:35:43 gfredericks interesting
2016:06:25 18:36:20 bronsa (although there are also other constructs that don't work with idx bigger than Integer/MAX_VALUE like dotimes IIRC?)
2016:06:25 18:37:43 bronsa ah, I'm thinking of for and doseq, they're constrained by Collection.count returning int
2016:06:25 19:08:06 alexmiller Yes widening from just long to other fixed precision was the reason
2016:06:25 19:08:43 alexmiller In particular to capture int
2016:06:25 19:11:08 alexmiller Conceptually we would prefer if users primarily focus not on the Java types but on integer fixed vs integer arbitrary precision as the main "types"
2016:06:25 19:40:53 mathias_dw probably a dumb question, but I’m having to call s/instrument-all every time I make a change to the code or the specs. Is there a way of automating that, or is my workflow simply wrong?
2016:06:25 19:42:21 mathias_dw (or is it maybe some nrepl/cider-related thing?)
2016:06:25 19:47:08 alexmiller You shouldn't have to do that
2016:06:25 19:47:52 mathias_dw ok, thanks. I’ll try without the cider/nrepl stuff
2016:06:25 19:48:19 alexmiller That was true of the very first release but was fixed early on
2016:06:25 19:48:43 mathias_dw I’m using alpha5
2016:06:25 19:49:20 alexmiller Yeah shouldn't be a problem
2016:06:25 19:49:35 alexmiller Must be something in the workflow
2016:06:25 19:49:51 mathias_dw ok, thanks. Will try to pinpoint it and raise it in the relevant place
2016:06:25 20:45:14 gfredericks I'm having trouble with a recursive spec on master; not a problem on alpha7: https://www.refheap.com/120803
2016:06:25 20:45:33 gfredericks I scanned the commit messages and didn't see anything suspicious
2016:06:25 20:46:30 gfredericks if I had to guess I'd say it's something to do with coll-of suggesting that there's ambiguity
2016:06:25 20:48:48 alexmiller That code is brand new and on my plate to test more on Monday morning so I will def look at it
2016:06:25 20:49:28 gfredericks cool, thanks
2016:06:26 07:03:20 seantempesta is there a way to test if a spec exists for a namespaced key?
2016:06:26 11:05:18 alexmiller There is a 'registry' function
2016:06:26 11:05:55 alexmiller In master for the next alpha there will be 'get-spec'
2016:06:26 12:41:01 jannis Neat
2016:06:26 18:35:23 fenton I have defined an api spec for front end to communicate with backend. The messages include an auth token that I strip off soon and forward the rest of the message to internal handling functions. It would be cool to be able to create a spec on my backend that takes a spec defined for the API and removes aspects of it, like for example the auth-token key of the map. I guess the generic manipulating of a spec like a regular hash or something would be interesting.
2016:06:26 18:36:46 fenton otherwise i have to duplicate the definition of the message....trying to keep it DRY.
2016:06:26 18:55:41 seancorfield @fenton: why not define the spec once for the back end, then define the spec for the front end as "back end" and "auth token"?
2016:06:26 18:57:22 fenton @seancorfield: I've pulled my api spec into a third project that is *.cljc with reader conditionals so it can be included as a library for both front and backend. I don't think I want to pull in my backend as a library to my front end....
2016:06:26 18:58:19 fenton not sure if thats the best way to do it tho?
2016:06:26 18:58:57 seancorfield Just the spec for the shared portion is what I'm suggesting.
2016:06:26 18:59:40 fenton ahh that makes sense
2016:06:26 18:59:43 seancorfield So if your API and your back end traffic in a common subset of data, that spec needs to be external to both as a shared spec.
2016:06:26 19:00:09 fenton yeah, i like that...thanks for the idea!
2016:06:26 19:02:02 fenton hmmm...might run into same problem about how to make a spec like that... can't merge a spec map.
2016:06:26 19:02:12 fenton map spec.
2016:06:26 19:03:57 seancorfield You use s/and to specify the API must conform to common spec and also to the auth token spec. You don't merge the maps.
2016:06:26 19:04:26 seancorfield Alex talked about that technique here a day or two ago.
2016:06:26 19:05:25 fenton that one?
2016:06:26 19:06:11 seancorfield I'm on my phone so I can't dig into code.
2016:06:26 19:07:25 fenton i didn't try that approach, but wondering if that doesn't make data that looks like: [{:auth-token "abc123"} {:method :search :search-term "blah"}]
2016:06:26 19:08:11 fenton which is okay...i guess...
2016:06:26 19:10:18 seancorfield Since maps are open, saying some data conforms to multiple map specs should still leave you with one map?
2016:06:26 19:11:58 seancorfield (Replete doesn't seem to have clojure.spec or I could try that out)
2016:06:26 19:12:03 fenton hmmm, not sure, making the (s/keys :req ... might be a problem the :req part...meaning each needs all?
2016:06:26 19:12:31 mfikes @seancorfield: It does, but it is under cljs.spec and slightly older at this point.
2016:06:26 19:14:57 seancorfield @fenton: if A needs key X and it needs key Y then that is additive and it needs keys X and Y - which is a merge and therefore what you want.
2016:06:26 19:15:41 gfredericks is (s/and (s/keys ...) (s/keys ...)) going to be special-cased to aid things like generation?
2016:06:27 17:09:51 angusiguess Args is always assumed to be a list of 0 or more arguments
2016:06:27 17:10:14 angusiguess Or rather, a spec for a list of 0 or more arguments
2016:06:27 22:43:28 bhauman So I'd like to do some spec work in an environment that needs to run on an earlier version of Clojure. I'd like to conditionally include the clojure.spec library if the Clojure version isn't 1.9 and if clojure.spec isn't already loaded. Is there an established smart pattern for doing this? Say requiring a library that does a conditional load?
2016:06:27 23:08:19 seancorfield I don’t know how "established" it is, but I did this for java.jdbc’s test namespace...
2016:06:27 23:41:59 bhauman @seancorfield: thanks! I appreciate it. I'll take a gander :)
2016:06:28 03:32:53 olivergeorge New favourite spec trick. Wrap your spec to generate warnings instead of erroring out.
(defn warn
"Throw warning if spec is invalid but always pass."
[spec]
(fn [x] (when-not (s/valid? spec x)
(js/console.warn (s/explain-str spec x) x))
x))
2016:06:28 06:52:23 olivergeorge And now not. Turns out the stack trace from normal errors is more informative (cljs).
2016:06:28 08:56:35 rauh I have within one ns two different specs for :db/id (one for data coming from datascript and one for datomic ids). How can I specify this in spec?
2016:06:28 12:03:28 alexmiller There is new support for this in next alpha
2016:06:28 12:03:39 jannis Cool
2016:06:28 12:04:05 jannis I have a lot of vector?-based specs that currently all require custom generators.
2016:06:28 12:04:51 alexmiller coll-of has many new options
2016:06:28 12:17:47 alexmiller Can now indicate the expected incoming type separately from the conformed and generated type
2016:06:28 12:51:22 alexmiller Also, Rich added s/merge yesterday to create spec that merges multiple map-validating specs and will gen as you expect (union of keys)
2016:06:28 12:53:20 alexmiller @jannis in next alpha you will be able to do (s/coll-of int? :kind []) to indicate a vector of ints
2016:06:28 12:54:17 jannis @alexmiller: Awesome 🙂
2016:06:28 12:58:15 puzzler When a spec is complicated and composed of many component specs, I'm having trouble figuring out, stylistically, which of those component specs to register to namespaced keys, versus just def'ing them to regular vars. Any tips on this?
2016:06:28 12:59:33 puzzler It seems to me that when spec'ing a bunch of optional keywords for a function, to get good error messages I really want to be able to say "These are the only keys allowed in this map of optional keywords and values". Is there any way to do that?
2016:06:28 13:11:00 alexmiller generally, I think you should not def anything to a regular var
2016:06:28 13:11:36 alexmiller and you should register specs for all map attributes and for any major “types"
2016:06:28 13:11:58 alexmiller anything recursive will require registration at the recursion points
2016:06:28 13:12:33 alexmiller and you might want to register things at a finer granularity if you find it gives you more meaningful explain errors
2016:06:28 13:13:39 alexmiller re restricted key sets - no, this is intentionally not provided, but you can s/and a constraint like #(every? #{:allowed :keys :in :map} (keys %))
2016:06:28 13:25:33 puzzler Thanks. Early on, I predicted that a lot of people would re-invent slightly different definitions of validating or conforming combined with assertions in the event of a failure, and I've been seeing that happen. Is something along the lines of assert-conform and assert-valid? likely to make it into a future alpha?
2016:06:28 14:10:13 alexmiller yes, there are likely to be a conform-ex and some kind of assert, but I don’t think those will be in next alpha
2016:06:28 15:48:58 bhauman I've been wanting to add an extended :reason to certain specs, especially if they have strange interdependencies
2016:06:28 16:05:31 bhauman If you look sideways at this you could see the possibility of specs with composable prose explanations
2016:06:28 20:41:47 kendall.buchanan Super quick question: If I’m referencing a spec in another namespace, do I need to :require that namespace too?
2016:06:28 20:42:10 kendall.buchanan I’m finding the app runs fine in the REPL, but at compilation, it can’t resolve the specs.
2016:06:28 20:42:28 bhauman yep
2016:06:28 20:43:30 bhauman If I'm understanding you correctly, all specs need to be required before use
2016:06:28 20:43:40 bhauman in order for the registry to populate
2016:06:28 20:44:05 kendall.buchanan Okay, I was afraid of that 😉
2016:06:28 20:44:35 kendall.buchanan Thank you!
2016:06:28 20:46:36 martinklepsch So, form-validation & clojure.spec, I assume people have tried it already, are there any writeups/snippets?
2016:06:28 20:48:19 martinklepsch @bhauman: @kendall.buchanan I find it interesting that this can create an implicit dependency between namespaces that is not declared via the ns form, maybe for "good style" you should require the ns containing the spec even if you don't need to (technically)?
2016:06:28 20:50:00 kendall.buchanan I’ll admit – I still find the coupling between specs and namespaces odd. Namespaced keywords seem to make sense when you’re trying to define a domain path, but sometimes those domain paths don’t match your namespace hierarchy. But Clojure basically says it has to.
2016:06:28 20:50:12 kendall.buchanan Still struggling to understand it. But, seems fine.
2016:06:28 20:50:54 bhauman @kendall.buchanan: you are aware that you don't have to use an existing namespace correct?
2016:06:28 20:51:04 bhauman just a namespaced keyword
2016:06:28 20:51:31 kendall.buchanan Yeah, totally, but I guess I’m struggling to see the established path forward.
2016:06:28 20:51:33 martinklepsch > but sometimes those domain paths don’t match your namespace hierarchy
2016:06:28 20:53:11 bhauman In terms of dependency management, I think if people look at the way they did plumatic/schema or any add hoc type validation.
2016:06:28 20:54:24 bhauman it's like any dependency
2016:06:28 20:55:29 angusiguess We've started defining a lot of our specs in a separate heirarchy. It's a bit early to tell how well that works but thus far it's been pretty useful in terms of reducing circular dependency issues, making specs equally usable in tests and code, etc.
2016:06:28 20:55:30 kendall.buchanan @bhauman, would you recommend (or consider as a viable path) to always use domain-based keywords school/:type and eschewing ::type (contained in my.app.model.school? Just ditching the latter pattern?
2016:06:28 20:56:06 angusiguess So that's one possibility, is to use ::type but to place those definitions in a namespace that is a little descriptive of the domain model.
2016:06:28 20:56:29 bhauman I wouldn't be the one to ask, but I would say no
2016:06:28 20:57:48 bhauman A possiblity: As long as your keys are distinct, you can keep it all in one file.
2016:06:28 20:57:53 kendall.buchanan @angusiguess: Kind of like app.spec.school/type? That’s more or less what I’ve done for objects that cut across many domains.
2016:06:28 20:58:14 angusiguess @kendall.buchanan: That's precisely what I do, including a namespace for more generic types.
2016:06:28 20:58:30 angusiguess Like
app.spec/uuid-str
or similar.
2016:06:28 20:58:35 kendall.buchanan Right.
2016:06:28 20:58:47 kendall.buchanan Yeah, exactly what we’ve started doing.
2016:06:28 20:59:22 angusiguess Then if we have aggregates that our functions take we'll define them in the function's namespace.
2016:06:28 20:59:27 kendall.buchanan And I think it works well. I guess I like the ::thing syntax, for brevity, and realize it ties things up into the namespace. I guess we’re learning as we’re going, though.
2016:06:28 21:00:37 kendall.buchanan (By the way, I have no proposed alternative, so not complaining. Just been a source of friction for me.)
2016:06:28 21:00:49 angusiguess Defining everything in-line bit us pretty quick.
2016:06:28 21:03:28 kendall.buchanan Right, @angusiguess… which, I have to say, clojure.spec is what it claims: built for composability. I’ve been a heavy Schema user, and have to say, I’m getting more reuse out of spec by far.
2016:06:28 22:41:53 richhickey Lots of new spec stuff in alpha 8 - conforming coll support, plus count/kind control, instrument can generate stubs, gen/spec overrides in instrument/test, merge keys specs
2016:06:28 22:44:23 bfabry awesome
2016:06:28 23:05:39 Tim wow sounds good
2016:06:29 02:47:01 alexmiller I’ve updated the guide for 1.9.0-alpha8 although there will be some additional updates http://clojure.org/guides/spec
2016:06:29 09:20:08 martinklepsch I have a question about spec'ing maps: lets say I have two keys that should follow the same spec, is the intended way to define proxy specs which I then use as map keys?
2016:06:29 11:28:03 alexmiller You can register the map keys to point to either the same spec definition or to another registered spec. I think doing this will be common. One layer of registered base "types" and another layer of domain attributes.
2016:06:29 12:41:25 martinklepsch Thanks Alex
2016:06:29 12:42:14 martinklepsch
- [changed] explain-data - output is now a vector of problems with a :path element, not a map keyed by path
Whats the reasoning behind this change? Could there be multiple problems referencing the same path?
2016:06:29 12:58:10 richhickey @martinklepsch: yes, multiple problems at same path happens with collections (was already happening with ‘keys' which jammed them all into same pred, now doesn’t)
2016:06:29 14:07:54 martinklepsch @robert-stuttaford: yeah kind of I guess
2016:06:29 14:08:50 martinklepsch if you want to use a generic spec in keys without using it's key you need a proxy like this
2016:06:29 14:12:08 robert-stuttaford yeah
2016:06:29 14:21:22 alexmiller @robert-stuttaford: there is a pos-int? predicate now btw in core
2016:06:29 14:22:34 alexmiller that ::types spec you have can now be accomplished with coll-of in alpha8 too: (s/coll-of ::type :kind set? :min-count 1)
2016:06:29 14:23:38 alexmiller that version has the benefit of automatically gen’ing too
2016:06:29 14:25:07 alexmiller And I think your ::spell, ::creature, ::planeswalker, etc can also be handled by s/merge now
2016:06:29 14:25:19 alexmiller new toys!
2016:06:29 14:30:39 robert-stuttaford oh that is rad
2016:06:29 14:34:41 robert-stuttaford how far, as a %, would you say the core.spec slated for 1.9 is?
2016:06:29 14:35:07 robert-stuttaford it seems like you've still got a lot of meat on the bone, given all the stuff that gets added with every alpha
2016:06:29 14:37:07 richhickey @robert-stuttaford: not a lot of big things - assertions, perf tweaks, reconciling regexes and vectors, some refinement of keys*, then details
2016:06:29 14:38:02 richhickey then moving to infrastructure around test-running etc, the whole model implied by instrument+test which is different from current practice
2016:06:29 14:39:48 robert-stuttaford fantastic, rich! very exciting set of tools you're building, here. fundamental, like the data-oriented approach and immutability are. going to take some time to soak in
2016:06:29 14:40:40 robert-stuttaford i realise this may well be answered with 'maybe :-)', but i'm super curious how you plan to leverage spec in Datomic 🙂
2016:06:29 14:42:25 richhickey @ghadi: thanks for the links
2016:06:29 14:42:54 robert-stuttaford Datalog is an obvious one, as is transaction data. wondering how else it may improve the experience of programming with it
2016:06:29 14:43:09 richhickey @robert-stuttaford: still TBD (but lots of obvious opportunities), first things first, and that’s a stable clojure.spec
2016:06:29 14:43:34 robert-stuttaford of course 🙂
2016:06:29 14:43:46 richhickey I can say we are getting a lot out of using it in the development and testing of Datomic already
2016:06:29 14:44:34 robert-stuttaford i can't think of a better way to dogfood it
2016:06:29 14:45:40 richhickey the new stubbing support is something to check out - you can now take a version of your wire interface, spec it, and get a no-wire stub for free from spec
2016:06:29 14:46:08 ghadi rad
2016:06:29 14:46:45 robert-stuttaford ah - if i understand you correctly, you mean inferring a spec from an existing data set?
2016:06:29 14:47:00 robert-stuttaford if so, thanks .. i was planning to do this as a learning exercise!
2016:06:29 14:48:15 richhickey no, let’s say you have a data-driven wire protocol that talks to a service - you spec the protocol, instrument its namespace saying :stub in the overrides, and then run tests of code that consumes the protocol. Now those tests don’t talk to a service at all but get checking of their payloads and generated returns
2016:06:29 14:49:18 ghadi aside: I really like the syntax of (s/alt :foo x :bar y). That is something I didn't get quite right in pex, my vm-based PEG parser; I made capturing things too verbose. Definitely going to adopt that approach
2016:06:29 14:49:41 richhickey you can selectively override gens to generate ‘stateful’ things like connections
2016:06:29 14:50:06 robert-stuttaford oh, right!
2016:06:29 14:50:52 richhickey @ghadi: yeah, the path system is something I think people resist initially, but it keeps paying dividends
2016:06:29 14:51:19 robert-stuttaford man. i feel like i'm going to need a spec sabbatical.
2016:06:29 14:53:33 robert-stuttaford so many new toys, but at such a fundamental place in the language.. it's going to touch everything!
2016:06:29 14:53:47 ghadi curious, why do people resist the path system?
2016:06:29 14:54:25 richhickey sometimes you don’t care about the labels or want to see them in the destructuring
2016:06:29 15:11:51 bhauman and that makes sense because the conformed value is just a string
2016:06:29 15:16:10 richhickey ‘and’ does flow conformed values - "Successive
conformed values propagate through rest of predicates."
2016:06:29 15:18:10 richhickey that may change for ‘merge’, which is what you should switch to for this application
2016:06:29 15:19:29 richhickey there may also be and and and-> variants since sometimes you need this (e.g. when preceding pred is a regex) and sometimes not
2016:06:29 15:24:31 bhauman great, looking at merge now
2016:06:29 15:29:36 bhauman the above works with merge right now and inspection reveals that the conformed values aren't flowing right now.
2016:06:29 15:32:39 richhickey @bhauman: merge does flow through conform, not explain (that’s broken)
2016:06:29 15:32:57 bhauman oh gotcha
2016:06:29 15:33:37 richhickey but no reason for it to flow through conform, that will change. The big benefit of merge is that it will gen where ‘and’ cannot for this
2016:06:29 15:34:05 bhauman good deal
2016:06:29 15:36:38 bhauman my use case is taking an existing keys spec and constraining it by moving some keys into :req
2016:06:29 15:37:54 richhickey I understand, and that will be common
2016:06:29 15:38:09 richhickey also various intersections
2016:06:29 16:17:39 seancorfield (but, yeah, great changes in Alpha 8… I updated java.jdbc to accommodate enough changes to get it running again but I haven’t dug deep into redoing the specs to leverage the new features)
2016:06:29 16:51:05 jannis An optional piece of data inside a tuple is not supported, I assume? (s/def ::my-tuple (s/tuple (s/? keyword?) symbol?)) claims ['foo] is invalid: [foo] fails spec: :workflo.macros.command/my-tuple predicate: (= (count %) 2)
2016:06:29 16:51:48 jannis My current "workaround" is to use (s/and vector? (s/cat :optional (s/? ...) :mandatory ...))
2016:06:29 16:52:10 jannis Which is ok, it does mean that you have to write a custom generator for it though.
2016:06:29 16:53:55 jannis Also, is there something like s/tuple for fixed-size non-vector sequences with a specific order of subspecs?
2016:06:29 16:57:05 alexmiller You can use s/cat, s/? etc for things like this
2016:06:29 16:58:09 alexmiller If you have optionality regex is definitely the right choice over tuple
2016:06:29 16:58:19 jannis Ok
2016:06:29 16:58:36 alexmiller We are still looking at options to also enforce vector on regex in a good way
2016:06:29 17:00:25 alexmiller s/and is sufficient as you saw but needs custom gen as you said
2016:06:29 17:04:18 jannis @alexmiller: Ah, s/cat didn't work because I forgot s/spec around it to make it a subspec and require the content of s/cat to be in a child vector or sequence.
2016:06:29 18:14:53 uwo my colleagues would like to create a single source of truth to generate both datomic schema and specs. Is this advisable? I was under the impression that perhaps spec would be written mostly by hand and not codegen
2016:06:29 18:48:31 alexmiller seems low on the crazy scale to me
2016:06:29 18:50:36 uwo haha. thanks
2016:06:29 18:54:50 alexmiller code is data and all that :)
2016:06:29 20:07:36 tyler Is clojure.spec.test/test meant to be used with clojure.test?
2016:06:29 20:13:50 tyler I guess more specifically, what is the idiomatic way to set up generative tests with clojure.spec is there an equivalent of defspec from test.check for clojure.spec?
2016:06:29 20:24:05 alexmiller fully utilizing the instrumentation and test facilities in clojure.spec.test is a different kind of process from the existing clojure.test and I don’t think we will provide a linkage like defspec.
2016:06:29 20:24:57 alexmiller we have more to say about this and I expect we will have more guidance to provide before 1.9 is released
2016:06:29 20:26:30 tyler Gotcha, is it expected to be a developer concern? Or will there be additional functionality added to integrate it with clojure.test? The return seems more data-centric than clojure.test.
2016:06:29 20:35:57 alexmiller I don’t currently expect anything that integrates it with clojure.test. The returns are more similar to the results from test.check as that’s what we’re actually invoking.
2016:06:29 20:37:17 alexmiller there are levels of question here - some around what’s provided in core, another level around runners, and another level around build tooling
2016:06:29 20:41:24 tyler I suppose the core of my question is around whether or not clojure.spec.test/test will integrate into the existing test tooling or is it expected that new test tooling will need to be built? instrument seems to fit into the current ecosystem but it seems like clojure.spec.test/test will require new tooling.
2016:06:29 20:44:44 puzzler Is there a difference between (s/def ::type vector?) and (s/def ::type (coll-of ::s/any :kind vector?)) ?
2016:06:29 20:47:56 martinklepsch When a required key is missing from a map the error looks like this:
Is there a way to get an easier to parse description that will allow me to (programatically) figure out which keys are missing from a map?
2016:06:29 20:50:04 angusiguess You can get that data structure with s/explain-data
2016:06:29 21:03:30 wilkerlucio @tyler: if you want to run generative tests and have report of it in clojure.test you can use the clojure.spec.test/test like this: (is (true? (-> (clojure.spec.test/test 'my.ns/some-fn) :result)))
2016:06:29 21:06:42 tyler Thanks @wilkerlucio. I had been playing around with that a little bit but the output isn’t very informative for failures for that. I think I’m going to go the route of experimenting with developing my own runner. Just wanted to make sure I wasn’t duplicating work/effort.
2016:06:29 21:11:50 martinklepsch @angusiguess: right, what I pasted is part of what s/explain-data returns, was just wondering if there's a better way to get a list of missing keys
2016:06:29 21:12:03 wilkerlucio @tyler: try using something like this:
2016:06:29 21:13:39 wilkerlucio not super pretty, but might be a good start 🙂
2016:06:29 21:13:51 tyler Thanks I’ll give that a go
2016:06:29 21:16:22 puzzler Is there an explanation somewhere of how the sampling of every works? I'd like to get a sense for the probability that an error is detected if I switch from coll-of to every.
2016:06:29 21:18:20 mcorbin Hello, i play with clojure-spec and i don't understand this :
2016:06:29 21:19:09 mcorbin Why explain returns Success ? The second condition in the "s/and" should return false with a string parameter
2016:06:29 21:24:49 leongrapenthin @mcorbin I believe the lambda pred is always invoked with a vector, i. e. [:string "Hello Clojurians"]
2016:06:29 21:27:13 bhauman @mcorbin: theres a discussion of this from earlier today ...
2016:06:29 21:27:51 bhauman see my posts above
2016:06:29 21:28:47 mcorbin @leongrapenthin: you win, thanks @bhauman i will look, thx
2016:06:29 21:41:52 alexmiller @puzzler: if you mean “how it samples” - no there intentionally is not, as it may change in undefined ways in the future
2016:06:29 21:43:11 alexmiller all you should count on is that it will sample no more than *coll-check-limit* elements
2016:06:29 22:03:23 seancorfield @tyler: FYI, we use Expectations to drive test.check stuff and ended up with a pattern like this /cc @wilkerlucio
2016:06:29 23:21:40 puzzler Suggestion: Right now it looks like every and coll-of check for distinctness with (apply distinct? ...). Would recommend (or (set? ...) (apply distinct? ...)) to short-circuit when a set is passed in to a function that expects a collection of distinct items.
2016:06:30 00:16:27 jjcomer I’m trying to use the new stub feature, but I can’t seem to get the stub to take hold. The println still happens for me:
2016:06:30 00:29:32 puzzler What happened to instrument-all and instrument-ns in the new alpha?
2016:06:30 00:33:07 jjcomer @puzzler: All is instrument with no params and ns is passing an ns to instrument
2016:06:30 00:49:13 puzzler thanks
2016:06:30 00:50:17 puzzler Is there any way to keep the instrumenting on during development? It seems that when you recompile a namespace it blows away the instrumentation.
2016:06:30 01:26:14 seancorfield Since instrument updates the Vars in place, I’d expect that (so, "no", I think is the answer).
2016:06:30 01:26:40 seancorfield If you recompile the namespace, you’re replacing all the Vars, so you’d need to re-run instrument.
2016:06:30 01:31:22 gfredericks puzzler: one option is to put (s/instrument ...) at the bottom of relevant namespaces
2016:06:30 01:31:33 gfredericks options that don't modify the source code would depend on how you're loading your code
2016:06:30 01:31:51 puzzler In theory, defn could be rebound to automatically instrument functions that have specs.
2016:06:30 01:33:49 seancorfield That would only help if the spec was defined ahead of the function...
2016:06:30 01:37:33 puzzler I'm not saying this should be the general definition of defn, I just mean that when you choose to instrument all, a new version of defn (and fdef to handle the case of function before spec) would ensure the instrumentation never gets out of sync at the REPL, which I'm finding to be an annoyance. Right now, you change the spec and the instrumented function doesn't see it, or you change the function and the spec-checking goes away. Too easy to forget to re-run instrumentation.
2016:06:30 01:38:10 seancorfield I have my specs in a different namespace...
2016:06:30 01:39:28 puzzler @seancorfield: Are you saying that having specs in a different namespace alleviates this issue? Seems to me it would just make it even more complicated.
2016:06:30 01:39:46 seancorfield I’m saying your suggestions wouldn’t address that problem.
2016:06:30 01:39:58 seancorfield i.e., it’s not a general solution
2016:06:30 01:40:46 seancorfield (and I don’t know what the general solution is)
2016:06:30 01:41:38 seancorfield For the specific case of java.jdbc, I have the test namespace — which loads the specs — also run instrument.
2016:06:30 01:42:09 seancorfield So the code is separate from the specs, but the tests depend on them.
2016:06:30 01:42:36 puzzler The first time you call instrument-all, it's calling your spec name space which registers the specs in a global registry. So a defn that was aware of the specs in the registry would help that direction. Going the other direction, if you edit the spec in the separate file, that spec refers to the var in the implementation namespace, so that function could be re-instrumented. So I think it would probably work. What am I missing?
2016:06:30 01:43:54 seancorfield I think it’s very dependent on workflow. I’m waiting to see what Clojure/core come up with (based on the discussion earlier about what’s coming for specs in the test tooling area).
2016:06:30 01:44:25 seancorfield What you raise is certainly an issue that can make working with spec in the REPL pretty annoying.
2016:06:30 01:44:58 puzzler I'm only imagining this would be helpful in a mode where all instrumentation is turned on for everything. I don't have a good handle on the workflow for spec yet, so I agree with your point that we'll see how this evolves.
2016:06:30 01:46:12 seancorfield Right now I’m only instrumenting on a per-namespace basis as I work with / test a particular namespace. I ran into some early weirdness with instrument-all (and no longer remember the details) and backed off that… may have been back when it caused generative testing to occur on higher-order function specs?
2016:06:30 01:46:42 seancorfield So it may be that separating the generative part of function specs from instrumentation has addressed the issues I ran into...
2016:06:30 01:47:39 puzzler The only issue I have with putting specs in a separate namespace is that it diminishes their value as a form of documentation. (Actually, this is also an issue with putting tests in a separate namespace, which I've gotten used to, so maybe I'll get used to putting specs elsewhere as well).
2016:06:30 01:48:43 puzzler In your split-namespace approach, do you see specs in the doc strings of your functions?
2016:06:30 01:49:47 seancorfield If you’re working in the test namespace (where the spec has been loaded), yes.
2016:06:30 01:50:17 seancorfield I haven’t settled on it as a normal way to work yet. We’re still exploring several options
2016:06:30 01:50:25 puzzler But users of your library generally wouldn't see that if they call doc on a function at the REPL, right?
2016:06:30 01:50:38 seancorfield If they load the specs, yes.
2016:06:30 01:50:50 seancorfield This allows the code to be used by pre-1.9 clients.
2016:06:30 01:51:07 seancorfield For java.jdbc, it supports all the way back to 1.4
2016:06:30 01:51:19 seancorfield and even in 1.9, the specs are optional but available.
2016:06:30 01:51:24 puzzler Ah, yes, I can see how that would be the most critical thing to keep in mind for java.jdbc
2016:06:30 01:51:46 seancorfield when I run the java.jdbc tests on 1.9, it loads the specs and instruments the ns (not all, just java.jdbc)
2016:06:30 01:52:36 seancorfield So, for library code that might have pre-1.9 users, separate specs is pretty much the only possibility.
2016:06:30 01:52:44 puzzler Could the main sourcecode namespace do that test on version number, and automatically load the specs for docstring purposes?
2016:06:30 01:53:15 seancorfield For application code that’s is bound to 1.9, I can imagine having specs intermingled — maybe.
2016:06:30 01:53:55 seancorfield Yeah, I guess java.jdbc’s source namespace could auto-load the specs on 1.9. I’m just not sure that would be what all its users would expect.
2016:06:30 01:54:37 seancorfield After all, if you (instrument) your code would you expect all the external libraries to slow down due to specs?
2016:06:30 01:55:13 seancorfield Some specs can have a really high performance overhead so I don’t think the library should dictate that.
2016:06:30 01:55:43 puzzler I'm thinking that my number one use for specs will be to get good error messages when working with other libraries, so yes, I think that's what I personally would want. The biggest source of errors in my code is having a false expectation about how someone else's code should work.
2016:06:30 02:06:28 seancorfield Good to know.
2016:06:30 02:07:45 seancorfield We’ve jumped on the Alpha builds at work (in production) in preparation for using clojure.spec fairly heavily but we’re still in the early exploration phase. We’re already leveraging the new predicates in production code tho’.
2016:06:30 02:09:03 seancorfield We haven’t fully decided what we’ll spec out yet vs what we won’t bother with. I’ve been exploring using java.jdbc with namespaced keys as part of this work.
2016:06:30 02:10:34 seancorfield We have over 30Kloc production code and over 11Kloc tests so whatever we do will need to be a gradual process...
2016:06:30 02:11:39 seancorfield …expanding what we can do with generative testing alone might bring us a good "bang for our buck".
2016:06:30 02:18:32 Tim @seancorfield: at world singles?
2016:06:30 02:19:58 seancorfield Yup. Just pushed a build to QA based on Clojure 1.9.0 Alpha 8 so that’ll go to production in our next build.
2016:06:30 02:20:13 seancorfield (we already have Alpha 7 in production)
2016:06:30 03:58:30 cap10morgan I'm getting Don't know how to create ISeq from: spec_demo.core$full_name when running (stest/instrument full-name) in an ns where I have a fn full-name and have (s/fdef full-name ...). It works if I leave the argument off (so it instruments everything), but I thought it was supposed to take symbol args now too in alpha8?
2016:06:30 04:22:27 seancorfield (stest/instrument 'full-name) <-- needs to be a symbol
2016:06:30 04:26:49 seancorfield To explain the error message, (stest/instrument full-name) will evaluate full-name, since it's a function it evaluates to its class name which is a class full_name inside the package spec_demo.core because the Clojure is spec-demo.core/full-name.
2016:06:30 04:26:57 seancorfield Hope that helps @cap10morgan ?
2016:06:30 04:50:30 puzzler I'm having trouble ascertaining the purpose of the retag in multi-spec. Can someone clarify when you'd actually see the retag?
2016:06:30 11:35:04 alexmiller In gen
2016:06:30 11:35:27 cap10morgan @seancorfield: ah, yes, thanks. I was trying #'full-name and getting a var but totally forgot about 'symbol.
2016:06:30 11:37:02 cap10morgan I'm giving a presentation tonight on clojure.spec at Den of Clojure (Denver meetup). So putting together as many live demoes as I can.
2016:06:30 11:44:10 cap10morgan Hmm, but it still doesn't work. (stest/instrument 'full-name) doesn't seem to turn on instrumentation (I get an exception when an invalid arg blows up in the fn body rather than spec telling me it's invalid) and (stest/test 'full-name) just returns (). These all worked with just (stest/instrument) and (stest/test).
2016:06:30 11:46:50 alexmiller If you use back tick it will resolve the symbol to fully qualified
2016:06:30 11:47:08 alexmiller The symbol has to be fully qualified
2016:06:30 11:47:36 alexmiller Symbol, not var
2016:06:30 11:49:22 cap10morgan OK, that fixed it. Thanks, @alexmiller. I think that's the first time I've used back tick outside of a macro. Makes sense that it would need to be fully-qualified since it's going into a central registry.
2016:06:30 11:59:09 cap10morgan Did clojure.spec.test.check go away or change in alpha8? Trying to change how many tests get run in clojure.spec.test/test and test's docstring says to pass a map with a :clojure.spec.test.check/opts keyword.
2016:06:30 12:11:13 alexmiller That's just a keyword namespace, not an actual namespace
2016:06:30 12:22:40 cap10morgan huh, OK. not sure how to alias it, then. but I guess I don't strictly need to.
2016:06:30 12:32:18 jjcomer Does anyone have an example of stub or replace? Neither seem to be working for me, the original fns still execute.
2016:06:30 12:52:34 puzzler @alexmiller I thought multi-specs couldn't auto-generate data since the possibilities can't be enumerated. And if you have to write your own generator, I don't see how the retag comes into play.
2016:06:30 13:25:18 bhauman I'm sure that this has probably been discussed before, a bunch, but I am late to the game and curious about and flowing conformed values to successive predicates. Initially, it seems very surprising that a successive predicate is operating on a different__ value. This seems semantically more like and->. Is this in the Guide? It's also curious that the underlying implementation of merge is named and-preds.
I know there is a reason for this and admit that I haven't looked closely enough at it.
2016:06:30 14:57:02 fenton @puzzler: you can auto-generate data for multi-specs.
2016:06:30 15:47:10 alexmiller @puzzler did you read the doc-string for multi-spec? I think it answers all these questions.
2016:06:30 15:47:47 alexmiller @bhauman: and vs and-> is under consideration. for now, and flows conformed values.
2016:06:30 15:48:35 bhauman cool, i didn't know and-> was under consideration
2016:06:30 15:49:33 alexmiller Rich mentioned it in the back chat
2016:06:30 15:51:56 bhauman oh and a big big + 💯 on every
2016:06:30 15:53:18 flipmokid Hi, I'm looking to write a spec for a map in clojurescript which is used to check user input. One of the keys requires that the input be an int and be in a set of allowed values which comes from a server (so it's an async call). What is the best way of achieving this?
2016:06:30 15:59:32 bhauman if its deep in the spec, make the call to the server first, and then make a binding, and have your predicate read the binding
2016:06:30 16:02:21 bhauman but otherwise I would separate this concern from the spec
2016:06:30 16:02:51 bhauman and do a second pass
2016:06:30 16:03:23 alexmiller you don’t have to make it a fn spec to use spec - you can explicitly call s/valid? to make that check
2016:06:30 16:08:45 flipmokid @bhauman: Thanks, that make sense (having that separate from the spec). BTW thanks for figwheel!
@alexmiller I wanted to originally do something like (s/and integer? (call-service)) where call-service returns a channel which eventually returns a set of ints. I wasn't sure of the best way of doing this as that function isn't a predicate
I'm fairly new to clojure btw so please bear with me!
2016:06:30 16:09:51 alexmiller have you considered just writing some Clojure code? :)
2016:06:30 16:10:08 alexmiller I’m not sure spec is buying you anything for what you’re describing
2016:06:30 16:12:09 flipmokid I will give it a go. Time to crack open Clojure Applied again 🙂
2016:06:30 16:14:59 alexmiller well nothing about spec in there :)
2016:06:30 16:15:12 alexmiller those idiots were too dumb to write about it
2016:06:30 16:17:07 flipmokid Lol. The book is great, thank you! I meant using the book to write some clojure code. I thought I should do the server check separately from the spec but just wanted to make sure.
2016:06:30 16:17:11 flipmokid Thanks guys
2016:06:30 16:17:14 flipmokid Much appreciated
2016:06:30 16:18:35 alexmiller you shouldn’t register a spec that’s calling a server, but you could use s/validate? to verify a value conforms to a spec (which has a predicate, which is based on a set obtained from elsewhere)
2016:06:30 16:19:05 alexmiller however, that “check” is so simple, that you might as well just check if the value is in the set (without using spec)
2016:06:30 16:23:51 flipmokid With the first option, what would be the best way of waiting for the set to come back out of the channel in clojurescript inside the predicate? The best my mind could do was poll the channel until there was something on it.
The current map spec I have does have a lot of keys which are much more involved than this one, but this is the only one that needs to check against a remote resource.
2016:06:30 17:10:56 fenton where do we report issues found with spec?
2016:06:30 17:13:07 fenton but change :matching-p to :matching and it works okay... seems hyphens don't work for matching values.
2016:06:30 17:16:23 fenton hmm... hold on that... maybe dirty repl
2016:06:30 18:49:49 blance I just read about clojure.spec and it seems like a great tool for testing with test.check. To utilize it for unit testing, should I specify a spec for every functions using fdef in my application so that I just call clojure.spec.test/test to do unit testing?
I've never used schema before so I'm not sure which function/args/data I should specify those and which not to
2016:06:30 18:58:52 seancorfield @blance: Have you read the Rationale for clojure.spec? It talks about when to use spec and what sort of stuff it’s designed for.
2016:06:30 18:59:38 bhauman So here is another thing, the :in path on key value errors in (s/every-kv ..) end withs a 0 1 presumably to disambiguate between the key and the value, but this renders the :in path pretty tough to use to look up a value in a nested structure. I'm thinking that intention was to have the :in path terminate with the map key and the 0 or 1 should be at the end of :path only
2016:06:30 19:01:16 blance Yea I've read both.. It must be that I read through them too fast. I'll read them again.
2016:06:30 19:02:08 blance I mean I know how to use them, but I'm not sure on where to use it. Does it make sense to say have a fdef for each single function in my app?
2016:06:30 19:03:20 eggsyntax A reasonable shorthand might be: any function that you would otherwise have hand-written a test for.
2016:06:30 19:03:59 eggsyntax (and of course for fns where spec is valuable in other ways, eg runtime validation of inputs)
2016:06:30 19:04:24 eggsyntax That's just my take on it, though, and I'm a spec beginner too. As I guess are we all 😉
2016:06:30 19:05:36 blance you would want your unit test to cover as much functions as possible right? so write spec for as much functions as possible as well?
2016:06:30 19:06:23 blance I feel like some fns would take as long time to write a spec to validate its input&output as to write the actual function itself
2016:06:30 19:06:53 eggsyntax Matter of taste. Personally I don't shoot for complete test coverage, just coverage of fns that a) are tricky, b) are at risk for regression bugs, c) could use a test as usage documentation.
2016:06:30 19:07:59 eggsyntax And yeah, it's definitely arguable that hand-written tests are a better tool to reach for in some cases than spec is. It'll be interesting to watch as different ideas about best practices for spec emerge.
2016:06:30 19:08:41 eggsyntax (although the new stub/mock functionality makes it a tool I'm likely to reach for a lot more often)
2016:06:30 19:10:40 blance that make sense. and I would assume you only need to use clojure.spec.test/test for unit testing? Do you still hand write test to cover corner cases?
2016:06:30 19:10:56 blance what's the stub/mock functionality? is that also in spec?
2016:06:30 19:11:20 blance sorry I have lots of questions as I'm new to testing as well:sweat_smile:
2016:06:30 19:12:46 eggsyntax To me it seems like generative testing is, in general, more likely to catch corner cases than hand-written tests, unless there's a specific corner case you want to test for that isn't likely to be generated.
2016:06:30 19:13:17 eggsyntax (oh, :stub is an optional arg to instrument btw)
2016:06:30 19:13:25 blance that's cool! didn't see that in the doc
2016:06:30 19:13:43 eggsyntax It's new as of alpha8. I think the guide isn't quite caught up yet.
2016:06:30 19:14:23 blance Ill take a look at it. Thanks!
2016:06:30 19:16:25 bfabry @fenton: there's no way and could combine the generators. there's a new function s/merge specifically for that scenario though
2016:06:30 19:16:50 fenton @bfabry: okay, will check that.
2016:06:30 19:29:42 seancorfield @blance: If you specify "every" function then your system will be brittle: changing anything will likely break specs and you’ll have to constantly rewrite them — it’ll be hard to do refactoring. I think specs make sense on "module" boundaries and on "library APIs".
2016:06:30 19:31:25 seancorfield As for generative testing, you can test a function against a spec (in the test suite) without needing to fdef it in the main code. That seems like a reasonable approach to me.
2016:06:30 19:31:30 fenton how to spec for a string that should be an alpha-numeric?
2016:06:30 19:31:59 seancorfield Use a predicate with a string regex?
2016:06:30 19:33:42 fenton @seancorfield: sorry not understand, example?
2016:06:30 19:39:10 fenton @seancorfield: even nicer!
2016:06:30 19:39:54 eggsyntax It'd be nice to do something like (s/def ::alphanumeric (s/and string? #(Integer/parseInt %))), but that'll throw an exception rather than failing...
2016:06:30 19:40:50 blance @seancorfield: I'm actually working on clojurescipt. most of my fns are api calls, transition db to new states, and UI stuffs. No need to write spec for any of them I guess?
2016:06:30 19:41:10 seancorfield @eggsyntax: you would need both a predicate to match numeric strings and a conformer to convert to Long I think?
2016:06:30 19:41:13 bhauman I looked into the :in path problems for maps that I mentioned above, it seems like ::kfn isn't enough to eliminate the following tuple key.
2016:06:30 19:41:25 eggsyntax @seancorfield: makes sense.
2016:06:30 19:41:44 eggsyntax (& at that point the regex solution is nicer)
2016:06:30 19:41:54 seancorfield @blance: Hard to say in absolute terms. Try a few approaches and see what works best for you?
2016:06:30 19:42:42 blance sounds good. Thanks!
2016:06:30 19:42:55 seancorfield I’d probably spec the API itself (between client and server) and then maybe some of the bigger components within each side. But I suspect you have a lot of "trivial" functions which wouldn’t be worth spec’ing?
2016:06:30 19:43:22 eggsyntax New motto: "Don't spec the small stuff"
2016:06:30 19:43:28 eggsyntax 😉
2016:06:30 19:43:36 blance api calls are async, or rather returning a core.async channel
2016:06:30 19:43:42 blance so not sure how to spec that
2016:06:30 19:47:53 bhauman its expressive enough for me to fix it in use though
2016:06:30 19:48:25 bfabry I figure fdefs probably live at a similar level to unit tests. if you unit test every function in your application it'll be annoying to change, but unit testing still holds value at some "right" level. same with fdefs
2016:06:30 19:48:56 bfabry fdefs have the added advantage that they also give you runtime contract checking and easier to understand documentation than unit tests
2016:06:30 19:49:07 puzzler @alexmiller I read the doc-string about multi-spec but (sg/generate (s/gen my-multi-spec)) throws ExceptionInfo Unable to construct gen at: [] for: so I must be misunderstanding the point about generation.
2016:06:30 20:17:42 fenton @puzzler: did u see the example i made for u?
2016:06:30 20:29:26 puzzler @fenton no, didn't see that. Where?
2016:06:30 20:29:48 puzzler @fenton, ah, I see it now.
2016:06:30 20:32:11 puzzler @fenton, hmm, not sure why it isn't working for me. Maybe one of the individual multimethods can't generate but the error message is misleading, making it seem like the overall multispec is failing.
2016:06:30 20:32:46 fenton @puzzler: did my example work for you?
2016:06:30 20:38:49 puzzler @fenton: no, I copied and pasted your code and tried it at the REPL but I'm getting NullPointerException clojure.test.check.generators/call-gen (generators.cljc:41) This is on the latest alpha. Not sure what's going on.
2016:06:30 20:41:40 puzzler @fenton, oh I called gen and generate in the wrong order on your code. So yours is now working for me. But I had it right on my example which wasn't working. So still trying to figure that out...
2016:06:30 20:43:27 puzzler @fenton, ok figured it out. I wasn't calling gen on the keyword associated with the multispec, I was calling it on the actual multispec.
2016:06:30 20:54:54 puzzler In some cases the docstring info provided automatically by spec isn't terribly useful, especially in the case of multi-specs. Is there any way to manually add info to the spec docstring?
2016:06:30 21:19:37 fenton @puzzler glad u got it work....dunno about ur last comment tho. 🙂
2016:06:30 23:20:15 blance Would you put all spec defination at one place? say my.appns.spec? If so, any trick to refer to a namespaced keyword?
2016:06:30 23:21:20 blance I assume nobody want to type {:my.appname.is.very.long/key1 "foo" :my.appname.is.very.long/key2 "bar"} `
2016:06:30 23:25:17 blance Also i'm wondering if clojure.spec can spec arbitrary nested vector? for example, geoJSON coordinates can be [1 1] or
2016:06:30 23:27:37 alexmiller You have optionality and recursion, so without knowing more than those examples, yes
2016:06:30 23:28:57 alexmiller @blance if you are making a spec namespace, I would propose the plural (specs) rather than spec. We will be doing clojure.core.specs for example.
2016:06:30 23:29:32 alexmiller For namespaces, you can alias with :as in require then autoresolve with :: keywords
2016:06:30 23:31:05 blance like my.appname.specs :as specs then ::specs/key1 right?
2016:06:30 23:31:17 alexmiller Yeah
2016:06:30 23:32:15 blance cool thanks!
2016:06:30 23:41:53 blance On a side note, I feel like clojure team does not want us to put all specs at one place, otherwise we would have the option to just describe un-namespaced key which is more convenient.
2016:07:01 03:30:59 seancorfield I’m working with instants and ended up with this code:
(defn years-ago
"Given a number of years, return an Instant that long ago."
[y]
(-> (LocalDate/now) (.minusYears y) .atStartOfDay (.atOffset ZoneOffset/UTC) .toInstant))
(defn age-18-120?
"Given an Instant, return true if it represents a date of birth such
that someone would be between 18 and 120 years old."
[i]
(s/inst-in-range? (years-ago 120) (years-ago 18) i))
(s/def ::date-of-birth (s/with-gen age-18-120?
(fn [] (s/gen (s/inst-in (years-ago 120) (years-ago 18))))))
Could this be done more cleanly? (without the duplication of the range of years)
2016:07:01 04:26:28 bfabry though the age-18-120? fn seems totally redundant at that point, but if you still wanted it
2016:07:01 04:28:20 seancorfield No, because the years-ago calls must happen when the predicate is applied, not just once at compile time.
2016:07:01 04:28:50 bfabry riiiiight, makes sense
2016:07:01 04:34:01 seancorfield And I'm assuming the with-gen generator-returning fn is called each whenever the spec is exercised / test-gen'd but that's not really as important since tests are short-lived, whereas the spec has to be long-lived and check the correct range of dob each time it's conformed / used for validation.
2016:07:01 09:58:38 olivergeorge @seancorfield: could "now" be part of the data structure you spec/validate? that makes it a simple input rather than implied context.
2016:07:01 11:13:19 olivergeorge Will it make sense to update defn to allow a spec to be provided as metadata? seems like it could overlap with pre/post...
2016:07:01 11:58:56 alexmiller No, we won't be doing that
2016:07:01 13:09:31 seantempesta So, specs can also be used to convert invalid data into valid data? I’m reading this…
conformer
macro
Usage: (conformer f)
(conformer f unf)
takes a predicate function with the semantics of conform i.e. it should return either a
(possibly converted) value or :clojure.spec/invalid, and returns a
spec that uses it as a predicate/conformer. Optionally takes a
second fn that does unform of result of first
So how do you use the feature to get the response (possibly converted)?
2016:07:01 13:11:22 bhauman @seantempesta: you have to provide a function that returns :invalid or a result that is the converted value
2016:07:01 15:33:22 wilkerlucio @ghadi: one simple option that I see is to move the cached data to the map meta-data, this way the map validation can stay simple
2016:07:01 15:36:20 glv But I’d like to validate the cached data as well. For purposes of the code, it’s not valid if those keys aren’t there.
2016:07:01 15:36:42 glv (That was clumsily worded, but you know what I mean … the code expects those keys.)
2016:07:01 15:44:09 wilkerlucio @glv: I had a different idea, what you think on this:
2016:07:01 15:44:42 wilkerlucio here, I used a conformer to remove the namespaced keys before doing the map validation, but checking the keys before doing it
2016:07:01 15:44:58 wilkerlucio this way all namespaced keys will be validated, then removed just to check the rest of the map
2016:07:01 15:53:32 glv Hmm … that’s better than my current horrific solution. 🙂
2016:07:01 16:30:54 seancorfield @olivergeorge: Interesting idea… The data structure comes from the database and we already decorate it with some computed fields that are transient (short-lived) so adding a current timestamp to the data wouldn’t be a hardship.
2016:07:01 16:56:44 jjcomer I’m trying to use spec-test/test in my clojure.test tests (that was a lot of test :)). When I try to realize the result of test I get the following exception
java.lang.ClassCastException: clojure.lang.AFunction$1 cannot be cast to clojure.lang.MultiFn, compiling:(clojure/test/check/clojure_test.cljc:95:1)
2016:07:01 16:56:52 ghadi @alexmiller: good luck!
2016:07:01 16:56:58 jjcomer If I run the test outside of a deftest it works no problem
2016:07:01 16:57:16 jjcomer Does anyone have a strategy they are using to use spec-test/test within a deftest?
2016:07:01 17:18:16 glv @alexmiller no worries … whenever things calm down is fine.
2016:07:01 19:14:17 akiel What is the idiomatic way to spec an empty :args seq? I currently use (s/cat).
2016:07:03 14:27:18 alexmiller You should probably also look at s/int-in for int ranges with gen support
2016:07:03 21:50:51 bhauman I'm curious if it's an eventual goal to make explain's :in path work with core functions like get-in? Right now the spec'ing of maps as a sequential collection via (every (tuple k? v?)) is causing some ambiguity in the :in path.
2016:07:04 08:16:25 dhruv @alexmiller: thank you very much! That explanation clears it up for me, plus s/int-in works a treat for my purposes :+1::skin-tone-3:
2016:07:04 13:51:55 eggsyntax It'd be useful to have an undef fn or something like that, to clear a spec from the registry. Or some way to clear the registry entirely. Mostly just for use during development. Or, for that matter, just make the registry-ref atom publicly accessible.
2016:07:04 14:07:07 eggsyntax I didn't realize that'd get around the private declaration. Good to know in general.
2016:07:04 14:32:04 alexmiller @eggsyntax we’ve talked about this and I expect there will be something public like this
2016:07:04 14:32:19 alexmiller eventually :)
2016:07:04 14:32:58 eggsyntax Sounds good. & Andre's hack will do me fine for now 🙂
2016:07:05 01:50:31 wildermuthn Reading “Out of the Tarpit” (http://shaffner.us/cs/papers/tarpit.pdf), and maybe I’ve just got Spec on my mind, but this looks pretty familiar:
2016:07:05 02:04:21 wildermuthn The article’s description of "Functional Relational Programming” includes a section on “Essential Logic” that seems to correspond very well with Spec!
2016:07:05 09:58:51 alexmiller I think it's no accident that Clojure+Datomic overlap pretty well with Out of the Tarpit
2016:07:05 13:06:48 alexmiller s/and flows conformed values through the predicates and the s/+ conforms to a vector
2016:07:05 13:07:36 alexmiller Rich has mentioned a split into s/and and s/and-> to have both choices
2016:07:05 14:22:28 vikeri @alexmiller: Thanks! Yes it could probably be useful sometimes to not have it flow through the predicates but evaluate them separately.
2016:07:05 15:01:20 alexmiller separately, the need to validate this specific case of a regex and vector? is common and one I’ve run into many times in spec’ing macros and there might be some addition specifically for this
2016:07:05 20:27:26 aengelberg Has there been any talk about allowing local registries to use with clojure spec functions rather than only using the global registry? Similar to (make-hierarchy) and derive.
2016:07:05 20:35:56 alexmiller I haven't heard such talk :)
2016:07:05 20:51:11 pdlug I’m trying to use multi-spec but it doesn’t seem to work with defmethod’s :default dispatch, is this a bug or intentional?
2016:07:05 20:55:33 kidpollo I don’t think multi-spec behaves exactly like that. I believe there is no default to be had.
2016:07:05 21:00:54 pdlug That’s unfortunate. Is there a better way to handle the validation where a field in the map determines the type and you want to explicitly constrain some types but provide a default base?
2016:07:05 21:01:40 pdlug Ex: modeling analytics events, the :type provides the type of event, for :search events :terms are required, for :page-view events :url is required but for all other events there are no other constraints on the map
2016:07:05 21:02:39 kidpollo It seems it only dispatches on key defined in s/multi-spec not on the this
2016:07:05 21:05:01 kidpollo hmmm but I just tried (s/def :foo/bad-entity (s/multi-spec entity :does-not-exist))(gen/sample (s/gen :foo/bad-entity) 1) and the program goes into an infinite loop
2016:07:05 21:06:32 kidpollo wait! not infinite loop I just get ExceptionInfo Couldn't satisfy such-that predicate after 100 tries.
2016:07:05 21:07:45 kidpollo makes sense it either matches a spec or it does not. I am back to my original argument 😛
2016:07:05 21:08:36 pdlug I think it’s unexpected behavior if multi-spec works w/ multimethods but then doesn’t support the default behavior of them
2016:07:05 21:08:40 kidpollo @pdlug: yah I had the same issue I want to dispatch on type but it has to be based on a key on your map 😞
2016:07:05 21:08:57 pdlug I still don’t see anything else that supports my example use case
2016:07:05 21:11:49 kidpollo yeah, my problem is that I want to dispatch on something else than a namespaced key on the map. I want to dispatch on meta from the map. Not always you want to expose the “type” on the actual entity
2016:07:05 21:47:12 alexmiller @pdlug you might also see if s/merge helps you in combining base+specific s/keys
2016:07:05 22:03:08 kidpollo Yeah but you would be able to have a generic event spec for all the other types.
2016:07:06 12:23:13 ikitommi Is there a easy way to select a conformer based on external input? E.g. for the web - conform strings to numbers only for query parameters, not for json params.
2016:07:06 14:51:41 pdlug @alexmiller: I’m not familiar with the clojure bug process, that issue CLJ-1935 says the status is “Triaged” what does that mean exactly? Is there a way to tell if it will be addressed or slated for a release?
2016:07:06 14:57:50 artemyarulin hi, what is the right way to enable spec checks globally for my project? I would like that by default everything got checked. I guess I can put something like that in project file:
:repl-options {:init (do (clojure.spec/instrument))}
and then everything should be checked when I’m in a REPL? Oh what’s the right way?
2016:07:06 14:59:33 pdlug @petterik: ah that’s very helpful. Thanks!
2016:07:06 15:53:57 alexmiller @pdlug in this case, it means “I think Rich should look at it” but he has not yet
2016:07:06 16:23:27 pdlug @alexmiller: ok, thanks
2016:07:07 01:50:45 seancorfield I thought alias had been updated so you didn’t have to have a real namespace for the second argument?
2016:07:07 01:52:40 seancorfield It seems not. I was hoping to use alias to shorten a lot of qualified keywords (without having to actually load a namespace).
2016:07:07 02:13:41 alexmiller no, we’re just considering that
2016:07:07 02:49:13 seancorfield Now we’re beginning to use namespace-qualified keywords a lot more, I can say that would be a very convenient addition! 🙂
2016:07:07 03:37:49 seancorfield That’s an interesting hack! 🙂
2016:07:07 06:07:28 mandragoran I was wondering how I would spec an async function (a function returning a core.async channel). Would I need to write an async version of fdef or is there another way?
2016:07:07 08:19:08 wagjo I'm a bit struggling to choose between :foo.vocabulary/term-uri and :foo.vocabulary.term/uri or just :term/uri. I know it's probably a bit soon, but any best practices for choosing the name(space)s?
2016:07:07 09:14:21 mandragoran @wagjo: I've been taking advantage of the new ::alias/some-keyword feature. So I'd probably call it ::term/uri and map the term alias in all namespaces using it to foo.vocabulary.term.
2016:07:07 10:13:11 vikeri How can I reference the first argument of a function in fdef :arg when the argument does not have a name since I’m destructuring it immediately with {:keys [a b]}?
2016:07:07 10:48:12 vikeri nvm, realized the keywords in :arg (s/cat are not tied to the variable names
2016:07:07 11:07:30 alexmiller @wagjo if you're in a lib or other public project, you should add enough context to be distinguishable from anyone else (something like your maven group/artifact id). If in a private app, I would probably do less just for typing sake. I think it will be useful to put specs for the domain in a small number of namespaces (maybe 1). It might be useful to separate generic specs from domain attributes (which alias them) too.
2016:07:07 11:10:42 alexmiller @mandragoran: we will probably have some async related specs eventually. Until we do I think it's better to do type checks on ReadPort and WritePort than specifically on ManyToManyChannel as those are more generic
2016:07:07 12:45:07 lvh How do I write a spec for a map of unqualified keywords where the keyword itself is something dumb that’s going to collide a lot? In particular, swagger (the api format) has a toplevel key, also called “swagger”.
2016:07:07 13:08:23 alexmiller can you give an example? not getting the question.
2016:07:07 14:06:04 lvh Sure! Let’s say I’m parsing swagger
2016:07:07 14:06:36 lvh
(def sample-swagger
{:swagger "2.0"
:info {:title "my sample api for puredanger"
:version "elventy flive"}})
2016:07:07 14:07:50 lvh Let’s say I’m in a ns defining swagger in general (doesn’t matter much, could also be :swagger/whatever). I’m doing something like:
(spec/def ::swagger
(spec/keys :req-un [:swagger/swagger
::info]))
(spec/def :swagger/swagger
;; The top-level key is called swagger. This feels dumb.
#{"2.0"})
(spec/def ::info
(spec/keys :req-un [::api-title
::api-version]))
(spec/def ::api-title string?)
(spec/def ::api-version string?)
2016:07:07 14:08:44 lvh It seems like I’m naming :swagger/swagger what it is because :req-un expects to find the names of map keys that way, but having a ::swagger and a :swagger/swagger looks confusing
2016:07:07 14:09:32 lvh Similarly, ::api-title actually needs to be : or something, but the tons of ns’es make it annoying to type
2016:07:07 14:09:50 lvh I guess I’m asking if there’s a way to distinguish between the key under which something appears in a map, and its spec name
2016:07:07 14:10:30 lvh IIUC there are some new reader macros in the current alpha, I don’t know if they help with this
2016:07:07 14:10:53 lvh they might limit the typing to just the spec defs so at least the destructuring looks good, I suppose?
2016:07:07 14:12:09 lvh if I had a choice, in this code, I would refer to that specced value (esp. after conform) as ::api-title — the only reason I wouldn’t do that is because the JSON I’m getting from swagger (a standard, so I can’t change it) uses keys that don’t collaborate well with that
2016:07:07 14:13:07 lvh If I could provide, say, an alias, or say that under key :title I’m expecting to see spec ::api-title or something, that’d help too
2016:07:07 14:16:20 lvh bailing out and using a fn to just assert that some keys are some particular value doesn’t feel very idiomatic either
2016:07:07 14:33:23 lvh I guess you can define specs in terms of other specs? That doesn’t seem to help a lot but I’ll play with it
2016:07:07 15:40:20 donaldball Has anyone run across or is writing a lib of commonly useful strong specs, e.g. for urls, hostnames, unix ports, valid sql identifiers, etc.?
2016:07:07 15:52:18 alexmiller I haven't seen anyone doing so yet but it seems like a reasonable and perhaps inevitable thing to do
2016:07:07 16:40:35 alexmiller @lvh aliasing is a good and useful thing, so creating api-title and api-version are just fine as base specs (you may want to use some other namespace though like :swagger/api-title or :swagger.api/title), then s/def code-localized versions to match the unqualified attributes that alias those base specs, like (s/def ::title :swagger/api-title)
2016:07:07 16:41:21 lvh OK, so I define the “real” spec under :swagger.ap/title, and then a local alias. Makes sense. Thanks!
2016:07:07 16:42:26 alexmiller if you create the keyword namespace as an actual namespace, you can alias it, then use the alias in the autoresolved keyword like ::sw/api-title (after something like (alias ‘sw ‘swagger.api))
2016:07:07 16:44:22 alexmiller switch to (and thus create) the namespace, then switch back, and alias
2016:07:07 17:32:38 leongrapenthin Today we wanted to create a spec that validates a percentage in five percent steps. It occured to us that int-in doesn't support a step argument like range does. Has this been discussed or is a ticket worth a shot?
2016:07:07 17:39:54 sparkofreason Before I get too far in reinventing the wheel, is anyone else out there working on converting spec to Datomic/Datascript schema?
2016:07:07 17:48:57 alexmiller @leongrapenthin: no, it's not going to do that
2016:07:07 17:49:08 leongrapenthin @dave.dixon: I have thought about it and realized that generating a spec from Schema is likely more desirable
2016:07:07 17:50:05 donaldball I’m not sure what I’m doing wrong with this simple spec:
2016:07:07 17:55:07 leongrapenthin @alexmiller: Yeah, but that's gonna be quite the error message 🙂 - We came up with (s/and (s/int-in 0 101) #(zero? (mod % 5))) if I remember correctly. Generator worked. Projecting this scenario on per permille or larger spaces was what made us wonder why a step is not supported.
2016:07:07 17:56:10 alexmiller because it’s a min/max check, and you’re specifying enumerated values. that’s just not what it is intended to do.
2016:07:07 17:56:38 alexmiller I think your spec is better anyways
2016:07:07 17:57:01 alexmiller you could combine int-in with it too
2016:07:07 17:57:15 alexmiller nvm, you’re doing that
2016:07:07 17:58:06 leongrapenthin Yeah, the reasoning makes sense
2016:07:07 17:59:17 alexmiller the hard part of int-in is generating uniformly across a well defined set and you’re leveraging that by building on top of it
2016:07:07 18:00:10 leongrapenthin Looking at that from a different perspective it is only enumerated because of the min-max check
2016:07:07 18:00:34 leongrapenthin You could argue that everything in a min-max interval is enumerated then
2016:07:07 18:01:48 alexmiller well enumerations have obvious usage issues when there are a lot of values :)
2016:07:07 18:04:19 leongrapenthin My case is a very specific enumeration and can't be discarded as just any enumeration. Every step range of allowed values is a subset of an integer range with the same boundaries.
2016:07:07 18:05:13 leongrapenthin So from that perspective I think it still makes sense as a feature of a min/max check.
2016:07:07 18:05:43 leongrapenthin The min/max check is also an enumeration with a step of 1, then.
2016:07:07 18:14:53 leongrapenthin Better said, for our usecase it would be preferable if it was an in-range check than a min/max check with the word range meant in the same way as in clojure.core/range
2016:07:07 18:17:21 alexmiller well, range does a lot of stuff (reverse ranges, infinite ranges, half-open ranges, support for all number types, etc)
2016:07:07 18:18:23 alexmiller in-range implies to me a check that would need to enumerate the range to determine conformance and gen would be a lot harder.
2016:07:07 18:18:37 alexmiller int-in is intentionally much narrower
2016:07:07 18:25:36 leongrapenthin Agreed. I think what I want to say is that int-in as a name doesn't necessarily limit the spec to a min/max check
2016:07:07 18:27:45 leongrapenthin A fully featured in-range is likely an overkill for the rare amount of usecases
2016:07:07 18:44:02 leongrapenthin Thanks for discussing. Having thought about it I won't be able to resist to submit a small patch over the weekend for reconsideration, though 🙂
2016:07:07 18:46:10 alexmiller I think Rich is unlikely to ok that
2016:07:07 18:48:51 kidpollo I am running stest/check on an fdef that never finishes. It just keeps going and I don't know If its stuck or what is the deal. I have independently tested the input and output specs for possible problems and am getting nowhere. Any tips on how to debug this?
2016:07:07 18:58:12 kidpollo exercise-fn seems to work fine
2016:07:07 19:02:30 leongrapenthin @kidpollo: Isolating the problem by starting with dumbed down specs and adding until the problem occurs?
2016:07:07 19:02:44 kidpollo I also tried (s/exercise-fn my-fn 100) and it takes like 1 min but it works.
2016:07:07 19:03:25 leongrapenthin @kidpollo: Probably recursion in s/map-of or every-kv?
2016:07:07 19:04:52 leongrapenthin Might be that it just takes so long to print the result because it prints a huge generated value
2016:07:07 19:05:10 kidpollo Could be.
2016:07:07 19:05:24 kidpollo I am running on the repl
2016:07:07 19:05:31 kidpollo on intellij
2016:07:07 19:06:06 leongrapenthin You could eliminate this quickly with setting print-length
2016:07:07 19:07:18 alexmiller dump stack and see what it’s doing
2016:07:07 19:07:57 alexmiller I think intellij has a button for it
2016:07:07 19:08:19 kidpollo ohh let me see
2016:07:07 19:10:47 alexmiller I guess maybe that’s only in the debugger mode
2016:07:07 19:14:30 alexmiller will serve in a pinch
2016:07:07 19:16:06 kidpollo yah I was able to dump them in debug mode
2016:07:07 19:16:33 kidpollo just see a bunch of locked threads in generators
2016:07:07 19:18:11 kidpollo damn 700% cpu usage
2016:07:07 19:21:35 kidpollo I guess i need to find what spec is going crazy. Start with simpler specs maybe? The input and output to this fn are multispecs. The maps are not so complicated maps.
2016:07:07 19:47:51 kidpollo @alexmiller @leongrapenthin the problem seems to be happening in alpha-9. I went back to alpha 8 and the test finishes. It takes a while but it finishes
2016:07:07 19:57:44 alexmiller I think check changed from 100 to 1000 iterations in alpha9
2016:07:07 20:20:00 kidpollo oh my that would explain it
2016:07:07 20:20:38 kidpollo when it produced errors it did end earlier
2016:07:07 20:21:11 kidpollo I guess Ill leave it over night to see how long it actually takes
2016:07:07 20:40:19 alexmiller That count is an option you can pass to check though iirc
2016:07:07 21:12:02 kidpollo what is iirc?
2016:07:07 21:13:48 jr (if I recall correctly)
2016:07:07 21:17:52 kidpollo nice! snme in spanish 😛 (si no me equivoco)
2016:07:07 21:18:14 kidpollo lol ^ totally does not exist in spanish 😛
2016:07:07 21:26:32 kidpollo It seems it would look like:
2016:07:07 21:28:01 kidpollo Is it encouraged that all clojure projects start using fully qualified keywords?
2016:07:07 21:47:22 alexmiller Nothing categorical. Qualified keywords have utility - if they make sense for you, use them
2016:07:07 21:48:05 alexmiller I think spec and the other syntax changes have increased their utility and decreased their verbosity, maybe changing that equation
2016:07:07 21:52:13 danielcompton And their visibility 🙂
2016:07:07 21:52:58 alexmiller Well I don't think that changes their utility :)
2016:07:07 21:53:01 seancorfield I can see us (World Singles) using qualified keywords very heavily as it will make some of our code much more explicit. Hence the recent changes to java.jdbc to make it easier to get qualified keywords back in result sets (the new :qualifier option to everything that produces result sets).
2016:07:07 22:43:50 ajss In spec/fdef, is there any way to make one element of :args depend on another? I've got a function that takes two maps, but wants the keys of each map to be the same.
2016:07:07 22:55:03 kidpollo you can define the relationship between inputs and outputs in :fn
2016:07:07 22:55:50 glv Right, but not constrain inputs based on relationships.
2016:07:07 22:56:36 glv (For example, I have functions that take a grid and a coordinate within that grid, and obviously the coordinate must be within the bounds of the grid to be valid. But I don’t think there’s a way of expressing that.)
2016:07:08 00:49:31 donaldball Would very much appreciate feedback on design, etc. It’s my second day using spec in anger, as it were.
2016:07:08 01:25:37 olivergeorge it'd be nice if there was a way to modify how instrument reports. the default spec-checking-fn can be super verbose when turning values to str for the ex-info message.
2016:07:08 01:26:49 olivergeorge I rely on cljs-devtools to make the console.debug output to be expadable
2016:07:08 02:07:41 alexmiller @glv I have much to report re the “hybrid map” stuff, but not quite ready yet to do so
2016:07:08 02:08:08 alexmiller have been working on it much of yesterday and today
2016:07:08 02:11:09 alexmiller @olivergeorge: you have explain-data ex-data coming back in the exception - why not write a function that customizes output from that, rather than hacking spec-checking-fn?
2016:07:08 02:13:28 glv That tells me that it's a legit problem that y'all are taking seriously and working on, so I'm cool with that. :-)
2016:07:08 02:16:57 bbloom @glv: There some considerable advantages to just using another map. I’m curious what @alexmiller is working on, but still: you should just add a :grid key or similar
2016:07:08 02:17:21 bbloom what if you want to be able to represent a board without a starting coordinate?
2016:07:08 02:17:40 bbloom you can easily take a grid and merge in some extra options, but how easy is it to dissoc them?
2016:07:08 02:18:05 bbloom what if you want to change the representation? now you’ve coupled your param spec with your grid spec
2016:07:08 02:18:51 bbloom what advantage is there to abusing the fact that keywords and grid coordinates are disjoint? slightly shorter syntax? seems not worth the complexity
2016:07:08 02:18:56 glv This isn't the representation of the grid; it's the result of a particular kind of analysis of the grid, which always, inherently has a starting coord.
2016:07:08 02:19:38 bbloom composite structures come in two flavors: homogenous and heterogenous
2016:07:08 02:19:52 bbloom of course if you have disjoint key sets, you can combine them
2016:07:08 02:19:56 bbloom but it’s just clearer to keep them separate
2016:07:08 02:20:38 bbloom also when combined, you can’t enumerate keys without filtering extra keys
2016:07:08 02:20:40 glv I'm certainly willing to entertain arguments that I should push the coord->distance mapping down a level.
2016:07:08 02:20:59 bbloom i guess i’m making such arguments 😛
2016:07:08 02:21:13 bbloom but more importantly: what advantage is there to flattening it?
2016:07:08 02:21:21 glv So far, I don't find them convincing. :-)
2016:07:08 02:21:49 glv I see three advantages.
2016:07:08 02:21:58 alexmiller @glv totally legit - I have a couple different solutions with existing tools and something new for just this problem that may help if it comes together
2016:07:08 02:22:52 bbloom i’m not saying hybrid maps are never useful
2016:07:08 02:23:09 bbloom i’m saying that it’s pretty rare and barring additional info, i don’t think you’ve found a justified use case 🙂
2016:07:08 02:25:01 glv 1: the coord->distance mapping is the primary result of the algorithm. The ::start-coord, ::max-coord, and ::max-distance entries are annotations that could be reconstructed (with some cost) if they weren't present. But the only way to make a non-hybrid map would be to leave the secondary annotations at the top level and push the primary info down into a nested map. That doesn't seem right to me.
2016:07:08 02:26:08 bbloom you can return those keys + a ::grid key and then create a helper function which simply extracts the grid key — then the caller can decide what is “primary” or not
2016:07:08 02:26:50 glv 2: the code that's involved—both the code that builds the map and that which uses it—would get more complex and clumsy (and slower) that way
2016:07:08 02:27:37 bbloom really? how?
2016:07:08 02:29:13 glv 3: That's the way it is now, and it has never caused a hint of a problem; the only reason to change would be to accommodate clojure.spec.
2016:07:08 02:29:41 alexmiller I don’t think it’s that rare/weird
2016:07:08 02:29:59 alexmiller others (including me) have run into same problem
2016:07:08 02:30:41 glv How? Isn't that obvious? The vast majority of the updates and accesses will be to the coord->distance mapping, so pushing that a level deeper means extra lookups in the vast majority of cases.
2016:07:08 02:31:04 bbloom alexmiller: any examples you can share? i don’t think i’ve ever encountered such a hybrid map that didn’t (surprisingly quickly) need a (->> hybrid keys (filter #(… that could just be replaced by (-> hybrid :foo keys)
2016:07:08 02:31:22 bbloom @glv: just build the grid and then assoc the extra keys in at the end
2016:07:08 02:31:47 alexmiller sure, I ran into it with map destructuring (so, a syntax example) - the base map is symbol->any but also special options like :keys, :syms, :or, :as etc
2016:07:08 02:32:17 bbloom alexmiller: heh, yeeaaaah i was wondering if you were going to mention macros. destructuring is like the ultimate core macro 😉
2016:07:08 02:32:31 alexmiller it’s fun
2016:07:08 02:32:47 bbloom @alexmiller it’s funny b/c syntax is the one place where “just a little bit shorter syntax” is totally worth it for a common operation
2016:07:08 02:32:57 glv @bbloom: even if that were sensible (it's not) it would only help on the creation side of the process, not the use side.
2016:07:08 02:34:17 bbloom (:grid (let [start 123, m …build up m here….] {:start start :m m}))
2016:07:08 02:34:25 glv It's not sensible because keeping track of those values through the process (so that you could assoc them in at the end) would be clumsy compared to just tracking them in the map as I go.
2016:07:08 02:34:26 bbloom seems pretty sensible to me 🙂
2016:07:08 02:35:02 bbloom er i meant: :grid m
2016:07:08 02:35:05 glv ::start-coord is predefined; the other two emerge out of the process itself.
2016:07:08 02:35:14 bbloom gotcha
2016:07:08 02:35:19 bbloom i’d use mutation 😉
2016:07:08 02:37:04 bbloom @alexmiller: one thing i’ve discovered tinkering on language design: it turns out that language designers have to do the things that they tell other people not to do, heh
2016:07:08 02:37:25 alexmiller that is a thing
2016:07:08 02:38:24 glv cf. Stu's recent tweet about why clojure.core can't be idiomatic.
2016:07:08 02:41:08 bbloom anyway, @glv definitely wait to see what alex et al come up with. the emitting things during the middle is IMO almost a reason to abuse the key space 🙂
2016:07:08 02:41:55 glv "abuse" :-)
2016:07:08 02:42:38 bbloom i’m not above loaded language 😛
2016:07:08 02:43:00 bbloom thanks for the discussion! gotta run
2016:07:08 02:43:11 glv Sure!
2016:07:08 06:41:10 olivergeorge @alexmiller thanks for the suggestion. I would love to do that but don't know how. I presume you are suggesting a big try/catch around my code... my code is typically om/re-frame UI stuff which complicates that.
2016:07:08 14:01:13 donaldball Just to confirm, we don’t yet have a way to unite the specs for e.g. homogenous (`map-of`) and heterogenous (`keys`) maps?
2016:07:08 14:07:55 glv Which isn't bad at all, but could be made more expressive.
2016:07:08 14:14:03 alexmiller no, that’s bad :)
2016:07:08 14:14:45 alexmiller b/c it turns it from a declaration of truth into a process
2016:07:08 14:15:41 alexmiller there are a couple of ways to do it with existing tools
2016:07:08 14:18:30 alexmiller one other way is to treat the map as a coll-of s/or of s/tuples (where each tuple describes different kinds of map entries). For cases where the “hybrid”-ness does not include registered attributes, this is probably a good choice. So something like a map that was either string->string or number->number.
2016:07:08 14:19:17 alexmiller you can handle registered attributes in the same way, but then you lose the goodness of s/keys and the attribute semantics
2016:07:08 14:20:03 alexmiller so in that case, you want to s/merge an s/keys (for the registered attributes) with something that more accurately states the truth of the map definition
2016:07:08 14:20:30 alexmiller sorry for not being detailed here - I hope to write a blog with the detail
2016:07:08 14:21:14 alexmiller and then I’m working on something new that will make this a little more palatable, but it’s not quite done yet
2016:07:08 14:23:53 glv Well, compared to my attempt at a solution, it’s not bad. 🙂
2016:07:08 17:29:07 eggsyntax In 1.9, is there a way to disable namespaced-map output (eg for data that's going to be parsed by a process that's unaware of the idiom)? ie to get output like {:foo/bar 1} rather than #:foo{:bar 1}? [Edit:simplify example]
2016:07:08 18:12:00 donaldball Are double-in and friends macros so that clojure core doesn’t require test check or as a performance optimization?
2016:07:08 19:56:47 donaldball exercise-fn is pretty rad, but correct me if I’m wrong, it looks like if the fn in question has an options map specified with s/keys using :opt or :opt-un, the chances of getting a map with those keys is… small
2016:07:08 19:57:21 donaldball Should the generator for s/keys bias towards optional keys’ presence?
2016:07:08 20:08:25 jr I think they are macros because they build on other macros (like spec)
2016:07:08 20:11:35 stuarthalloway @donaldball: double-in is a macro to capture the form for error reporting, IIRC
2016:07:08 20:18:35 alexmiller in general, all of the spec creators are macros to capture forms
2016:07:08 20:20:29 donaldball I’m afraid the implication is not clear to me 😕
2016:07:08 20:26:48 alexmiller by using let from test.check, you’ve also made test.check a production time dependency for you
2016:07:08 20:27:00 alexmiller (which is a bummer given how nice let is - I keep wanting it too)
2016:07:08 20:29:24 alexmiller but you can easily rewrite that with gen/fmap
2016:07:08 20:29:33 donaldball Yeah, I plan to replace it with fmap … jinx
2016:07:08 20:30:30 alexmiller double-in as a macro (and spec under it) allows the form of the predicate to be captured as a form as well as evaluated as code
2016:07:08 20:31:39 donaldball Thanks, that makes good sense
2016:07:08 20:31:59 donaldball Btw any opinion on my optional keys generator observation?
2016:07:08 20:34:17 alexmiller did you maybe not generate enough to tell?
2016:07:08 20:44:04 alexmiller it’s a composition of &, a conformer, and keys
2016:07:08 23:15:10 ajss not sure if others are interested, but I've figured out the nice way to do dependent types for spec/fdef arguments : use with-gen on the :args parameter
(defn sum [a b] (+ a b))
(spec/fdef sum {:args (spec/with-gen (spec/tuple int? int?) (constantly (gen/fmap (fn [x] [(* x 2) (* x 3)]) gen/int)))})
2016:07:09 01:23:06 donaldball I’m kinda wishing s/def allowed a docstring. The generated docstring is okay, but I think I’d like to be able to communicate the intent also.
2016:07:09 02:01:30 donaldball Weirdly, I had a good test running in an earlier repl with stest/check but now it’s raising
2016:07:09 02:01:40 donaldball
Caused by: java.lang.ClassCastException: clojure.lang.AFunction$1 cannot be cast to clojure.lang.MultiFn
2016:07:09 14:35:00 robert-stuttaford so we have s/merge to make a single s/keys out of two other s/keys. i'm wondering, how do i spec a map that has some base set of req+opt keys, but also has an optional set of keys that must appear together as if speced with s/keys :req ... ?
2016:07:09 19:38:34 arohner has anyone seen java.lang.IllegalArgumentException: No implementation of method: :conform* of protocol: #'clojure.spec/Spec found for class: clojure.spec$spec_impl$reify__13361
2016:07:09 19:38:41 arohner I’m doing something wrong, but I haven’t found it yet
2016:07:09 19:40:47 arohner hrm, looks like I needed to clean after upgrading clojure
2016:07:09 20:50:41 kendall.buchanan @donaldball: I’ve been experiencing the same with check-var
2016:07:10 12:17:15 jjcomer @donaldball: I get that error when I execute in a deftest.
2016:07:10 15:12:27 donaldball I get it both when executing in a deftest and in cider, though it works fine from a repl
2016:07:10 15:16:42 donaldball I was just experimenting with ignoring Rich’s advice and expressing specs for strings as spec regexes. I find I actually quite like a couple of things that fall out: the form is easier to understand that the string regex, and you get a generator for free.
2016:07:10 15:19:26 donaldball But unless I’m missing something, you pretty much have to write one spec for the seq of chars and another spec for the string that unwraps it as a seq
2016:07:10 15:20:24 donaldball Setting aside the merits of the idea, is there a way I could do with s/and and a conformed value?
2016:07:10 18:25:54 craigy hey all, I'm trying to figure out if I can use spec for parsing strings (sentences) into structured data. I'm struggling with whether or not I will need to split around spaces, and how to handle things like phrases (sequences of words) if I use s/cat. is this kind of use case intended at all, or am I better off with something else?
2016:07:10 19:02:02 misha not-sure-fry
2016:07:10 19:07:10 xcthulhu How do I write my own generator for cljs.spec?
2016:07:10 19:07:26 xcthulhu I have data structures like {0 0.0, 1 0.1, 2 0.2, 3 0.3, 4 0.4, 5 1.0}
2016:07:10 19:07:41 xcthulhu (maps of integers from 0 to 5, increasing in value as the index increases, where one of the values must be 1.0 and all of the values must be within [0,1])
2016:07:10 19:08:06 xcthulhu I can generate these things with this function:
2016:07:10 20:27:08 xcthulhu I'd rather not rope in test.check. Too bad that clojure.spec doesn't let you register generators separately from specs
2016:07:10 20:28:01 xcthulhu Because I wouldn't mind associating a generator with my specs in my unit tests where I do use test.check
2016:07:10 20:34:16 xcthulhu I think you can do this with just just spec since it gives you 'cat' and 'fmap'
2016:07:10 20:35:28 gfredericks xcthulhu: I think clojure.spec is designed with the intention that you rope in test.check for non-trivial generator needs
2016:07:10 20:35:46 gfredericks is it just that you don't want test.check required in a non-test namespace?
2016:07:10 20:37:27 xcthulhu Yup
2016:07:10 20:38:25 xcthulhu But like I said, they do give you enough so you don't need test.check, it's just clumsy
2016:07:10 20:47:55 gfredericks a custom var deftype
2016:07:10 20:59:00 xcthulhu Yeah, it's pretty hacks. Anyway, if I read this right they lazily load vector from test.check. Sadly, clojureScript doesn't give you double*, only double
2016:07:10 21:00:30 xcthulhu Anyway, thank you I see how to do this crazy thing now
2016:07:10 22:00:48 xcthulhu Here's the generator I ended up going with BTW:
2016:07:10 22:03:16 xcthulhu Not with high probability, but I'll go add a restriction on to the such-that
2016:07:10 22:03:30 gfredericks should be high enough to matter
2016:07:10 22:03:33 gfredericks unless I screwed something up
2016:07:10 22:04:27 gfredericks roughly 1% of the time
2016:07:10 22:12:10 xcthulhu Here we go:
(defn nan?
"Tests if a value is a NaN or not"
[x]
#?(:clj (and (double? x) (.isNaN x))
:cljs (js/isNaN x)))
;;;;;;;;;;;;;;;;
(gen/fmap
(fn [values]
(let [max-value (apply max values)
min-value (apply min values)]
(->> values
sort
(map #(-> % (- min-value) (/ (- max-value min-value))))
(drop 1)
(zipmap (range)))))
(gen/vector
(gen/such-that (complement (some-fn #{infinity, (- infinity)} nan?))
(gen/double))
7))
2016:07:10 22:12:35 xcthulhu Hopefully this yak has been shaved by now
2016:07:10 22:17:19 gfredericks :)
2016:07:11 02:22:49 alexmiller There is s/double-in with infinity and nan support
2016:07:11 06:32:42 robert-stuttaford forgive the repost, folks, but i'm a little stumped 🙂
2016:07:11 06:34:04 robert-stuttaford how do i specify that an optional set of keys must either appear together or not at all?
2016:07:11 08:34:54 mandragoran @robert-stuttaford: you can write your own predicate (-> (s/keys :opt [...]) (s/and #(every (partial contains? %) [...]))
2016:07:11 09:04:49 robert-stuttaford true, thank you. i wonder if it's a use-case that has been considered by Rich et al?
2016:07:11 12:51:38 alexmiller Have you looked at the and/or support in s/keys :req?
2016:07:11 12:59:30 alexmiller was thinking something like (s/def ::combined (s/keys :req [(or (and ::a ::b) (and ::a ::b ::c ::d))]))
2016:07:11 13:00:18 alexmiller however b/c of open maps, I don’t think the latter two will fail as expected - for that you probably need and an additional pred
2016:07:11 14:48:43 robert-stuttaford thanks Alex
2016:07:11 17:17:20 semperos I wonder if there’s a place for an s/xor within the clojure.spec space where one, and only one, of the provided specs may conform, or if this can be concisely expressed with a combination of the existing features
2016:07:11 19:31:22 semperos Alex or others, would appreciate your thoughts if/when you have time ^^
2016:07:11 20:55:42 arohner Here’s a fun one. If I start up a repl and clojure.test/run-tests, everything is fine. If I lein test, I get java.lang.Exception: :clojure.spec/unknown is not a fn, expected predicate fn
2016:07:11 20:56:01 arohner AFAIK, I’m not doing anything weird. Everything is required in the proper order, not using load, etc
2016:07:11 21:02:12 arohner Also AFAIK, I’m not referring to any spec that hasn’t been required
2016:07:11 21:25:54 arohner more weirdness: I run my tests from a repl, everything is fine. I run c.spec.tests/instrument, run the same tests, and I get No implementation of method: :conform* of protocol: #'clojure.spec/Spec found for class: clojure.spec$spec_impl$reify__13372. Reload all my source namespaces, everything works again
2016:07:11 22:44:37 alexmiller I have seen some stuff like this with mixture of aot and non aot on classpath
2016:07:12 16:37:48 peeja I'm afraid I'm still failing to understand the difference between keys and keys*. I understand that only keys* works in a cat, but I don't understand why. Is this explained somewhere?
2016:07:12 16:39:59 alexmiller @akiel rich hasn't looked at it yet
2016:07:12 16:41:52 alexmiller @peeja keys is a spec for a map. keys* is a spec for a sequential list of alternating keys and vals
2016:07:12 16:42:11 peeja Ohhhhh!
2016:07:12 16:42:26 peeja So they both work in the same contexts, but match different things
2016:07:12 16:43:03 alexmiller I'd say they work in different contexts to match the same thing :)
2016:07:12 16:43:37 peeja By "context" I mean where they go in a spec
2016:07:12 16:44:08 akiel @alexmiller it’s ok, as long as it’s on your list - thanks
2016:07:12 16:44:28 peeja Yeah, I was misreading the guide. Thanks!
2016:07:12 18:51:25 donaldball Revisiting the “spec strings as char seqs” idea from the weekend, I just threw this together:
2016:07:12 18:51:57 donaldball Bad idea or simply misguided? 😛
2016:07:12 19:16:14 alexmiller @donaldball: from a spec perspective I think it's much better to use an actual regex for matching
2016:07:12 19:17:38 donaldball The things I’m finding attractive about expressing specs on char seqs are that the spec regex forms are often more readable, but more to the point: you get working generators for free
2016:07:12 19:18:19 alexmiller There is a project for assembling readable regexes, can't remember the name
2016:07:12 19:23:35 arohner that doesn’t use java.util.regex.Pattern though
2016:07:12 20:24:26 alexmiller yeah, that one
2016:07:12 20:24:37 alexmiller and also test.chuck has support for generating strings based on a regex
2016:07:12 21:23:00 sparkofreason When writing regex specs, is there a way to get at the bindings "so far"? For instance, say I want to validate a datom [e a v], where a is keyword that represents a spec. I'd like to use that spec to validate v, and get "nice" output from explain that the failing spec is the one represented by a.
2016:07:12 22:04:51 danielcompton @alexmiller: what is the thinking behind it being under consideration? What are the downsides of allowing docstrings on specs?
2016:07:12 22:13:12 sparkofreason @alexmiller: would be nice if there were a way to pass explain results out of custom predicates. Though perhaps not required in most cases.
2016:07:12 22:24:03 alexmiller @danielcompton asking the wrong person :)
2016:07:12 22:25:27 alexmiller @dave.dixon: while it's tempting, I don't think Rich is inclined to support that for uniformity.
2016:07:13 00:00:45 sparkofreason @alexmiller: Anyway, I'd rather see support for this specific case, which seems common in Clojure, where we have a sequence of keywords followed by associated values. Just supporting the case of having the value validated against the spec named by the keyword would cover a lot of ground, as opposed to having a general dependent spec scheme. There's a nice symmetry with the map behavior then.
2016:07:13 00:02:18 alexmiller Actually you could do this with keys* couldn't you?
2016:07:13 00:11:02 seancorfield One thing I’m finding with clojure.spec as applied to user input is that I often seem to need a two-step process: the first is a very basic predicate and a conformer, the second is the real "spec" (and perhaps and conformer).
2016:07:13 00:11:27 seancorfield Example, user inputs a country code and I’d like it to be case-insensitive.
2016:07:13 00:12:48 seancorfield So the first spec is that it’s a two-character string and the conformer is clojure.string/upper-case; then the second spec actually looks up the (now upper-case) two-character string and verifies it’s a valid ISO country code (and could conform it to country information).
2016:07:13 00:13:29 seancorfield Is that somehow a poor use of clojure.spec? Or is there perhaps a better way to deal with input that needs a little "cleanup" before being validated/conformed?
2016:07:13 00:17:10 luke @seancorfield hm. If a non-uppercase string is to be considered “valid” input, then I’d say it’s on your country-code lookup function to handle case insensitive input. That seems like it would be easier than having two specs.
2016:07:13 00:18:21 luke Unless you want to have a spec for your intermediate data format for some reason.
2016:07:13 00:19:56 seancorfield Well, several of the inputs — and country is actually possible here — are sets of strings. For example, ::gender is #{"male" "female"} so it validates and generates nicely, but if you want it to be case-insensitive then you need to do "something" that isn’t necessarily easy with a single spec.
2016:07:13 00:20:46 seancorfield (`::country` could be (set (Locale/getISOCountries)) for example)
2016:07:13 00:23:36 seancorfield I started leaning toward two specs because internally I wanted ::gender #{:male :female} for the domain model and "appropriate strings" for the input and the conformer there would be (comp keyword str/lower-case) … assuming you can validate the input strings with a different spec.
2016:07:13 00:24:23 seancorfield Perhaps I’m making life more complicated for myself by trying to avoid with-gen here...
2016:07:13 00:24:28 luke Yeah… two specs isn’t bad if you do want separate models (one for the domain, one for the input) and are ok spec’ing them separately.
2016:07:13 00:25:46 luke It is also possible in theory to write a case-insensitive helper, which given a set of strings returns a (generating) spec that accepts any capitalization.
2016:07:13 00:30:26 seancorfield Hmm, I’ll have to give it more thought. So far have specs for input fields that conform to the domain model values, and specs for the domain model itself is feeling like the right way to go, as it allows me to generate data for both layers: API (input) and "system" (domain model).
2016:07:13 00:32:54 seancorfield (we’re pushing very hard to tease apart what is some rather complected logic in our current codebase — we currently do a sort of transform-validate-update operation on each field and we’ve figured out a nice clean way to separate validate from update — with a view to pushing the updates out to a queue and applying them asynchronously — but that pre-validation transform on a few fields is proving problematic 😸 )
2016:07:13 00:33:59 seancorfield Thanks for the input so far @luke
2016:07:13 00:34:27 luke yep if you want separate layers, and you’ve fully reified both layers, then specs for each layer seems like the way to go!
2016:07:13 00:34:55 luke no hard answers, just what comes off the top of my head when I hear the question 🙂
2016:07:13 01:05:21 sparkofreason @alexmiller: Thanks, that works great. Knew there had to be a way to do that, but had my head stuck in regex-land...
2016:07:13 03:17:58 lvh Is there a way to specify “strings that match a particular regex” in a way that produces the correct generator?
2016:07:13 03:18:10 lvh (test.chuck I think has that)
2016:07:13 03:20:09 lvh it seems like test.check would have a real bad time generating strings until they accidentally match that regex
2016:07:13 03:20:24 lvh (that also seems like a general problem, which is why I’m assuming there’s a general soltuion)
2016:07:13 03:23:39 gfredericks the test.chuck generator could be moved to test.check proper if somebody wants to rewrite the instaparse regex parser in pure clojure
2016:07:13 03:24:10 lvh What is it now? Bunch of Java?
2016:07:13 03:24:27 lvh Also hi gfredericks burning the midnight Chicago oil too I see 🙂
2016:07:13 03:24:51 lvh I haven’t really read the spec source code any maybe I should just go do that
2016:07:13 03:25:07 lvh I’m presuming there’s a way to map predicates to generators
2016:07:13 03:26:50 lvh that doesn’t sound very fun
2016:07:13 03:27:31 lvh how would you feel about clojure.spec-specific things being added to test.chuck?
2016:07:13 03:27:53 lvh say, a spec that understands how to map to generators in test.chuck, like, say, hypothetically, if someone wanted to match a regex 😉
2016:07:13 03:28:43 gfredericks I keep thinking I'm going to want a similar clojure.spec utility library
2016:07:13 03:29:15 gfredericks clojure.schpec
2016:07:13 03:29:57 lvh 💯
2016:07:13 03:30:03 lvh would depend upon.
2016:07:13 03:31:23 lvh I guess I’m screwed anyway because this is a keyword, not a string, so unless there’s something that understands keywords, I’m going to manually assign a generator
2016:07:13 03:32:03 gfredericks I get the impression that manually assigning generators is not meant to be too rare
2016:07:13 03:32:38 lvh same here
2016:07:13 03:33:13 lvh I’ll defer to you since you obviously have a lot more experience here, but just generating samples based on only predicates is going to break really badly really quickly
2016:07:13 03:33:50 lvh (basically as soon as the probability of the predicate passing is not very big)
2016:07:13 03:34:02 gfredericks yeah I don't think filtering on predicates is expected to be a very useful approach
2016:07:13 03:34:55 gfredericks clojure.spec has that behavior in some places because there's not any other reasonable default, except maybe just refusing to try
2016:07:13 03:35:33 lvh yeah, fortunatelyi t’s mostly just for dev and test that you do that
2016:07:13 03:36:03 gfredericks mostly?
2016:07:13 03:36:08 lvh so it’s fine if the generator just shrugs and gives up after n consecutive failures
2016:07:13 03:36:25 lvh I have used test.check in other directions 🙂
2016:07:13 03:36:36 lvh technically still a test I guesS?
2016:07:13 03:36:50 lvh specifically, I generated sequences of events/actions on openstack (rackspace public cloud)
2016:07:13 03:37:54 lvh so it found things like “it gets unhappy if there are two expected networks these servers should be attached to and one of them disappears"
2016:07:13 14:08:18 alexmiller First in a series of spec screencasts ^^
2016:07:13 15:14:59 peeja Is it possible to match data against a spec to find the paths at which a particular spec is used?
2016:07:13 15:15:30 peeja That is, I want something like conform, but rather than look things up by tag, I want to look things up by spec keyword
2016:07:13 15:15:46 peeja (and find the tag-based paths to those parts)
2016:07:13 15:16:25 peeja "Find me every Foo in this data structure"
2016:07:13 15:38:54 alexmiller I’d say no, nothing like that. might help back up to the problem you’re trying to solve though
2016:07:13 15:49:59 peeja Yeah, I think I'm actually down the wrong path here. (But it's hard to tell, since I haven't entirely decided on what I'm trying to do.) 🙂
2016:07:13 15:57:51 alexmiller that’s usually a good first step :)
2016:07:13 17:51:41 joshmiller A question I posed originally in #C03S1KBA2: I’m having some trouble understanding how keywords work with respect to clojure.spec… Why is it that (s/def :spec-test-ns/name string?) works, but (s/def (keyword “spec-test-ns” “name”) string?) fails with CompilerException java.lang.AssertionError: Assert failed: k must be namespaced keyword or resolvable symbol? The type of both is clojure.lang.Keyword.
2016:07:13 18:06:30 semperos this is the assert that is being tripped @joshmiller : (c/assert (c/and (named? k) (namespace k)) "k must be namespaced keyword or resolvable symbol”)
2016:07:13 18:10:21 semperos (I know not fully helpful, sorry)
2016:07:13 18:10:37 joshmiller @semperos: Yeah — but (instance? clojure.lang.Named (keyword "spec-test-ns" "id”)) is true and (namespace (keyword "spec-test-ns" "id”)) is "spec-test-ns”, so I don’t understand how that assert is failing.
2016:07:13 18:12:08 joshmiller I’m digging into what happens with macroexpand on s/def to see if maybe that’s the issue.
2016:07:13 18:14:14 semperos and a List is not a named thing, note the ’ in the macro definition of s/def
2016:07:13 18:15:01 joshmiller Yeah, I see (clojure.spec/def-impl (quote (keyword "spec-test-ns" "name")) (quote clojure.core/string?) string?) vs (clojure.spec/def-impl (quote :spec-test-ns/name) (quote clojure.core/string?) string?) when doing it the “right” way
2016:07:13 18:48:14 alexmiller I don’t know that hardly anyone has noticed or used the instrument :stub/:replace functionality, which lets you do some pretty interesting things both for example tests or in tandem with check.
2016:07:13 18:58:29 joshmiller I ended up being able to dynamically generate the keywords for specs in other namespaces by using a macro where I could unquote the keyword generation, but if anyone knows of another way to go about it, I’d love to hear it.
2016:07:13 20:04:11 olivergeorge Is there a standard way to add a generator to complement an instance? spec based on a constructor fn. I imagine you would want a spec/gen for each constructor arg.
2016:07:13 20:17:50 olivergeorge I think the answer is a combination of s/with-gen, s/gen and gen/fmap
2016:07:13 21:08:21 puzzler Watching the screencast about how spec accomplishes multiple things at once got me thinking about how it is difficult to get all the listed benefits simultaneously. For example, if you put your specs in the same namespace as the code, you can't easily make custom generators because it depends on test.check which is generally only a dev dependency. If you put your specs in a separate namespace, you can't rely on predicates defined in your main namespace. I have not yet found a practical way to divide up code involving specs without giving something up. A lot of that has to do with Clojure's limitation/restriction regarding circular dependencies. I hope more info about how to use spec in the context of a real-world project will eventually be provided.
2016:07:13 21:13:41 donaldball You can still make custom generators if you restrict yourself to the vars in clojure.spec.gen
2016:07:13 21:21:55 mike_ananev Is there any recommendation to put spec and fn's in one file or separate them?
2016:07:13 21:23:45 puzzler @donaldball: the guide says the vars in clojure.spec.gen just dynamically link to clojure.test-check, so those vars won't do anything unless you have clojure.test-check listed in your (dev) dependencies. Assuming I understand the guide correctly...
2016:07:13 21:24:58 puzzler @donaldball: Or is your point that the code will at least pass compilation, so it doesn't matter if those vars don't actually do anything? (I haven't tried that myself, so I don't know what happens if you don't have test.check active).
2016:07:13 21:25:19 donaldball The latter
2016:07:13 21:25:44 puzzler @donaldball: Ah, that's helpful to know.
2016:07:13 21:27:39 puzzler @mike1452: Generally, people have been reporting that they have been separating the specs from the implementation, but there are limitations that come with that approach (as far as I can tell).
2016:07:13 21:33:33 mike_ananev @puzzler thanx! may be new screencasts from Cognitect will show us a code arrangement with spec.
2016:07:13 21:38:26 seancorfield @puzzler: We’re mostly keeping specs separate from code. So our spec ns requires our code ns, and any code that wants the specs applied requires the spec ns (and the code ns if needed). That’s how we’ve solved the circular reference issue.
2016:07:13 22:42:45 donaldball Is that mostly a concession to the fact that you’re working in a codebase that needs to support older versions of clojure or an explicit design choice?
2016:07:13 22:43:20 donaldball Where I’m playing with specs, and I spec my fns, I tend to like having them adjacent to the fn definitions
2016:07:13 23:03:01 seancorfield @donaldball: The bulk of the specs are for layers of our domain model — data — so that’s independent of functions, so it made sense to have a separate namespace for them. And then to put the handful of function specs in there too.
2016:07:13 23:03:36 seancorfield And as long as you load the spec namespace into the system before you run the functions that attempt to conform data to those specs, you’re good.
2016:07:13 23:26:56 wilkerlucio I'm choosing now to use they adjacent of the code, like donald is doing, I'm not sure if it's the best way but I'm enjoying it so far, will be nice to see the benefits/problems on the long run as people mature specs on their codebases
2016:07:13 23:28:14 wilkerlucio for those using on a separeted namespace, what name conventions are you using for the specs namespaces? @seancorfield @puzzler
2016:07:13 23:31:02 seancorfield @wilkerlucio: It depends on what we’re spec’ing. We have a set of specs for domain model data (in ns like ws.domain.something where something is the particular domain model "object" being described) and then we have a set of specs for input fields for APIs (in ns like ws.something.field).
2016:07:13 23:31:12 seancorfield But it’s early days and we’re moving code around. A lot.
2016:07:13 23:32:28 seancorfield We also have structures that tie things together (so we have hash maps that describe the relationship between domain model things, database tables and columns, validation and update routines — using qualified keywords that are resolved to symbols later).
2016:07:13 23:36:30 wilkerlucio and for specs about functions, are you using something like: for namespace x.y.z have a x.y.z-specs, something like that?
2016:07:13 23:54:05 seancorfield No, those specs just go in the namespace for the "something" that those functions would (primarily) operate on. They’re fairly coarse-grained domain concepts (for our systems — "member" covers a lot of ground).
2016:07:14 04:40:12 mjmeintjes Hi. Is there a way to use spec to generate maps with "dependent" properties. For example, suppose my map spec requires than both :a and 😛 contain strings that have to be the same length?
2016:07:14 04:42:27 mjmeintjes That should be - Is there a way to use spec to generate maps with "dependent" properties. For example, suppose my map spec requires than both :a and :b contain strings that have to be the same length?
2016:07:14 04:56:52 seancorfield @mjmeintjes: Well, you can use s/and to restrict what a spec generates but that might not be the easiest way to make the generator work...
2016:07:14 05:05:04 seancorfield Hmm, perhaps it's OK after all:
2016:07:14 14:43:52 glv After converting some property-based tests to use spec’s generators, I noticed that some of the tests became much slower. Investigation reveals that the growth characteristics of spec’s generators can be quite different from those of test.check. Here’s an example:
2016:07:14 14:46:19 glv In most cases I doubt the difference would be very noticeable, but in my case two such generated integer values were being used as the dimensions of a two-dimensional matrix structure, so the size really blew up quickly.
2016:07:14 14:49:03 glv I can see a rationale for making the growth a little steeper than the test.check version, but this seems a bit severe. 🙂
2016:07:14 17:04:30 glv Especially since the default for :num-tests is 1000. 😮
2016:07:14 17:24:07 seancorfield I would say, in this case where the amount of data you test is N-squared based on random generated values, you probably want a custom generator that heavily restricts those dimension values?
2016:07:14 17:24:38 seancorfield How big, realistically, is a matrix you would be handling?
2016:07:14 17:26:10 glv Oh, sure … I’ve done exactly that, now. Just wondering whether this is an appropriate default.
2016:07:14 17:27:42 glv Given the actual behavior over 1000 iterations, you’d run into problems even if the amount of data was linear wrt the integer value.
2016:07:14 17:29:06 glv And given that the visible symptom is just “my test run seems to just stall” it’s a bit confusing.
2016:07:14 17:55:16 glv (Oh, and … my first attempt at constraining the generator was just to slap a #(< % 25) predicate on there, and then I got sporadic "Couldn't satisfy such-that predicate after 100 tries” failures. Which was easy enough to fix, but still leads me to think that the generator growth is just too fast.)
2016:07:14 20:19:48 Tim what is the justification of having spec in separate namespaces than the actual functions?
2016:07:14 22:04:13 ghadi I think he'll be back tomorrow Glenn
2016:07:14 22:04:23 ghadi out
2016:07:14 22:04:24 glv OK, thanks!
2016:07:15 12:13:27 vikeri Can I spec an infinite sequence?
2016:07:15 12:14:09 gfredericks Like (s/* ___)?
2016:07:15 12:14:35 gfredericks I suppose the validation is the sticky part
2016:07:15 12:14:37 vikeri Exactly
2016:07:15 12:14:45 gfredericks I think that's what s/every is for, if you need incomplete validation
2016:07:15 12:15:14 gfredericks I guess technically s/* isn't the same thing as infinite
2016:07:15 12:15:19 gfredericks s/∞
2016:07:15 12:16:33 vikeri With s/+ I just got an infinite loop. (It tries to check every element in the infinite sequence I guess) s/every looks promising.
2016:07:15 12:25:07 vikeri s/every did the trick! Thanks!
2016:07:15 12:39:55 vikeri Or, it works to validate it like this (s/valid? (s/every :my/pred) (repeat my-item)) but not inside :args in s/fdef. Any ideas?
2016:07:15 12:55:34 gfredericks do you just mean that it's not validating at all because you haven't instrumented the function? or is it breaking in some way?
2016:07:15 13:08:50 vikeri @gfredericks It goes into an infinite loop when I put it in my fdef and then execute the function.'
2016:07:15 13:09:14 vikeri If I test it in the repl with s/valid? it works fine
2016:07:15 14:33:39 rickmoynihan just starting to play with spec... How would you say "A map must have either the keys :foo or :bar or both (but not neither)"
2016:07:15 14:35:36 rickmoynihan guessing I just need to supply all three combos
2016:07:15 15:42:53 rickmoynihan Am I right in thinking that the 'or symbol there is actually clojure.core/or - and spec interprets the symbol itself as meaning or?
2016:07:15 15:43:05 rickmoynihan ahh yes keys is a macro
2016:07:15 15:43:12 richhickey ‘or’ and ‘and’ are syntax inside keys
2016:07:15 15:43:33 rickmoynihan when I first saw spec, I wondered why you hadn't done that! Super nice!
2016:07:15 15:45:18 rickmoynihan symbols are data too
2016:07:15 15:53:47 stuarthalloway @glv have any more info on test.check fns “ignoring size guidance”? Starting to read the source now, and it looks like we might need to explicitly opt in to sized in some places
2016:07:15 15:55:23 glv A lot of the generators for built-in predicates use large-integer* from test.check, which doesn’t use sized. Perhaps instead use the more specialized things like pos-int, s-pos-int, etc.?
2016:07:15 15:58:12 stuarthalloway @glv I am confused because large-integer* calls large-integer** which does use sized
2016:07:15 15:58:51 glv Hmmm … I thought I traced through all that yesterday.
2016:07:15 16:00:00 stuarthalloway @glv your proposal still may be the right thing, I just want to understand why we are getting the current behavior
2016:07:15 16:01:12 stuarthalloway @glv it looks to me like s-pos-int use size on integers, and large-integer** uses size on bit count
2016:07:15 16:01:42 glv Ah, right.
2016:07:15 16:02:39 stuarthalloway and if we switch to s-pos-int we would never get larger values
2016:07:15 16:02:50 glv That might explain the weird, long-cycle behavior of (s/gen int?) in that example I pasted yesterday.
2016:07:15 16:02:57 stuarthalloway
(apply max (gen/sample gen/int 1000000))
=> 99
2016:07:15 16:03:32 stuarthalloway grr, maybe neither growth pattern is ideal
2016:07:15 16:04:21 glv Right. I think (I’m no expert) that’s why test.check has both. pos-int (and family) for integers that are used as sizes for things, and large-integer for integers used in mathematical functions.
2016:07:15 16:04:55 stuarthalloway whereas spec has specific (int-in et al) and non-specific (pos-int et al)
2016:07:15 16:05:08 glv Yes.
2016:07:15 16:05:34 glv Which is what I switched to.
2016:07:15 16:07:49 glv And the only reason that kind of annoys me is that those are generators only, not specs. It doesn’t seem good to have to specify a custom generator for such a common case. (My hunch, which may be wrong, is that in most systems, functions that build and manipulate data structures are more common than functions that do complex math on integers.)
2016:07:15 16:09:08 stuarthalloway @glv in cases where you don’t specify, I would like to see it grow more slowly, but still grow
2016:07:15 16:09:21 glv Agreed.
2016:07:15 16:09:27 stuarthalloway hitting the hammock on this one
2016:07:15 16:09:52 stuarthalloway to be more honest, grabbing some lunch 🙂
2016:07:15 16:10:22 stuarthalloway but we do have a hammock in the office
2016:07:15 16:10:48 glv OF COURSE YOU DO
2016:07:15 16:19:11 glv @stuarthalloway: Oh wait … of course int-in is a spec, not just a generator. The real issue is that I want a large upper-bound in production, but smaller in tests (because a 100x100 matrix will be much slower with a negligible chance of exposing bugs that wouldn’t be seen in a 20x20 matrix over a bunch of tests. But there’s no reason to restrict it so tightly outside of the tests. (Maybe my case is just weird and special. But I do think the existing growth pattern is surprising, and I strongly suspect that it negatively affects shrinking.)
2016:07:15 17:22:08 hlship So I’m seeing something odd in the interaction of s/every-kv and s/conform.
2016:07:15 17:28:03 seancorfield @hlship: Why not use simple-keyword?
2016:07:15 17:29:55 hlship I can’t find that … do you have a link to docs?
2016:07:15 17:32:03 hlship Anyway, any insight into the s/conform part?
2016:07:15 17:32:39 seancorfield Use s/map-of instead.
2016:07:15 17:32:52 seancorfield I don’t think every-kv conforms the values?
2016:07:15 17:33:26 hlship Thanks for the tip … must have missed that.
2016:07:15 17:35:23 hlship Thanks, just what I needed.
2016:07:15 17:37:45 seancorfield Yeah, just confirmed via the docstrings — map-of "Unlike every-kv, map-of will exhaustively conform every value." — every-kv (based on every) "Note that every does not do exhaustive checking, rather it samples *coll-check-limit* elements. Nor (as a result) does it do any conforming of elements."
2016:07:15 17:38:48 seancorfield (I did not know that before you asked so, thank you, "learn something new every day" 🙂 )
2016:07:15 17:47:42 glv So you can do without ::unqualified-keyword and just use simple-keyword? directly in map-of.
2016:07:15 18:14:11 stuarthalloway @glv so test.check does two flavors of things:
2016:07:15 18:14:42 stuarthalloway 1. built-ins that bottom at choose use the size argument twice, to control the rate of growth and to limit the values reached
2016:07:15 18:16:13 stuarthalloway 2. built-ins that bottom at large-integer use the size argument only once, to control (approximately) the number of bits allowed
2016:07:15 18:17:30 stuarthalloway note the 13 (bits) vs 1000 (value) passed as size to gen/sample-seq
2016:07:15 18:18:53 stuarthalloway spec does not (yet) even provide a path to setting the size argument
2016:07:15 19:55:34 stuarthalloway @glv ignore that crap explanation ^^. Bottom line is we want a generator that will grow nicely with a default max-size, working on it.
2016:07:15 20:29:12 arohner With multispec, is there a way to refer to a specific instance by name?
2016:07:15 20:29:44 arohner I’d like to set up my multi-spec hierachy, but then in some cases say ‘this fn takes this specific type from the hierarchy'
2016:07:15 20:30:13 arohner I know I can (s/and ::foo #(= :type (:foo %)), but it’d be nice to have some sugar on that
2016:07:16 00:01:40 gfredericks stuarthalloway: resizing with some multiple of square root would give another size growth pattern
2016:07:16 01:14:53 alexmiller I think Stu left this on my plate so I will take a look next week, thx
2016:07:16 04:13:56 sparkofreason If we conform a map to a spec using the unnamespaced keys, is there any way to get the map returned using the fully-qualified specs keywords as keys? I threw together a hacky and incomplete version of this, hoping there's an easier way.
2016:07:16 12:37:37 alexmiller @dave.dixon spec doesn't provide an explicit tool for that
2016:07:16 12:38:43 alexmiller It is possible if you did an s/merge of s/keys and s/map-of with :conform-keys true and a conformer key pred
2016:07:16 14:57:57 lvh Hey; is there any particular library for common specs like URI/URLs, dates, &c that’s forming/informally?
2016:07:16 14:58:08 lvh It might be useful to do so generally because a lot of this stuff appears to be string matching a regex
2016:07:16 14:58:30 lvh meaning that if it’s third-party, test.chuck can automatically give you generators
2016:07:16 15:06:01 gfredericks so you're saying it's time for me to create clojure.schpec
2016:07:16 15:07:40 lvh I’m writing a library that I think I will call ehm, specimen or something
2016:07:16 15:07:43 lvh it’s json-schema => spec
2016:07:16 15:08:08 lvh because I was translating swagger => spec and that was really boring repetitive work
2016:07:16 15:09:45 lvh but I figured I could get a computer to do it for me
2016:07:16 15:09:52 lvh the json-schema spec is also itself available as a json-schema
2016:07:16 15:09:55 lvh I’m not sure what that means yet
2016:07:16 15:47:30 gfredericks dammit that happened with spuck too
2016:07:16 15:47:52 lvh unfortunate connotations?
2016:07:16 15:47:54 gfredericks yeah
2016:07:16 15:48:43 lvh I don’t recongize that one, but yeah that sucks
2016:07:16 15:48:53 lvh it’s weird because I work with a ton of former intelligence people
2016:07:16 15:49:05 lvh so spook is a generally accepted word for them internally
2016:07:16 15:49:26 gfredericks according to alexmiller at least; I'm not sure if he was thinking of the urban dictionary entry, which has the usual difficult-to-assess relevance level
2016:07:16 15:51:53 lvh I’ve heard first, mostly outside of the US (Jewish parts of Antwerp, so may be reasonably common Yiddish though), the rest look made up,
2016:07:16 15:52:28 lvh spook is very definitely relevant according to some of my non-former-intelligence coworkers from the deep south
2016:07:16 15:55:36 donaldball spooky or spookily don’t carry the spy connotation
2016:07:16 15:59:16 lvh donaldball: Not worried about the spy connotation, worried about the racist epithet for people of color 🙂
2016:07:16 15:59:42 lvh donaldball: it was weird to the deep south coworkers because they weren’t used to the former but definitely heard the latter growing up
2016:07:16 15:59:48 lvh I think spooky doesn’t have that one either though
2016:07:16 16:05:46 donaldball Wow. I grew up in Tidewater and thought I’d heard ‘em all, but that’s a new one to me.
2016:07:16 16:05:57 glv Wow. I heard it used that way as a kid (visiting family in Mississippi, probably) but had completely forgotten that usage. That's … progress, I guess?
2016:07:16 16:06:03 donaldball Cache invalidation and naming things, right?
2016:07:16 16:08:17 gfredericks sperc
2016:07:16 16:14:32 lvh gfredericks: sperc sounds derpy, for sure
2016:07:16 18:02:47 gfredericks That's the only potential improvement to test.check that I've heard discussed that would have to be released before 1.9 is released
2016:07:16 18:18:03 lvh gfredericks: So, JSON Schema… I was thinking of just having something that generates specs for me; except of course it’s hard to check that something was (keys :req-un [...]) or whatever because they don’t compare identically; so I guess now I’m building a schema, using test.check to generate samples, and then using a JSON schema validator that consumes to validate that the things that were generated validate the original schema
2016:07:16 18:18:46 gfredericks your problem is you can't compare clojure.spec specs for equality?
2016:07:16 18:20:09 lvh yeah
2016:07:16 18:20:19 lvh I have a json schema and I want a fn that returns the equivalent specs
2016:07:16 18:20:35 lvh as a map, probably, because having a fn eval a bunch of spec/def exprs seems bad
2016:07:16 18:20:39 gfredericks can you use the data representation of the schema to do that in a good enough way?
2016:07:16 18:21:11 lvh I don’t know, that’s the hypothesis, yes
2016:07:16 18:21:28 lvh are you asking if the very idea of going from json schema -> spec is possible?
2016:07:16 18:21:44 lvh I also want it as data because I presume that you will want to modify/extend it
2016:07:16 18:25:09 gfredericks I mean clojure.spec lets you get the form of a spec
2016:07:16 18:25:16 gfredericks somehow
2016:07:16 18:25:27 gfredericks I might be misunderstanding
2016:07:16 18:26:08 lvh I’ll take a look at the api docs then
2016:07:16 18:26:25 lvh I was going to say that the main problem I have then is that the registry is global state, and I’m generating them for tests
2016:07:16 18:26:39 lvh I’m hoping I can with-redefs it or something
2016:07:16 18:48:54 gfredericks probably with any direct pred
2016:07:16 18:49:12 gfredericks (s/def ::string string?) and (s/form ::string) would work I bet
2016:07:16 19:36:31 sparkofreason @alexmiller: Thanks, slick solution much better than my hand-mangling of the spec. Almost worked, s/merge caused the unnamespaced keys to also make it through. Using s/and did the trick.
2016:07:16 20:21:16 glv @lvh if it's a problem that the registry is global state, do you have to use it? Can you just generate the specs and handle them as you need in local state, rather than using s/def?
2016:07:16 21:24:02 lvh glv: I’m speccing maps, so wouldn’t I need to use keys or every-kv?
2016:07:16 21:24:06 lvh keys takes a spec name
2016:07:16 22:15:26 glv Hmm … you're right about keys, but every-kv just takes two predicates. (And you'd probably want to use map-of instead of every-kv, unless you don't want full validation.)
2016:07:16 22:18:50 glv That is an interesting problem, though … I can see how keys would be useful for spec'ing data structures in contexts where you wouldn't want to use the global registry.
2016:07:16 23:30:37 sparkofreason Are there going to be specs for spec?
2016:07:17 00:14:11 gfredericks I think so
2016:07:17 01:30:42 alexmiller Yes to some degree. I went through and wrote most of them at one point but things have drifted. However, we explicitly don't check macro fdefs in spec as that will cause an infinite loop.
2016:07:17 01:31:06 alexmiller So they may exist mostly as documentation
2016:07:17 13:14:00 gfredericks the uncheckable macro?
2016:07:17 21:01:18 lvh Why is keys a macro?
2016:07:17 21:15:52 gfredericks I expect the same reason most of the spec definers is a macro
2016:07:17 21:15:59 gfredericks s/is/are/
2016:07:17 21:35:02 lvh I don’t know what that reason is (I just know that it’s annoying when I’m dynamically defining specs and it’s complaining that (some expr) isn’t a bunch of namespaced keywords)
2016:07:17 21:45:07 gfredericks "Yes, this means that more of the surface area of clojure.spec will be macros, but specs are overwhelmingly written by people and, when composed, manually so."
2016:07:17 21:45:37 gfredericks why're you generating your specs
2016:07:17 21:48:42 lvh I have some json schema, I’m trying to produce specs
2016:07:17 21:50:00 gfredericks why not go the other direction?
2016:07:17 21:50:03 lvh so, they’re macros because they want to see the exact expr being used?
2016:07:17 21:50:07 lvh how do you mean?
2016:07:17 21:50:09 lvh oh
2016:07:17 21:50:25 lvh I see. I’m not going the other direction because I’m not writing the json schema
2016:07:17 21:50:46 lvh it’s other people’s json schema, for other services; also json-schema for swagger, which I in turn use to generate specs for even more third party stuff
2016:07:17 21:51:03 lvh unfortunately it’s unlikely that I get all of these vendors to port their swagger specs to clojure.spec 😄
2016:07:17 21:51:40 gfredericks yes they want to see the exact expression
2016:07:17 21:52:14 gfredericks so you probably have to think of it more as a pre-compile step
2016:07:17 21:55:26 lvh as in, have the macro-hood leak through?
2016:07:17 21:55:40 lvh that’s a good idea, right now I solved it with eval, but bleh
2016:07:17 21:56:16 gfredericks eval's okay
2016:07:17 21:56:30 lvh I’d like to have the specs available easily as a map, because since it’s autogenerated there’s a pretty good chance that you’ll want to amend the result
2016:07:17 21:56:45 lvh e.g. specify a generator, or make a constraint more precise (or both)
2016:07:17 21:56:45 gfredericks similar is (defmacro load-specs [] (code to make spec forms)) (load-specs)
2016:07:17 21:57:01 lvh yeah that’s basically what I’m doing after
2016:07:17 21:57:25 lvh I should probably learn the difference between generating intern and def in a macro
2016:07:17 21:57:39 lvh but that’s definitely not ontopic for this channel 🙂
2016:07:17 21:58:41 gfredericks if you want to really wash your hands of dynamic runtime monkeydoodling you could generate the code and spit it to a source file that goes into version control
2016:07:17 21:58:58 gfredericks so it turns into a dev-time task
2016:07:17 21:59:16 lvh that sounds like something a go programmer might do 😉
2016:07:17 21:59:27 lvh I’ve already dodged that bullet with caesium (generating an interface)
2016:07:18 16:12:24 kendall.buchanan Any hints on limiting :num-tests in stest/check? (stest/check b/match-klasses {:clojure.spec.test/opts {:num-tests 10}}) isn’t working. It’s gotta be something simple I’m missing.
2016:07:18 16:13:35 kendall.buchanan Whoops, found it!:
2016:07:18 19:28:54 codonnell I'm struggling to create a spec for dates with a custom conformer, unformer, and generator. I'd love input on how to make this work or if I'm just trying to fit a square peg into a round hole. See the snippet for my attempts.
2016:07:18 20:08:43 wilkerlucio @kendall.buchanan quick tip: you can probably replace :clojure.spec.test.check/opts with ::stest/opts
2016:07:18 20:09:32 kendall.buchanan Ah, will ::stest/opts create an alias?
2016:07:18 20:10:17 wilkerlucio when you import a namespace into a short name (like stest for clojure.spec.test.check) you can use double colon notation :: and the namespace will be expanded
2016:07:18 20:11:41 codonnell @kendall.buchanan @wilkerlucio I think in this case stest resolves to clojure.spec.test, not clojure.spec.test.check, so you can't use it here.
2016:07:18 20:12:02 kendall.buchanan True.
2016:07:18 20:12:08 wilkerlucio correct
2016:07:18 20:12:27 wilkerlucio that was the part I wasn't sure, to where the alias was pointing, eheh
2016:07:18 20:12:51 kendall.buchanan In any case, I made myself a helper function and referenced that instead.
2016:07:18 20:13:08 wilkerlucio I'm looking forward for the feature that will enable us to alias namespaces without having to require it (specially good for referencing namespaces that not exists, for namespacing only), alex said it's on the table
2016:07:18 20:13:44 codonnell that would be handy
2016:07:18 20:34:09 seancorfield @wilkerlucio: Did you see the workaround Alex posted? Using in-ns to create the namespace in memory without needing a file for it.
2016:07:18 20:34:31 wilkerlucio @seancorfield: I didn't, seems cool, do you know if that works with cljs?
2016:07:18 20:36:31 seancorfield Sorry, no idea.
2016:07:18 20:59:00 wilkerlucio @seancorfield: do you have the snippet for that? I tried to search here, but no lucky
2016:07:18 21:42:49 seancorfield Oh, you just use (in-ns 'qualified.keyword.namespace) (in-ns 'original.namespace) to create qualified.keyword.namespace in memory and then switch back to whatever ns you were in.
2016:07:18 21:43:58 seancorfield Then you can do (alias 'q 'qualified.keyword.namespace) — because that ns exists now — and then ::q/k expands to :qualified.keyword.namespace/k as "expected".
2016:07:18 23:22:24 codonnell If anyone is curious, I solved my problem above with a dirty hack that relies heavily on implementation. (See snippet.)
2016:07:18 23:26:10 codonnell If with-gen* passed unc along to spec-impl, I could do it with just spec.
2016:07:19 13:42:34 borkdude @seancorfield: any reason you're not using create-ns there?
2016:07:19 16:01:08 alexmiller yeah, create-ns is equally valid
2016:07:19 16:53:53 seancorfield @borkdude: I just lifted that code from the example @alexmiller pointed us all to 🙂
2016:07:19 22:29:12 seancorfield Remind me: is there an easy way to create a generator based on a regular expression string?
2016:07:19 22:31:58 seancorfield @gfredericks: Looks like I could use test.chuck for this? Although it says it doesn’t support anchors (yet)?
2016:07:19 22:49:19 gfredericks seancorfield: yes and yes; are you using nontrivial anchors or just the ^...$ sort of thing?
2016:07:19 22:51:20 gfredericks I can't remember if supporting general anchors was just a bit messy or if it was Really Hard the way backreferences &c. are
2016:07:19 22:52:22 gfredericks or maybe backreferences are easier but lookahead/behind are hard
2016:07:19 22:52:24 gfredericks man it's been a while
2016:07:19 22:53:27 seancorfield I’m only using #"^…$"
2016:07:19 22:53:51 seancorfield I can lift out the string between as a separate regex for generation if need be.
2016:07:19 22:58:06 gfredericks that's exactly what I've done
2016:07:19 22:58:23 gfredericks when in a similar position
2016:07:19 22:59:08 gfredericks seancorfield: do you need the anchors there? e.g., re-matches already only matches the whole string
2016:07:19 22:59:47 seancorfield I can change how it’s used (from re-find to re-matches).
2016:07:19 22:59:51 gfredericks I think I was using anchors because prismatic/schema required them
2016:07:19 23:06:10 seancorfield Looks like it doesn’t handle a pattern like [^\s] correctly maybe? I get spaces in the generated strings...
2016:07:19 23:06:44 gfredericks that sure oughta work
2016:07:19 23:06:56 gfredericks the thing is Surprisingly Robust™
2016:07:19 23:07:09 gfredericks if it can't handle something it should throw an exception when you try to create the generator
2016:07:19 23:07:27 gfredericks if you can share the regex I can try to reproduce
2016:07:19 23:07:43 seancorfield Sure… let me test something smaller...
2016:07:19 23:09:18 seancorfield Ah, no, it’s doing the right thing… that generated data is not what it seems 😐 Let me try something else...
2016:07:19 23:10:40 seancorfield What appeared to be a space is actually the character for decimal code 55335 (!) so it looks like we’re good. Excellent!
2016:07:19 23:11:07 gfredericks :)
2016:07:19 23:12:55 gfredericks speaking of which I'd love to hear if anybody has any thoughts about string generators and unicode
2016:07:19 23:13:12 gfredericks there are a couple test.check tickets about it and I have no idea what the best idea as
2016:07:19 23:13:21 gfredericks s/as/is/
2016:07:19 23:13:42 gfredericks it's a question of what a reasonable distribution is
(def email-regex
"Sophisticated regex for validating an email address."
(-> (str "(([^<>()\\[\\]\\\\.,;:\\
2016:07:19 23:17:12 gfredericks those are some really good email addresses
2016:07:19 23:17:29 seancorfield (I don’t remember where we got that from and, yes, I know you can’t really validate email addresses correctly with "just" a regex but this has been pretty good for us so far)
2016:07:19 23:17:59 seancorfield But I’m very impressed that I could just throw that at test.chuck and it worked right out of the box!
2016:07:19 23:18:12 gfredericks :D
2016:07:20 06:27:34 bbloom those generated emails are amusing 🙂 i’ve noticed that generated tests tend to have pretty wacky looking data - i guess that’s good for tests, but i can’t help but wonder: what about “example data”? anybody taken a crack at that? would be cool to lorum ipsum up a full database based on specs!
2016:07:20 07:04:46 robert-stuttaford i second that, @bbloom
2016:07:20 08:48:31 rickmoynihan Sometimes specs seem to dereference themselves... e.g. (s/def ::k (s/keys :req [::foo ::bar])) dereferences ::foo ::bar but I've noticed that for example (s/merge ::k ,,,,) blows up... instead you need to bind :k to a real var... i.e. it won't dereference the keyword into the specs value.
I'm just wondering if this is by design? And what the reason is... and how to know what to expect
2016:07:20 10:19:58 rickmoynihan actually this does make a lot of sense... sorry being stupid... keys/`:req` requires knowledge of the keys themselves... and clearly giving keywords symbol semantics would be insane
2016:07:20 10:24:46 rickmoynihan I guess you have to rely on the doc strings and specs specs... to know the difference between a spec and a keyword... I guess it's kinda like the distinction between quoted and unquoted symbols... I've only just got started with spec - so knowing when to use which isn't entirely clear to me
2016:07:20 10:25:33 mpenet is there a version of #(partial instance? %) in core or spec?
2016:07:20 10:28:17 mpenet apparently not
2016:07:20 10:28:58 mpenet I guess this could even simply be another arity of instance?, given how common this is
2016:07:20 10:28:59 rickmoynihan yeah I've written that one more than a few times
2016:07:20 10:29:31 mpenet ex: (instance? Cluster), which would be a partial on instance?/2
2016:07:20 10:30:11 mpenet @alexmiller: would you consider something like this, I'd gladly send a patch for it ?
2016:07:20 11:36:23 alexmiller No thanks, has been considered by Rich
2016:07:20 12:47:57 gfredericks are the lower-level details of clojure.spec, in particular the implementation of s/keys, meant to be public so that users/libraries can use them to write new kinds of specs?
2016:07:20 12:48:15 gfredericks it might just amount to implementing a protocol, in which case the question is if the protocol is meant to be public
2016:07:20 13:19:58 alexmiller I would say that you should consider all implementation details of spec to be just that - subject to change without notice
2016:07:20 13:21:22 alexmiller spec is meant to provide a relatively complete set of compositional parts, extensible (at the unit level) via predicates
2016:07:20 13:22:10 alexmiller which is not to say that extending via the protocol is necessarily wrong, just that we’re not committed to maintaining that api, so use at your own risk (like any other Clojure “internals")
2016:07:20 13:22:41 alexmiller the public api is what’s in the api docs
2016:07:20 13:24:38 codonnell @alexmiller: is there a reason with-gen does not pass along the unformer function to its return value?
2016:07:20 13:25:43 mpenet are there improvement coming to prevent having to manually create non existing kw namespaces?
2016:07:20 13:26:02 codonnell
;; conform works, unform fails, gen works
(def date-spec
(s/with-gen
(s/conformer date-conformer date-unformer)
(fn [] (s/gen inst?))))
2016:07:20 13:26:58 alexmiller @mpenet maybe
2016:07:20 13:27:19 alexmiller @codonnell: thx, I’ll pass along to Rich
2016:07:20 13:27:37 codonnell no problem
2016:07:20 14:41:38 mpenet and next in my wish list of stuff that will not happen for sure is a partially applied version of satisfies?😆. Probably an opportunity for a lib with helpers like these
2016:07:20 14:44:38 alexmiller knock yourself out
2016:07:20 19:08:43 gfredericks on that note
2016:07:20 19:10:45 codonnell that would be a fantastic name
2016:07:20 19:11:12 gfredericks well I'm doing it right now
2016:07:20 19:15:48 soloist any idea how to write a spec to check an aggregate value of a collection? eg: (s/valid? (= 5 (partial reduce +))) or something like that..
nvm mybad: (s/conform #(= 5 (reduce + %)) [2 3])
2016:07:20 19:28:26 gfredericks lvh: ^ I finally created schpec
2016:07:20 19:28:33 lvh awesome!
2016:07:20 19:28:43 lvh I just saw an email notification from the corner of my eye
2016:07:20 19:29:20 lvh "whatever other feature you miss from plumatic/schema” 💯
2016:07:20 19:30:18 lvh it’s distracting to see how many layers deep you can go — I was trying to do something useful with swagger specs and now I’m explaining to clojure.spec how json schema works so I can generate json schemata so I can generate specs from them so I can generate instances of those specs and then see if they match the original json schema
2016:07:20 19:30:23 lvh it’s specs all the way down 🙂
2016:07:20 19:31:20 gfredericks a shed to put bike sheds in
2016:07:20 19:53:51 bhauman I've got some bikes for that shed ...
2016:07:20 20:07:27 alexmiller @jjcomer: read the instrument docs carefully - "Instruments the vars named by sym-or-syms”, "Opts for symbols not included in sym-or-syms are ignored.”, etc - I think that right away rules out your -3 and -4 examples. I’ll look closer at -2. Some of this stuff really makes most sense when instrumenting a ns or all and passing an opts map that sets up the env you want.
2016:07:20 20:18:45 alexmiller for -3, if you replace
`inc-it
with
[`inc-it `print-it]
I think that works
2016:07:20 20:20:27 alexmiller for -2 and -4, interestingly if you do that instrument and check outside a defn, just at the repl, they work
2016:07:20 20:22:02 alexmiller (that is, don’t print)
2016:07:20 20:22:27 alexmiller which implies things about compilation and may be a problem
2016:07:20 20:37:29 alexmiller yeah, when the compiled class is loaded, the var has not yet been altered by instrument and that’s what is loaded into the class constants and used at runtime - the instrumented version of the var is not used
2016:07:20 20:42:09 alexmiller nah that doesn’t make sense - check takes the symbol, not the var and resolves at runtime
2016:07:20 20:46:28 kendall.buchanan I suspect I’m doing something wrong, but I’m reliably getting errors like this:
2016:07:20 20:46:31 kendall.buchanan java.util.concurrent.ExecutionException: java.lang.ClassCastException: app.business.klass_test$eval75458$fn__75459 cannot be cast to clojure.lang.MultiFn, compiling:(clojure/test/check/clojure_test.cljc:95:1)
2016:07:20 20:46:41 kendall.buchanan From stest/check.
2016:07:20 20:47:38 kendall.buchanan It’s typically when I restart the REPL and try to run check from my test files (without first navigating over and evaluating the functions it calls). Note I’m requiring the namespaces first.
2016:07:20 20:58:35 kendall.buchanan There isn’t a danger to calling stest/check within deftest is there?
2016:07:20 20:59:06 alexmiller shouldn’t be
2016:07:20 20:59:39 alexmiller is lein or other build tooling involved?
2016:07:20 20:59:52 alexmiller lein hacks up a lot of the clojure.test stuff
2016:07:20 21:00:18 kendall.buchanan It’s certainly possible – I’m using weavejester’s eftest.
2016:07:20 21:00:40 alexmiller does it happen if you don’t?
2016:07:20 21:01:07 kendall.buchanan It happens if I simply eval the form, yes...
2016:07:20 21:01:41 kendall.buchanan But it’s a fairly large project – it’s possible something else is intruding.
2016:07:20 21:02:12 alexmiller eftest is definitely also mucking with clojure.test/report
2016:07:20 21:02:23 kendall.buchanan Yeah, it’s a touch thing – if I go to the test namespace, require everything, and run a single form, it works.
2016:07:20 21:02:39 kendall.buchanan But if I restart the REPL, go to the namespace, eval everything all at once, it falls apart.
2016:07:20 21:03:32 kendall.buchanan K, thanks. Helps to know it’s likely in my environment. I’ll report back when I find it.
2016:07:20 21:03:43 kendall.buchanan But yeah, only happens on check.
2016:07:20 21:04:28 alexmiller both eftest and check are bashing the same clojure.test hook for reporting and that seems to be in the realm of the error you’re seeing
2016:07:20 21:04:50 alexmiller so I’m going to suspect that combination as the most likely probable cause
2016:07:20 21:05:51 alexmiller I guess not “cause”, I don’t understand why the two together fails but I suspect that’s involved
2016:07:20 21:06:17 kendall.buchanan Okay, great insights. I’ll start by disabling eftest.
2016:07:20 21:06:43 alexmiller if you can get a reproducible case that fails with both, feel free to file a jira and maybe one or the other can be made more tolerant/harmonious
2016:07:20 21:13:41 kendall.buchanan Okay...
2016:07:20 21:13:46 kendall.buchanan Disabling eftest altogether had no effect:
2016:07:20 21:14:21 kendall.buchanan That’s after taking any reference to eftest out of project.clj, and running lein test on its own.
2016:07:20 21:16:19 alexmiller are you explicitly requiring clojure.test.check.clojure-test?
2016:07:20 21:17:28 alexmiller maybe to use defspec?
2016:07:20 21:17:54 kendall.buchanan Made myself a couple helpers to simplify tests with check.
2016:07:20 21:22:21 alexmiller unless you’re loading it, it’s not clear to me why clojure.test.check.clojure-test is getting called at all. nothing in core calls it. Your definition above won’t call it, yet you are compiling it. are you AOT’ing?
2016:07:20 21:22:43 kendall.buchanan Seeing if I can create a reproducible lein project...
2016:07:20 21:22:55 kendall.buchanan No AOT.
2016:07:20 21:30:25 kendall.buchanan Cool, just reproduced it.
2016:07:20 21:30:38 kendall.buchanan Very well could be lein, cause I have a barebones example.
2016:07:20 21:36:05 kendall.buchanan I’m happy to post to JIRA too if you feel like it’s a legit reproduction.
2016:07:20 21:37:22 alexmiller gimme a sec to look at it
2016:07:20 22:27:29 alexmiller @kendall.buchanan: looks like this has nothing to do with spec per se
2016:07:20 22:28:39 alexmiller clojure.test.check.clojure-test assumes that clojure.test/report is a multimethod (which it is in core) and modifies one of it’s cases - this is loaded as a side effect when clojure.test.check is loaded via gen via spec.test/check
2016:07:20 22:29:06 alexmiller however, Leiningen monkeypatches core and replaces that clojure.test/report function with something that’s not a multimethod
2016:07:20 22:29:42 kendall.buchanan I noticed the project threw off another error about gen being missing… until I added the test.check dependency.
2016:07:20 22:29:43 alexmiller so lein test and test.check.clojure-test appear to be incompatible afaict
2016:07:20 22:29:58 alexmiller you can turn off the monkey patching in lein though
2016:07:20 22:30:02 alexmiller with :monkeypatch-clojure-test false in your project.clj
2016:07:20 22:30:28 alexmiller @gfredericks: ^^ you aware of any of this?
2016:07:20 22:30:30 kendall.buchanan Interesting. I’m assuming lein has documented the trade-off of turning it off?
2016:07:20 22:31:03 alexmiller I think you then lose “lein retest"
2016:07:20 22:31:35 kendall.buchanan Not sure I’d ever have known to look to lein for this… is it appropriate to introduce this into spec docs as a “gotcha”? (just curious how you guys deal with things like this…)
2016:07:20 22:32:39 alexmiller maybe, I was not aware of it before. the master version of test.check actually doesn’t load clojure-test anymore, so you wouldn’t even see this come up unless you included it explicitly
2016:07:20 22:33:11 alexmiller I gotta go, but will think about it more tomorrow
2016:07:20 22:33:24 kendall.buchanan Okay, I am loading explicitly in my project as well… good point.
2016:07:20 22:33:35 kendall.buchanan @alexmiller: Thanks much for spending so much time on that.
2016:07:20 22:33:43 kendall.buchanan Big help.
2016:07:20 22:33:55 alexmiller thx for the good and simple repro! that helped enormously
2016:07:20 22:34:02 kendall.buchanan np
2016:07:20 22:34:46 alexmiller @jjcomer I haven’t forgotten yours either - still looking at it
2016:07:20 22:38:09 gfredericks alexmiller: I've been using lein-test with test.check.clojure-test for years, so I assume it's something subtler than a full incompatibility. I wasn't aware of this before, no, so I'll look into it
2016:07:20 22:41:35 kendall.buchanan @alexmiller: One final note: turning off monkeypatching fixed it. Turning eftest back on broke it again.
2016:07:20 22:42:01 kendall.buchanan It’s a great start, though, to at least know where it breaks down.
2016:07:20 22:47:44 gfredericks kendall.buchanan: I'm trying to reproduce it now
2016:07:20 22:52:04 gfredericks kendall.buchanan: any tips beyond "using test.check and lein test"?
2016:07:20 22:52:36 kendall.buchanan gfredericks: Did you clone the repo above?
2016:07:20 22:53:53 gfredericks oh no, didn't see it
2016:07:20 22:59:20 gfredericks stest/check is explicitly about generative testing, right?
2016:07:20 22:59:39 kendall.buchanan Yes, it tests function specs.
2016:07:20 22:59:51 gfredericks I'm not sure how it could make sense to run that without test.check available, given the underlying "test.check is optional" design decision
2016:07:20 22:59:53 kendall.buchanan (It’s the only way to test the return values of functions, afaik.)
2016:07:20 23:00:05 kendall.buchanan Okay, I figured as much.
2016:07:20 23:03:42 gfredericks kendall.buchanan: I still don't quite understand how this happens (or rather, why it doesn't happen all the time), but I can at least tell you it's fixed on test.check master
2016:07:20 23:04:06 gfredericks I'm hoping to make a new release soon
2016:07:20 23:05:46 kendall.buchanan That’s great news, thansk.
2016:07:20 23:06:11 gfredericks oh actually I get it
2016:07:20 23:06:27 gfredericks I think lein does the monkeypatching after all the requires get loaded
2016:07:20 23:06:49 gfredericks so this is only a problem if you (or something) loads the test.check.clojure-test namespace later (at runtime)
2016:07:20 23:06:57 kendall.buchanan @gfredericks: Just curious… if stest/check is all about generative testing, why is it in clojure.spec.test? Does it not make more sense to move it to test.check (making one aware of the other, but not in both directions)?
2016:07:20 23:07:37 gfredericks kendall.buchanan: that might be possible; but spec might be too tied together with test.check to make that practical
2016:07:20 23:38:05 gfredericks I'm not planning to investigate it anymore (I considered closing the issue immediately after creating it) until I hear that it's a problem for someone on test.check master somehow
2016:07:20 23:38:18 gfredericks I figured I'd create the ticket to document the problem for future historians though
2016:07:21 01:15:57 gfredericks i.e., is there a difference between the real specs and the specs describing args to the functions to make specs?
2016:07:21 01:16:23 gfredericks I'm imagining maybe namespacing the function arg/ret specs one level deeper, using the name of the function as the last component of the namespace
2016:07:21 01:16:59 gfredericks which of course immediately brings up the old "aliasing a non-code namespace"
2016:07:21 01:17:02 gfredericks problem
2016:07:21 01:20:22 donaldball If s/keys allowed anonymous specs for :req-un and :opt-un, that would be the obvious choice, but absent that, using the fn name as the last component of the spec namespace is p reasonable
2016:07:21 01:21:18 gfredericks oh hi donaldball
2016:07:21 01:21:33 donaldball howdy, stranger
2016:07:21 01:22:29 gfredericks donaldball: would you say the main point of these specs is combining range-checks with type-agnosticism?
2016:07:21 01:28:31 donaldball I think of them as more describing domains in the math sense, but we’re probably saying the same thing
2016:07:21 01:31:32 bhauman sooner or later we are going to have a record like map spec that doesn't require namespaced keys
2016:07:21 01:37:47 donaldball I’ve written a few score specs now and am tentatively concluding that Rich’s encouragement towards namespaces keys is tremendously valuable for describing domain model maps, and not very useful, maybe even a bit abrasive, when describing function option maps or kwargs.
2016:07:21 01:39:13 bhauman I agree completely, did you see my suggestion on the PR?
2016:07:21 01:40:36 donaldball That’s an interesting idea, I haven’t tried it in that way yet
2016:07:21 01:41:53 bhauman a bit verbose but it feels like it fits the problem better
2016:07:21 01:42:07 donaldball Yeah, it’s more in the spirit
2016:07:21 01:42:14 bhauman it allows duplicate keys but ...
2016:07:21 01:42:34 donaldball If you had a bunch of fns that shared many of the same options, maybe a spec registry would be called for
2016:07:21 01:42:57 bhauman absolutely, in fact it would be shame not to
2016:07:21 01:44:42 donaldball kwargs do allow duplicate keys anyway, so your formulation may be more correct
2016:07:21 01:45:12 donaldball (even though it’s probably a usage bug when it happens)
2016:07:21 01:45:45 bhauman well its straightforward enough to add a check for that
2016:07:21 01:46:33 bhauman but a bit cumbersome ... really asking for a macro at that point
2016:07:21 01:49:51 gfredericks schpec.bhauman
2016:07:21 01:50:26 bhauman the notorious bad boy ns
2016:07:21 01:50:40 bhauman oh you're using that ...
2016:07:21 01:52:40 bhauman all the gen functions return (repeat 'figwheel)
2016:07:21 02:01:21 gfredericks bhauman: so the whole point of your comment code is to avoid the namespacing-requirement of keys*?
2016:07:21 02:01:47 gfredericks I bet that spec conforms a lot more awkwardly than keys* does :/
2016:07:21 02:17:07 bhauman agreed
2016:07:21 02:20:13 bhauman but I'm just trying to use the whats available to avoid registering in situations that don't merit it
2016:07:21 02:20:56 gfredericks we could make a macro that names them in the background with gensyms or something
2016:07:21 02:21:24 bhauman but you could also make a macro that doesn't require the registration
2016:07:21 02:21:48 gfredericks by using keys* or by avoiding it?
2016:07:21 02:22:43 bhauman by avoiding it
2016:07:21 02:23:06 bhauman but I don't really see whats wrong with the regex formulation
2016:07:21 02:24:27 gfredericks less readable, conforms weird
2016:07:21 02:24:29 gfredericks I think that's it
2016:07:21 02:24:50 gfredericks well more readable in one sense I guess
2016:07:21 02:24:59 gfredericks keys* requires you to go somewhere else for the rest of the spec
2016:07:21 02:25:15 bhauman and a macro could fix that
2016:07:21 02:25:36 bhauman the conforming, and readability ... but this isn't a big deal at all
2016:07:21 02:25:57 bhauman I'm just interested because this has come up a bunch for me
2016:07:21 02:26:19 bhauman I've been writing a spec for leiningen
2016:07:21 02:26:31 gfredericks you can change the conforming with a macro?
2016:07:21 02:26:46 gfredericks I'm still hazy on all the details of clojure.spec
2016:07:21 02:26:52 bhauman you mean the final conformed output?
2016:07:21 02:27:02 bhauman with a conformer yes
2016:07:21 02:27:21 gfredericks man I gotta learn this stuff better
2016:07:21 02:28:55 gfredericks so you could take the sequence of pairs that your style of regex creates and conform that further into a map?
2016:07:21 02:29:08 bhauman yes sireee
2016:07:21 02:29:17 bhauman you can do ANYTHING
2016:07:21 02:30:06 gfredericks this has been another episode of Turing Completeness with Bhauman
2016:07:21 02:30:50 gfredericks I'm interested in how useful conformers are for dealing with transforming things in and out of json and jdbc
2016:07:21 02:32:08 gfredericks I'd like to be able to precising describe the idiosyncratic shape of some funky json at the same time as describing the pristine clojure-friendly equivalent and automatically get functions to translate between them
2016:07:21 02:32:19 gfredericks ditto for goofy jdbc types
2016:07:21 02:32:24 bhauman well it can be done
2016:07:21 02:32:37 gfredericks that's all I wanted to hear
2016:07:21 02:32:41 bhauman and lets face it ... it should be done and done now
2016:07:21 03:03:14 seancorfield clojure.spec FTW: as we’re specifying more of our domain model and rewriting our validation logic on top of s/conform, we’re discovering some interesting edge cases that we hadn’t considered before, as well as uncovering inconsistencies between how we treat certain pieces of data, in different parts of our application. It’s very enlightening… and it’s also making us feel a lot more confident in our updated validation code!
2016:07:21 03:05:55 seancorfield Not all of our validation is suitable for clojure.spec tho’: we have several places where the validation is contextual, based on current data from the environment, but all of the basic "shape" validation is much nicer now.
2016:07:21 06:06:10 bbloom surely clojure.spec vNext will have context sensitive logic variables 😉
2016:07:21 06:09:35 bbloom unrelated: I’m all for the openness of maps etc, but I’ve run in to some real bugs where typos or misplaced data gets ignored. Is there anything currently or planned for identifying “extra” data? I’ve found it incredibly useful to log unknown data in the past.
2016:07:21 06:10:19 bbloom As an example from a Go program I worked on recently, we were parsing a Toml config file and our tester lost a few hours to a flag that was in the wrong section. So I added a call to this “Undecoded” method: https://github.com/BurntSushi/toml/blob/99064174e013895bbd9b025c31100bd1d9b590ca/decode_meta.go#L113 — logged all the undecoded keys and he never had this problem again
2016:07:21 06:11:17 seancorfield @bbloom: I think the only solution right now is s/and with a set operation on the keys of the map?
2016:07:21 06:11:56 bbloom @seancorfield: gotcha, thanks. was hoping not to treat it as invalid only to discover that it exists
2016:07:21 06:13:04 seancorfield Yeah, if you're dealing with just :req keys you're OK but once you get into optional key territory it's tricky 😞
2016:07:21 06:13:04 bbloom something like s/conform but like s/unconformed that returns paths to all the data that was ignored
2016:07:21 06:13:35 bbloom that Go code just returns a collection of paths
2016:07:21 06:39:55 mpenet another case like this in our context: maps as database records, an invalid map-entry could cause a query execution exception, invalid column etc (a map missing a key because of the typo is valid too, which doesn't help)
2016:07:21 06:42:54 mpenet @seancorfield: do you have an example of this keys+and code?
2016:07:21 06:43:22 mpenet I guess this can be done with map-of as well
2016:07:21 08:23:58 mpenet " Instruments the vars named by sym-or-syms, a symbol or collection
of symbols, or all instrumentable vars if sym-or-syms is not
specified."
2016:07:21 08:25:05 mpenet I guess you need to call ns-publics and then pass the coll to instrument, or just call instrument without arg
2016:07:21 10:58:39 mpenet maybe this is something that changed recently
2016:07:21 11:34:00 mpenet replacing (s/+ string?) by (s/coll-of string? :min-count 1 or (s/spec (s/+ string?)) works, but I am not sure why "+" failed here
2016:07:21 11:53:39 mpenet "When regex ops are combined, they describe a single sequence. If you need to spec a nested sequential collection, you must use an explicit call to spec to start a new nested regex context." okay
2016:07:21 13:19:30 gfredericks @bbloom: regarding discovering typos/etc in open maps, I've been pondering the idea of a pair of schemas -- the normal one, for which violations are a bug/error-condition/whatever, and a more restrictive one, for which violations are merely suspicious, and might warrant logging, etc.
2016:07:21 13:22:59 gfredericks E.g. "this string is probably a uuid", "this map probably has no unknown keys", ...
2016:07:21 13:29:12 gfredericks @bbloom: your conform/unconform idea suggests that you could just roundtrip the data and do a diff, but that would only work with map keys and not general mistakes I think
2016:07:21 13:29:38 mpenet @gfredericks: something that could be nice for speck would be a schema->spec converting function/macro. When you have large'ish Schemas (ex multi-level maps) it get tedious/verbose fast. I might actually give it a try if I get some free time this week end
2016:07:21 13:30:49 gfredericks By "schema" you mean something like plumatic/schema's api where the schema is the same shape as the data?
2016:07:21 13:30:54 mpenet yes
2016:07:21 13:31:05 mpenet I meant plumatic/schema schemas
2016:07:21 13:31:10 gfredericks Yeah I definitely imagined that would show up
2016:07:21 13:31:16 mpenet since it's so wide spread atm
2016:07:21 13:31:24 gfredericks Oh you mean their whole API?
2016:07:21 13:31:37 gfredericks Or as much as possible at least
2016:07:21 13:31:48 mpenet no, I mean just an utility to convert schemas from their format to spec
2016:07:21 13:31:58 mpenet some of their api is hard to port to spec
2016:07:21 13:32:06 mpenet ex all the coercion stuff
2016:07:21 13:32:15 gfredericks A 1-time dev utility or a runtime thing?
2016:07:21 13:32:34 mpenet well I guess if you do the latter the former spawns in a couple of lines
2016:07:21 13:33:21 gfredericks The former can be less robust
2016:07:21 13:34:13 mpenet In my case I am mostly concerned about the ton of map schemas we have in our projects, would be nice to be able to port these without spending ages on it and laying a bug field in the process
2016:07:21 13:34:34 mpenet I could just not bother as well, and use both in parallel
2016:07:21 13:37:06 gfredericks Just punt on certain things and point the user to where it's broken
2016:07:21 13:37:14 mpenet the example of nested maps is a good one I think, a very concise schema from plumatic Schema can end up being a tons of lines of mostly s/def's for k/v
2016:07:21 13:38:00 mpenet could end up being a more concise way to write map specs as well
2016:07:21 13:38:22 gfredericks It definitely would be
2016:07:21 13:42:52 l0st3d Hi ... I've just been looking at spec (alpha10) and am confused by some results I'm seeing ...
2016:07:21 13:43:37 l0st3d the docs from fdef suggest that it should instrument the function referred to
2016:07:21 13:43:45 l0st3d but it doesn't seem to
2016:07:21 13:43:52 gfredericks Not by itself
2016:07:21 13:44:14 gfredericks You have to call c.s.test/instrument
2016:07:21 13:44:47 l0st3d and that will permanently replace the var with a function that checks it's args?
2016:07:21 13:45:15 l0st3d the docs for fdef say:
2016:07:21 13:45:17 l0st3d Once registered, function specs are included in doc, checked by
instrument, tested by the runner clojure.spec.test/run-tests, and (if
a macro) used to explain errors during macroexpansion.
2016:07:21 13:47:37 gfredericks "Checked by instrument" is a reference to the function I mentioned above
2016:07:21 13:48:06 gfredericks The wording is confusing though since that's not obvious
2016:07:21 13:48:10 bhauman @gfredericks: I have a strict-keys macro that uses a dynamic variable to set it's level of strictness
2016:07:21 13:49:13 gfredericks @bhauman: I'm in the middle of pushing more customization into plumatic/schema to support this kind of thing
2016:07:21 13:49:26 l0st3d @gfredericks: thanks
2016:07:21 13:50:46 bhauman I think a better solution is to have both the dynamic variable with a spec level override to force a certain level for certain specs
2016:07:21 13:50:51 gfredericks @l0st3d: I think the intention is for you to do more explicit checks if you want production checking
2016:07:21 13:51:53 gfredericks @bhauman: cuz e.g. defproject is always open but certain submaps are more restrictive?
2016:07:21 13:53:10 l0st3d @gfredericks: fair enough ... just thought that the docs suggested it would do those checks for me if I included the fspecs ... just trying to work out the api atm i think 😉 .. thanks for your help
2016:07:21 13:53:13 bhauman just that you would want certain maps to never allow extraneous members
2016:07:21 13:53:28 bhauman there is actually a really interesting case for misspellings
2016:07:21 13:53:44 bhauman and still having an open map
2016:07:21 13:54:20 bhauman having a level of verification that does a fuzzy-selection of the set of keys
2016:07:21 13:55:20 bhauman so the provided keys are checked with a distance to the spec'ed keys
2016:07:21 13:56:35 bhauman This could frankly be done all the time for something like (s/keys ) where a warning will be generated
2016:07:21 13:57:11 bhauman @bbloom: ^
2016:07:21 13:57:52 bhauman but being able to set a dynamic var to adjust this level of checking seems appropriate
2016:07:21 14:10:26 sundbp i’m struggling with a recursive map spec. what i’m trying to do is something like this:
2016:07:21 14:45:55 alexmiller there’s a simple example - I can’t quite work out what you’re trying to do
2016:07:21 15:54:53 sundbp thanks. i’ll check it out
2016:07:21 15:58:01 sundbp first time i tried something like that i thought i got a complaint about the spec not existing.. think was because i didn’t have ::graph-seq and ::node-inputs in the right order.
2016:07:21 15:58:07 sundbp simple mistake
2016:07:21 15:58:16 sundbp fine if i move those 2 around
2016:07:21 16:12:59 seancorfield @mpenet: My understanding is that, despite the docstring, you can use instrument to turn on instrumentation for an entire namespace using that call.
2016:07:21 16:13:26 mpenet Hmm I tried but it didn't seem to do anything.
2016:07:21 16:13:47 mpenet it returned [], which would seem to indicate it instrumented nothing
2016:07:21 16:14:46 mpenet I am not at work anymore, but I guess I ll check the souce next chance I get
2016:07:21 16:22:37 seancorfield Yeah, something definitely isn’t working right now — I changed an fdef spec and the tests still pass… investigating.
2016:07:21 16:22:54 seancorfield It definitely used to work.
2016:07:21 16:23:24 mpenet @alexmiller: would it be a bug or is it an intentional change?
2016:07:21 16:23:58 mpenet ^ instrument no longer working with a ns argument
2016:07:21 16:24:39 alexmiller that’s intentional - use st/enumerate-namespace to produce a list of syms in an ns
2016:07:21 16:24:51 mpenet ok makes sense
2016:07:21 16:27:33 seancorfield Hmm, I missed that change in the release notes @alexmiller
2016:07:21 16:27:45 alexmiller alpha8
2016:07:21 16:27:53 seancorfield Although now I can’t get my fdef specs to work at all
2016:07:21 16:27:55 alexmiller iirc
2016:07:21 16:28:22 alexmiller for check or instrument?
2016:07:21 16:28:51 alexmiller I have just tracked down a problem with check
2016:07:21 16:28:58 seancorfield When I call fdef, the spec does not subsequently show up on doc...
2016:07:21 16:29:19 alexmiller how are you calling it? expects a fully-qualified symbol
2016:07:21 16:29:57 alexmiller actually, it resolves, so not necessarily fully-qualified
2016:07:21 16:32:53 seancorfield I had (s/fdef drop-table-ddl …) after referring in :all and that used to work but doesn’t now. I changed it to (s/fdef clojure.java.jdbc/drop-table-ddl …) and it works now.
2016:07:21 16:41:22 alexmiller yeah, if you use a bare symbol for def or fdef, it will treat that as <current-ns>/sym
2016:07:21 16:41:34 alexmiller so it’s not going to pick up refer's
2016:07:21 16:44:02 seancorfield That changed at some point. This code used to work.
2016:07:21 16:44:25 seancorfield (not a big deal but the silent "failure" is disturbing)
2016:07:21 16:51:44 alexmiller fdef stuff changed around 6/7 - there were some major rewrites of it in there
2016:07:21 16:52:16 alexmiller you can use st/instrumentable-syms to verify that things are speced maybe?
2016:07:21 16:56:23 sundbp with these changes - if you want to run your clojure.test tests with specs instrumented and checked for fdef’s, what’s the proposed setup?
2016:07:21 16:59:48 alexmiller you’ll need to turn on instrumentation
2016:07:21 17:00:03 alexmiller so you can do that per-test, in a fixture, etc
2016:07:21 17:02:58 sundbp if one would like to turn on instrumentation for “everything” in a fixture - does one have to manually enumerate the NS to then do instrumentable-syms and finally instrument? i.e. there’s nothing that just turns on instrumentation for all NS in project?
2016:07:21 17:06:38 alexmiller just (st/instrument) is supposed to do that
2016:07:21 17:07:07 alexmiller that’s like the old instrument-all
2016:07:21 17:07:31 alexmiller and macro fdefs are always instrumented in macroexpansion
2016:07:21 17:16:03 seancorfield Here’s what I ended up with in java.jdbc:
2016:07:21 17:16:47 seancorfield And I had to qualify all the symbols in clojure.java.jdbc.spec fdef calls in order to get those working again.
2016:07:21 17:16:55 seancorfield Life on the bleeding edge … 🙂
2016:07:21 17:17:57 seancorfield Thanks @mpenet for the heads up on that — I hadn’t noticed it was broken!
2016:07:21 17:24:40 fenton in my app is use GPS data. Normally my data structure looks like: {:lat 33.3 :lng -129.3}. With spec things are looking like: {:pc.api/latitude 33.3 :pc.api/longitude -129.3}. I'm finding it less fun to type that everywhere. I guess I could alias my namespace to [pc.api :as a] and change latitude and longitude to lat/lng then get {:a/lat 33.3 :a/lng -129.3}. R others doing/finding something similar? I wonder if having keywords as short as lat/lng leads to being unclear? Thoughts?
2016:07:21 17:29:29 seancorfield You can use unqualified keywords in maps if you want @fenton
2016:07:21 17:29:49 mpenet :) @seancorfield i looked a bit at clj.jdbc to spec parts of alia, that s how i spotted this
2016:07:21 17:30:16 seancorfield I’m cutting 0.6.2-alpha2 with those updates.
2016:07:21 17:30:38 fenton @seancorfield: do you mean un-qualified?
2016:07:21 17:33:02 fenton @seancorfield: I guess I was suggesting the ns qualified keywords seem a bit of a pain to type everywhere...maybe I should look into using un-qualified keywords....not sure the implications off the top of my head... I'm thinking, how do you use unqualified keywords in other files? The way I'm using specs is to put them into a third *.cljc file that is shared by my front and backends. So dont i have to use qualified keywords in that case?
2016:07:21 17:35:58 fenton i guess i missed that, i'll go check it again...
2016:07:21 17:36:10 seancorfield Scroll down to where it shows :req-un being used.
2016:07:21 17:36:38 fenton ok...
2016:07:21 17:37:00 fenton i'll give that a try....that'll clean up my code nicely i think.
2016:07:21 17:37:04 fenton thanks!
2016:07:21 18:25:16 fenton @seancorfield: I'm unfamiliar with the #::a{:lat 3.3 :lng 3.3} syntax. i.e. the pulling of the namespace out in front of the map. is there some documentation about that somewhere?
2016:07:21 19:00:58 alexmiller @jjcomer I can explain what you’re seeing now btw. spec.test/check returns a lazy sequence of test results per sym. In your test, you are not realizing those results, then immediately uninstrumenting them, then later realizing (actually running the check on the uninstrumented functions). If you wrap a doall around your two calls to check, that addresses why you’re not seeing the instrumentation.
2016:07:21 19:01:22 alexmiller while check does document this laziness, I admit it was a surprise to me.
2016:07:21 19:02:09 jjcomer Gotcha. Thanks for the debug :)
2016:07:21 19:02:26 alexmiller and then the other thing was not including the sym you’re replacing in the instrument list
2016:07:21 19:05:34 jjcomer Awesome, I'll give it another go this afternoon. Thanks again
2016:07:21 19:09:43 alexmiller np, thx for the repro
2016:07:21 22:50:16 seancorfield If you define a spec with a custom generator (using s/with-gen), does clojure.spec filter the generated values using the spec itself, or does it trust that the generator will only ever produce conforming values?
2016:07:21 22:51:26 seancorfield (I ask because I have a spec with a very complex validation predicate and so I have to write a custom generator and I’m not certain whether the generator I’ve written is only going to produce conforming values by itself…)
2016:07:21 23:14:52 alexmiller It does not trust and always re-checks the custom gen
2016:07:21 23:15:52 alexmiller It's not filtering though - it will error if the gen produces a bad value
2016:07:21 23:19:04 glv @alexmiller: in http://clojure.org/reference/reader#_maps, an example of the #:: behavior would be helpful. I think it’s accurate as it is, but not particularly clear.
2016:07:21 23:21:26 seancorfield Thanks @alexmiller that’s good to know. I couldn’t produce a bad value in 10,000,000 generations so I think I’ll trust it as working for now 🙂
2016:07:21 23:21:47 alexmiller @glv Will consider
2016:07:22 02:20:55 lvh What’s the preferred way to check if a coll has only unique elements?
2016:07:22 02:21:05 lvh Just a pred? I’d like the generators to be efficient.
2016:07:22 02:35:50 codonnell @lvh: I think coll-of and every use clojure.test.check.generators/vector-distinct under the hood
2016:07:22 02:36:29 lvh ah; that solves part of the problem
2016:07:22 02:36:57 codonnell rather, they use vector-distinct if they get :distinct true
2016:07:22 02:37:20 lvh ah! I missed that opt
2016:07:22 02:37:49 lvh does coll-of take the same opts as every?
2016:07:22 02:38:47 codonnell yes
2016:07:22 02:39:20 codonnell this is the entire body of the coll-of macro: backtick before ( ---> (every ~pred ::conform-all true
2016:07:22 02:40:02 codonnell damn, how do you get a backtick in a code block -_-
2016:07:22 03:12:45 lvh ah, awesome
2016:07:22 03:12:54 lvh I should really just read the source of that ns
2016:07:22 03:33:30 lvh I ran some test.check specs, but even with only 100 samples I’m getting actual: java.lang.OutOfMemoryError: GC overhead limit exceeded. I guess recursive data structures can grow big.
2016:07:22 03:40:35 lvh Is there something extra weird that goes on when you test against (for-all […] true)? I was expecting that to trivially pass.
2016:07:22 03:40:54 lvh (that seems like it would be the normal behavior once your code works.)
2016:07:22 03:46:52 glv @lvh is a generator in control of the size (depth or breadth) of the generated structure? The current integer generators in spec grow very quickly. By the 20th test, you're probably in 8-digit range.
2016:07:22 04:06:54 madstap Is there a way to say that if there are any numbers, the string also needs to be there, but if there aren't, it's optional?
(s/def ::xs (s/cat :str (s/? string?) :nums (s/* number?) :key keyword?))
2016:07:22 04:31:10 madstap
;; so this should be valid
(s/valid? ::xs ["s" 2 3 4 :k])
;; And this
(s/valid? ::xs ["s" :k])
;; but not this
(s/valid? ::xs [2 3 4 :k])
2016:07:22 05:24:20 bfabry @madstap: (s/or :regex1 (s/cat ...) :refex2 (s/cat ...)) seems simplest to me
2016:07:22 07:52:03 mpenet What do you think makes more sense for a lib with optional specs: ship with specs in a separate namespace or have specs in a separate repo even. former requires macro hackery to allow to run/use with clj1.9- latter is just a dependency.
2016:07:22 13:00:07 mpenet any idea why this fails when trying to validate with it:
(s/valid? ::foo (fn [x] 1)) -> ExceptionInfo Unable to construct gen at: [:err] for: (instance? clojure.lang.ExceptionInfo %) clojure.core/ex-info (core.clj:4724)
2016:07:22 13:03:40 mpenet oh I see on the guide
2016:07:22 13:04:34 codonnell @mpenet: regarding your earlier question, can you not use the port someone made of spec to clojure 1.8?
2016:07:22 13:04:59 mpenet it was more of a general question when publishing oss
2016:07:22 13:05:44 codonnell oops, I missed your "specs in a separate namespace" option
2016:07:22 13:06:09 mpenet about my recent issue, I wonder why gen is coupled like this to preds
2016:07:22 13:07:12 mpenet I am mostly interested in instrumentation in that case, yet I have to specify a generator apparently
2016:07:22 13:08:10 codonnell I would guess that valid? checks validity by generating some arguments using the :args generator and then checking that the return values match your :ret and :fn predicates
2016:07:22 13:12:27 mpenet apparently
2016:07:22 13:17:08 mpenet so instrumentation works without having to specify more here. good
2016:07:22 13:19:13 codonnell that's good to know
2016:07:22 13:20:33 mpenet well actually no it doesn't
2016:07:22 13:20:35 mpenet damnit
2016:07:22 13:22:58 codonnell it worked for me here:
2016:07:22 13:25:38 codonnell it tries to call valid? on the function returned rather than doing something like propogate the instrumentation to the returned function
2016:07:22 13:26:13 mpenet same when it's on the test suite via instrumentation of the parent fn
2016:07:22 13:29:48 mpenet I understand it's necessary when running valid? on the single spec, but via instrumentation not really
2016:07:22 13:30:26 codonnell I tend to agree.
2016:07:22 13:30:45 mpenet it probably is slow too
2016:07:22 13:31:52 mpenet @alexmiller: any thoughts on this?
2016:07:22 13:35:34 mpenet In my mind instrumentation should "only" be pre/post assertions based on specs. no gen involved (same as plumatic/schema actually)
2016:07:22 14:04:26 lvh at line 36, you’ll see ::properties is a map-of keywords to ::schemas
2016:07:22 14:04:29 lvh (so recursive there)
2016:07:22 14:04:47 lvh it’ll be worse when I add arrays, because those are also recursive json schemas
2016:07:22 14:09:28 glv Oh … no, it doesn’t look like a generated value is in controlling the depth … in fact, nothing is controlling it except chance.
2016:07:22 14:10:57 glv So on line 36, there’s a 1-in-7 chance that the generator will recur and add another level to the structure … but at line 32, there’s a 50% chance. It’s easy to imagine the process rolling the dice just right so that the structure grows until it fills the JVM’s memory allocation, and that seems to be happening.
2016:07:22 14:16:07 glv The problem for writing the specs is this: you’ve accurately captured the semantics for the purposes of verification, but when generating for tests, you want to add some additional constraints. I ran into this when testing a 2D grid … in production code I don’t want to arbitrarily restrict the allowable sizes, but in my specs, I was ending up with grids that had many millions of cells, which was slow, failure-prone (because I also got out-of-memory errors) and also pointless (because a bunch of tests with 25x25 grids will catch any problems — the size of the grid isn’t really the important factor).
2016:07:22 14:16:46 glv So I ended up with this spec:
;; restrict grid sizes for testing, but allow larger grids in production.
(s/def ::grid-dimen (s/with-gen (s/and integer? #(>= % 2))
#(s/gen (s/int-in 2 25))))
2016:07:22 14:17:20 glv It doesn’t enforce an upper bound for validation, but it does for generation.
2016:07:22 14:18:51 glv You might need something similar, but rather than limiting a size (because you don’t have one) you can alter the probabilities, so that the generator for ::additional-properties only chooses to recur 10% of the time, say, instead of 50%.
2016:07:22 14:52:51 alexmiller when running check etc you can supply generator overrides
2016:07:22 14:53:45 alexmiller same for exercise, gen, and instrument
2016:07:22 14:54:06 alexmiller this allows you to override the generator with a more specific one at test time
2016:07:22 14:54:24 alexmiller also see s/*recursion-limit*
2016:07:22 14:56:34 mpenet I saw this. But shouldn't it be possible to avoid generators usage for instrumentation?
2016:07:22 14:57:29 mpenet since it's all predicates, just wrap args/ret
2016:07:22 15:00:25 alexmiller you can do so with the replace/stub functionality in instrument
2016:07:22 15:00:57 alexmiller or maybe I’m not understanding your question
2016:07:22 15:01:16 mpenet I don't know if you read my issue earlier
2016:07:22 15:01:44 alexmiller the fspec thing?
2016:07:22 15:01:54 mpenet yes
2016:07:22 15:02:59 mpenet I imagined that instrumentation would work without having the need to stub anything since it's all specified, I didn't expect it to run "gen" on arguments (hundreds of calls)
2016:07:22 15:03:11 alexmiller there is still some unfinished work in this area
2016:07:22 15:03:34 mpenet oki, glad to hear that, so in the final design "gen" wouldn't be required by instrumentation code
2016:07:22 15:21:00 glv You’ll get a nested schema as the generated value only 10% of the time.
2016:07:22 15:21:40 gfredericks alexmiller: ugh that "open intervals" email makes me wish I'd made the bounds opts on those numeric generators named #{:< :<= :> :>=} :/
2016:07:22 15:21:49 glv (sg is aliased to clojure.spec.gen)
2016:07:22 15:22:22 gfredericks alexmiller: I'd consider deprecating the old opts and adding those four if you think that'd be useful for spec
2016:07:22 15:24:31 alexmiller I don’t think anything needs to change with it, it’s fine
2016:07:22 15:29:38 gfredericks independently I also halfway regret how :NaN? and :infinite? interact with :min and :max
2016:07:22 15:30:06 gfredericks or don't interact rather
2016:07:22 15:30:41 gfredericks but that's harder to change without technically breaking
2016:07:22 16:10:53 rickmoynihan Hmmm... any tips for debugging stackoverflow exceptions in specs?
2016:07:22 16:16:32 rickmoynihan also is it possible to reset the spec registry?
2016:07:22 16:16:47 rickmoynihan pretty sure I've heard people ask about this before
2016:07:22 16:50:07 rickmoynihan I've probably got a mistake in my specs...
2016:07:22 18:39:09 rickmoynihan I had a typo along the lines of by mistake (s/def foo ::foo)
2016:07:22 18:40:24 rickmoynihan Is it possible to call an s/fdef'd function and validate its :ret spec?
2016:07:22 18:40:45 rickmoynihan I saw the guide said it wasn't supported because that's for testing
2016:07:22 18:41:08 alexmiller the only place that’s checked by spec is during check
2016:07:22 18:41:18 alexmiller you can of course obtain and check it yourself
2016:07:22 18:42:00 alexmiller something like (s/valid? (:ret (s/get-spec 'user/foo)) ret)
2016:07:22 18:42:17 rickmoynihan ahh thanks
2016:07:22 18:42:33 alexmiller or s/assert
2016:07:22 18:43:03 rickmoynihan nice - I'd missed that one
2016:07:22 18:53:21 xcthulhu @alexmiller: As I opined on reddit, I don't see any deep reason for why you can't just have in clojure.spec a schema transpiler:
2016:07:22 18:54:12 alexmiller Clojure is not going to provide a schema transpiler
2016:07:22 18:54:28 xcthulhu Why not?
2016:07:22 18:54:30 xcthulhu NIH?
2016:07:22 18:54:37 alexmiller b/c we have other things to do
2016:07:22 18:54:46 alexmiller people are more than welcome to make one
2016:07:22 18:55:28 alexmiller would we also create a truss transpiler?
2016:07:22 18:55:32 alexmiller and a herbert transpiler?
2016:07:22 18:55:33 alexmiller etc?
2016:07:22 18:56:15 alexmiller things in core are forever
2016:07:22 18:56:51 xcthulhu I suppose. IDK, clojure.spec.gen/fmap is in there which is awkward.
2016:07:22 18:57:12 xcthulhu I don't really grok the logic behind what gets in and what doesn't
2016:07:22 18:57:14 alexmiller why?
2016:07:22 18:57:35 alexmiller gen is just a dynamically loaded skin around test.check
2016:07:22 18:58:03 xcthulhu Well, so there's tiny little monad in a framework where everyone did a great job of dodging that sort of thing.
2016:07:22 18:58:33 alexmiller you can ignore it’s monadic nature if you like
2016:07:22 18:58:40 alexmiller it does what it does
2016:07:22 18:59:51 xcthulhu All I'm saying is that great pains were made to wrap test.check, but the lesser pain of wrapping schema type syntax was ignored for some reason.
2016:07:22 19:01:19 xcthulhu It's your product, everyone picks and chooses which wheels they want to reinvent I suppose.
2016:07:22 19:05:52 alexmiller test.check is a Clojure contrib library, following the same contributor agreement, license, and dev methodology as Clojure itself (to make things like this possible)
2016:07:22 19:06:41 alexmiller Schema is not (and I mean nothing negative towards Schema in this regard, they are just much different from a core perspective)
2016:07:22 19:07:51 xcthulhu Okay, so I guess we just need a clojure.contrib.spec.utils library.
2016:07:22 19:07:54 alexmiller spec was designed to solve a set of problems, not to replace Schema (even though it solves an overlapping set of problems)
2016:07:22 19:08:10 xcthulhu With schema and a dynamically loaded test.check/let
2016:07:22 19:08:21 xcthulhu Since that would be super nice
2016:07:22 19:08:29 alexmiller let is tricky as it’s actually a macro
2016:07:22 19:08:47 alexmiller Stu did the work on gen, but I suspect that’s the only reason it’s not there
2016:07:22 19:09:55 alexmiller The Clojure contrib libs are primarily standalone libs without external deps (there are exceptions, but that’s the ideal). There is not going to be a contrib lib related to schema. But there is nothing stopping someone from creating a lib to do what you suggest.
2016:07:22 19:10:58 alexmiller if we think something is fit for spec, it will go into spec, not into a lib
2016:07:22 19:18:04 xcthulhu Okay, I'll post it here for people to playtest and then do the patch, since that's a lot of tedium for 15 lines of code
2016:07:22 20:12:35 mpenet Seems like fmap appears twice in lazy-combinators call fyi
2016:07:22 20:22:21 glv Just for fun, I’ve been prototyping a little library to generate EBNF-ish syntax diagrams from specs. Delving into the spec internal representation has been … eeeenteresting. 😕
2016:07:22 20:31:58 xcthulhu @mpenet: Sounds like someone else is going to be signing something, issuing a JIRA ticket with a simple patch and waiting days and days for a fix
2016:07:22 20:44:14 bbrinck I have the following function defined (defn my+ [x y] (+ x y))
2016:07:22 20:45:12 glv Use a backquote instead:
(stest/check `my+)
2016:07:22 20:45:52 glv (That yields a fully namespace-qualified symbol.)
2016:07:22 20:45:58 bbrinck ah, thank you!
2016:07:22 20:52:05 alexmiller @mpenet thx, I’ll try to get that fixed the next time someone is paying attention
2016:07:22 21:18:46 xcthulhu > Just for fun, I’ve been prototyping a little library to generate EBNF-ish syntax diagrams from specs. Delving into the spec internal representation has been … eeeenteresting.
@glv: This is cool. Does it emit EBNF that instaparse can digest?
I would absolutely love if I could shake SQL tables out of specs. But that's super hard and probably will never happen.
2016:07:22 21:22:08 alexmiller why not model your data and shake specs and tables out of that?
2016:07:22 21:25:51 glv No, but it’s super basic right now. I’m more concerned with deciphering the structures and building code that can navigate them. It’ll be easy to play with the details of what gets generated later. (Plus, Alex said that those internals are still likely to change.)
2016:07:22 23:26:15 bsima can I spec tagged literals?
2016:07:22 23:38:19 bsima nvm, the problem I'm having is that aero does its own eval of the config file, so I can't really get access to the data before the literals are processed (even if I do spec the config file before I pass it to aero, I don't have aero's data readers loaded, so #profile and such is unrecognized). The solution is to spec the data after aero processes it and replaces the tagged literals
2016:07:23 00:06:29 codonnell I'm using clojure.spec to validate responses from a particularly poorly written API, and explain-data is seriously my new best friend. :+1:
2016:07:23 00:42:27 lvh glv: Thanks! That did appear to make things better, but this is still pegging my CPU so I’ll work at it a little more 🙂
2016:07:23 00:59:28 glv Yeah … in my experiments it helped some, but not dramatically.
2016:07:23 01:05:08 lvh part of the problem is that json schema is naturally extremely recursive
2016:07:23 01:05:15 lvh but the ones that actually exist are wide, not deep
2016:07:23 01:05:30 lvh the problem is that that implies statefulness in the recursion, IIUC
2016:07:23 01:05:47 lvh I want to recurse a lot high in the tree, less so at the bottom of he tree
2016:07:23 01:07:14 lvh Is there a shorthand for saying "no other keys” in a way that’s automatically generator efficient? I realize that these are normally intended to be open for extension, but this is an existing spec, and it is not open for extension
2016:07:23 01:24:57 glv I would expect test/check to peg your CPU, though. For this kind of thing it's definitely CPU-bound. If you're not getting out-of-memory errors anymore, that's an improvement.
2016:07:23 01:43:10 alexmiller You can override generators at paths
2016:07:23 01:43:29 alexmiller Could let you use different gens at different levels
2016:07:23 01:53:47 alexmiller Was added recently, not sure there are any docs other than the api
2016:07:23 01:54:09 alexmiller Any place that takes a gen override map can take a vector of keys
2016:07:23 01:54:50 glv Ah, interesting. Will check into that. Thanks!
2016:07:23 01:54:59 alexmiller Although I don't remember now if those are spec keys or path keys or both
2016:07:23 01:55:17 glv Some help you are. ;-)
2016:07:23 01:55:33 alexmiller Yeah
2016:07:23 02:16:35 alexmiller It has to be the path keys
2016:07:23 21:39:08 dominicm @bsima: You can access the reader literals in aero 0.4.0, but spec'ing after is okay too
2016:07:23 22:56:31 nha Is it possible to coerce the content of an edn file ?
My use case: coerce a configuration map (ex. coming from environ where all values are strings to be able to override them from env vars).
It seems that edn/read-string does not accept namespaced keywords. Using load-file seem to lead to problems when trying to aot.
2016:07:24 01:11:00 alexmiller Namespaced keywords should be fine in edn
2016:07:24 01:11:34 alexmiller Autoresolved keywords are not part of edn though (as there is no namespace context)
2016:07:24 01:12:05 alexmiller Do you have an example?
2016:07:24 01:34:45 lvh What exactly does calling spec with a map do? map-spec-impl suggests I can do that, but spec’s docstring only mentions predicates and regex specs; I guess a map can behave like a predicate since it’s an IFn?
2016:07:24 01:56:37 lvh Is there an easy way to get a keys that’s not open for extension? I’m digging into map-spec-impl and just extracting the generator bit isn’t super trivial.
2016:07:24 02:01:21 alexmiller in short, no
2016:07:24 02:01:57 alexmiller you can s/and with a restriction on the key set
2016:07:24 02:03:26 lvh yeah; I got that part; trying to fix the generator though
2016:07:24 02:03:54 lvh I guess I can fmap over the gen I get from keys, and then dissoc the noise it comes up with I don’t want
2016:07:24 02:04:13 lvh (I understand the general argument for having extra keys; it just doesn’t make sense for what I’m doing)
2016:07:24 02:07:18 lvh @alexmiller: Thanks! Much appreciated 🙂
2016:07:24 02:41:15 alexmiller yeah, def leverage the existing specs as much as possible for gen
2016:07:24 16:41:51 lvh When using with-gen, is there some kind of automatic spec verification, or are generators assumed to be correct
2016:07:24 16:48:06 lvh My excl-keys macro ostensibly works, but merging it doesn’t seem to.
Samples for ::a, ::x work fine, merging less so. I guess I’ll dive into how s/merge builds generators 🙂
2016:07:24 18:45:57 xcthulhu Hey guys, I've been working on a patch to port test.check/let to clojure.spec.gen
2016:07:24 18:46:12 xcthulhu Since we all have fmap blues from time to time.
2016:07:24 18:46:16 xcthulhu Here's the unit-test vector I want to push to clojure.test-clojure.spec:
2016:07:24 18:46:21 xcthulhu
(deftest gen-let-macro
(s/def ::a (s/spec keyword?))
(s/def ::b (s/int-in 7 42))
(let [t (s/tuple keyword? string?)
m (s/keys :req [::a ::b])]
(are [spec generator]
(s/valid? spec (gen/generate generator))
t (gen/let [x :abc, y "efg"] [x y])
t (gen/let [x ::a, y string?] [x y])
t (gen/let [[_ y] t] [::a y])
t (gen/let [] [keyword? string?])
t (gen/let [] t)
t (gen/let [[x y] (gen/let [] t)] [x y])
m (gen/let [{a ::a, b ::b} m] {::a a, ::b b})
m (gen/let [a keyword?] {::a a, ::b ::b})
m (gen/let [] {::a keyword?, ::b 7})
m (gen/let [] {::a keyword?, ::b #{7 8 9}})
int? (gen/let [i 'clojure.test.check.generators/int] i)
#{'abc} (gen/let [x 'abc] x))))
2016:07:24 18:47:14 xcthulhu That test vector should give a pretty good idea of how gen/let would work.
2016:07:24 18:47:56 xcthulhu My idea is that it's really annoying having to coerce specs, predicates and constants to generators so it just does it automagically
2016:07:24 18:49:33 xcthulhu I'll wait for @alexmiller to tell me if he likes it or not before I attempt a patch
2016:07:24 19:25:39 mike_ananev hi there! how to specify :ret value for fdef if function returns void?
2016:07:24 19:26:59 xcthulhu nil?
2016:07:24 19:28:36 mike_ananev :ret nil? or :ret #(nil? %) ?
2016:07:24 19:30:30 xcthulhu I think :ret nil? but I'm not 100%
2016:07:24 19:31:33 xcthulhu #(nil? %) is probably wrong
2016:07:24 20:11:50 xcthulhu @alexmiller: Since you're a category theory fan, I could generalize let from generators to specs if we model specs as having type (a -> Bool, Gen a) (á la Haskell)
2016:07:24 20:24:19 alexmiller @mike1452 either will work, I'd use the shorter one
2016:07:24 20:25:41 alexmiller @lvh re with-gen, the generator is not trusted and all generated samples will be verified by checking the spec as well
2016:07:24 20:28:20 alexmiller @mike1452: sorry read further back - functions are never void in Clojure, only comes up in some interop cases
2016:07:25 03:19:34 lvh I guess there’s no way to override that? The issue I’m running into is that merge assumes the key specs it gets are open for extension; so keys-excl merged with anything else fails to produce anything, because the keys-excl part of the spec that checks that the map only contains specific keys will fail on all the keys being added by anything that’s merged in
2016:07:25 07:35:42 mike_ananev @alexmiller: last call in my clojure function is java (interop) fn call which returns void. So in clojure if java fn returns void we see nil, if i understand correctly.
2016:07:25 11:41:38 alexmiller Yep
2016:07:25 12:56:29 xcthulhu @lvh: Instead of merge, have you tried to make an intersection operation?
2016:07:25 12:57:14 xcthulhu Sometimes the intersection of two keys-excl won't work either of course, since you have certain mandatory keys.
2016:07:25 13:01:03 mpenet "closed" key sets seems like a common request, hopefully Rich reconsiders his choice here. I get that he doesn't want to make it the default, but simply providing a way to spec this can't hurt
2016:07:25 13:22:22 alexmiller There are no plans to revisit that afaik
2016:07:25 13:24:58 alexmiller @lvh you might try going down the path of treating your map as a coll-of tuple entries instead (make sure to use :into {})
2016:07:25 13:25:40 alexmiller Then merge that with s/keys to pick up the attribute checks
2016:07:25 13:28:48 mpenet It's super easy to please everybody with a system with "closed" keysets by default (you can just add a k/v spec with any? if you want to "open" the map), but that apparently is out of the question. I personally do not understand the logic here. Never saw anybody complain about the "strictness" of prismatic/Schema for these things for instance.
2016:07:25 13:31:38 mpenet then again, I don't consider the openness of records a good thing either
2016:07:25 14:02:25 lvh @xcthulhu: I’m not sure I understand. As in, remove keys?
2016:07:25 14:03:02 lvh I mean, I guess I could, but that seems very gauche. What would that look like? I have a base spec that everything conforms to, and some stuff that’s specific per type (i.e. multi-spec)
2016:07:25 14:30:34 xcthulhu @lvh:Yeah, you would remove keys. If a map-spec isn't open under extension, it may be open under specialization.
2016:07:25 14:43:18 xcthulhu So provided that your map-specs s1 and s2 that have the form (keys-excl :opt [...] :opt-un [...]) then you could have
2016:07:25 14:44:40 xcthulhu And you'd have the rule:
(valid? (intersection s1 s2) m)
; if and only if
(and (valid? s1 m) (valid? s2 m))
2016:07:25 16:38:52 ghadi @mpenet: search slack history for @richhickey expounding on closed keysets making brittle systems
2016:07:25 17:54:43 mpenet It's kind of in the same class as static vs dynamic lang/systems imho: it depends, sometimes exactitude is necessary. My point was that allowing both would do no harm, it would be optional.
2016:07:25 18:02:12 mpenet I guess that ship has sailed anyway, and likely not open for debate as usual.
2016:07:25 18:41:31 ghadi that's a pessimistic attitude
2016:07:25 18:41:52 ghadi it's not open for debate (IMHO) because it's a bad idea, and the core team can justify why
2016:07:25 18:43:08 ghadi im guessing no one went back to the original discussion. This comes up often enough that it's probably worth documenting why it's not a good idea
2016:07:25 20:41:46 glv That’s a pretty good discussion, and includes a not-too-bad way to accomplish it if you really feel you must.
2016:07:26 03:50:40 wildermuthn Anyone have experience using clojure.spec to spec out macros for use in clojurescript?
2016:07:26 03:51:37 wildermuthn I get odd errors relating to clojure.spec not being found when I run clojure 1.9.
2016:07:26 03:51:42 wildermuthn Might just be my setup.
2016:07:26 04:08:07 glv Use the cljs.spec namespace in Clojurescript.
2016:07:26 16:29:19 nha I have been trying to use clojure.spec to coerce edn (config) files. EDN prevents autonamespacing keywords ( I get that now, thanks alexmiller ), but allow custom #readers. Clojure (well read-string and load-file) allow autonamespacing but prevent custom readers. So it seems that my only option is to explicitly have the keyword namespace every time in my configuration file, is that correct ? I am asking because I find it a bit cumbersome/repetitive to have to have these in my file.
2016:07:26 16:35:20 alexmiller Yes, you'll need full namespaces in edn config files
2016:07:26 16:36:19 alexmiller You could also write specs on unqualified keys with req-un and opt-un
2016:07:26 16:37:35 nha Ah I see - not super familiar with spec yet. Will look into those keys, thanks 🙂
2016:07:26 16:37:59 alexmiller That's in s/keys
2016:07:26 16:39:35 alexmiller Either the normal core reader fns or clojure.edn reader fns should be able to read tagged literals though with #
2016:07:26 16:40:19 alexmiller You just need to bind around the call to set up the readers
2016:07:26 16:41:45 nha Ah I see - I did not knew that. So all is fine 🙂
I will go with clojure.spec (s/keys :req-un [...]) for now.
2016:07:27 02:32:23 lvh Hm, I don’t really understand how to use retag
2016:07:27 02:33:20 lvh I mean, I can use it with a keyword, but now I have something a bit more complex: a schema is either a reference (`{:$ref “some-string"}`) or a schema with a type
2016:07:27 02:34:48 lvh I was hoping to teach the multimethod about that, since s/or doesn’t seem to work
2016:07:27 02:35:32 lvh (when I s/or, it complains that it can’t find an appropriate method, which is true I guess; I wasn’t expecting it to even try that branch)
2016:07:27 02:41:20 lvh I tried doing with with s/or this way:
(s/def ::schema
(s/or ::reference
(fn [m] (contains? m :$ref))
::direct-schema
(fn [m] (contains? m :type))))
2016:07:27 02:41:36 lvh Which doesn’t work because spec doesn’t know how to generate schemas from that pred.
2016:07:27 02:43:44 lvh I can (s/and map? …) but that doesn’t really make it easier to generate maps
2016:07:27 02:49:26 lvh I guess I can just use map? and it can later specify with keys — that appears to sorta work 🙂
2016:07:27 15:39:35 xcthulhu @lvh: I think you are supposed to do it in the style above.
2016:07:27 15:53:09 michaeldrogalis Hello. Is it possible to assert properties about keys and values in a map using s/keys without using the registry?
2016:07:27 16:12:37 xcthulhu Not that I know of.
2016:07:27 16:12:39 codonnell @michaeldrogalis: You can put keys into s/keys that aren't in the registry, and the spec will just check for their existence. It is not possible to provide inline specs to validate map values.
2016:07:27 16:13:36 michaeldrogalis @codonnell: Thanks for the confirmation.
2016:07:27 17:51:29 jonathanj Are there any thoughts on using spec for something like web form validation? Specifically having some way to generate user-readable messages from failed predicates, i.e. more fine-grained than "Name is wrong"
2016:07:27 17:51:55 jonathanj (I don't have much experience with spec, apologies if this is either a really frequent question or a really stupid one.)
2016:07:27 18:15:02 dominicm @jonathanj: IMO, the output would be okay for parsing for a web form.
However, spec doesn't play too well with cross validation, e.g. "is this email taken in the database?"
2016:07:27 18:16:03 dominicm I also found specifying equal fields difficult, with meaningful output, so you'd have a few special cases to handle outside of s/keys
2016:07:27 18:25:29 jonathanj Are you referring to the error data when you say output, @dominicm? I don't see how the error data would be suitable for a user-facing form, you need a layer of translation since arbitrary predicate names are unlikely to be of value to users in most circumstances.
2016:07:27 18:26:00 dominicm My apologies, suitable for translating to user information.
2016:07:27 18:28:40 jonathanj @dominicm: what about involving multiple values in a validation? We have a case where the SSN needs to be validated against the gender of the selected title (don't ask), would this be written as a separate spec that combines the others, with and perhaps?
2016:07:27 18:31:34 jonathanj And perhaps more interestingly, would the error data allow me to determine all the participants in the failing validation?
2016:07:27 18:31:42 dominicm User forms have plenty of awkward cases, domain logic runs rampant here.
I think and would be appropriate yes. You want to ensure the SSN, gender and selected title are all valid before doing the overall validation I assume? That would be s/and.
I don't know the api too well though, my interest dwindled when I tried to use spec for web forms and found limitations in spec.
2016:07:27 18:32:33 dominicm I did find that the output error data was very thorough though, and you could derive a lot of information from it.
2016:07:27 18:33:53 jonathanj @dominicm: You're quite right about awkward cases for user forms, did you go another route in the end?
2016:07:27 18:34:06 dominicm I started writing my own validation library.
2016:07:27 18:34:18 jonathanj Is the code public?
2016:07:27 18:34:29 dominicm Not yet 😞
2016:07:27 18:34:35 dominicm I should open source my attempts.
2016:07:27 18:35:55 jonathanj I've already written two (JS) validation libraries and they are plagued with reinventing existing predicates. I was hoping to find a better way of layering this on something else.
2016:07:27 18:37:50 jonathanj My experience was that a Boolean is not a good enough result for explaining errors to users, but having to decompose the data after the predicate failed was too much duplication of effort
2016:07:27 18:38:28 dominicm Nope, you must attach additional semantics to it. That's where vlad comes in.
2016:07:27 18:39:08 dominicm I just realised that the README does not do it justice.
2016:07:27 20:02:20 seancorfield @bsima: Which version of Clojure? That works for me on Alpha 10.
2016:07:27 20:02:35 bsima 1.9-alpha10
2016:07:27 20:02:40 bsima ¯\(ツ)/¯
2016:07:27 20:05:13 bsima I think a lein clean fixed it
2016:07:27 20:12:28 seancorfield lein clean solves so many evils 😈
2016:07:27 20:52:48 kendall.buchanan @gfredericks: Just following up real quick on that issue last week where explicit loading of test.check and clojure.spec.test/check lead to MultiFn errors. You mentioned master seemed to fix it. Any thoughts on when the next release will be cut?
2016:07:27 21:11:28 gfredericks Probably depends on whether it's still painful for @alexmiller
2016:07:27 21:32:10 gfredericks @kendall.buchanan: I'll prioritize getting a good release together ASAP
2016:07:27 21:33:00 kendall.buchanan @gfredericks: Thank you. I don’t mean to put pressure on you, or anything, but I do sure appreciate it.
2016:07:27 21:34:41 gfredericks No worries
2016:07:27 21:48:44 alexmiller I’m willing to do a release, just would prefer to do as few as possible as it will take me at least an hour to figure out how again
2016:07:27 21:49:09 alexmiller @gfredericks: a good change log wrt to things likely to affect spec would also be helpful to have
2016:07:27 21:50:27 gfredericks @alexmiller: like to send to you folks or for permanent public consumption?
2016:07:27 21:52:03 alexmiller public, not looking for anything special for us, just to have a place to start analyzing impact
2016:07:27 21:52:48 gfredericks Cool
2016:07:28 07:43:09 mpenet @gfredericks: chuck.gen/string-from-regex is nothing short of magic, impressive
2016:07:28 09:42:48 stammi Hi! Maybe someone can spot my mistake in this:
2016:07:28 13:49:49 mike_ananev hi! if i have (s/def ::decimal #(instance? BigDecimal %)) how to make (s/exercise ::decimal) working? I suppose a should create custom generator in spec, but can't find how to do it. thanx
2016:07:28 13:57:58 mpenet I guess something like (gen/fmap #(BigDecimal. %) gen/nat)
2016:07:28 14:46:27 gfredericks Oh right. I meant a good ratio generator :)
2016:07:28 15:53:30 michaeldrogalis I'm a little caught up on s/def being a macro. Is it possible to do something like:
(let [x :my/spec-name]
(s/def x string?))
2016:07:28 15:54:09 michaeldrogalis The problem seems to be that def expands, and the first argument retains the value of the symbol x, not the value that it refers to (`:my/spec-name`)
2016:07:28 16:20:40 featalion you can do it with another macro:
2016:07:28 17:34:51 stammi thanks @alexmiller. that looks better indeed
2016:07:29 09:50:31 mpenet how can I spec the return spec of a function that depends on the args it received : ex ring-spec normal handler vs async-handler: (if args.length == 3 : ret then is nil else response map)?
2016:07:29 09:50:55 mpenet I guess I can hack something with :fn, but it doesn't feel right
2016:07:29 09:53:56 mpenet a "multiple arity" version of fspec could make sense here. it would feel more natural maybe
2016:07:29 09:56:27 mpenet or maybe I can just write 2 fspec and just wrap them with (s/or ..)
2016:07:29 12:22:45 gfredericks You could make the ret spec nilable and describe that relationship in the :fn as you mentioned. Doesn't seem like that much of a hack to me
2016:07:29 12:23:59 mpenet I think I prefer the s/or with the 2 fspecs, makes the relation between args/ret clearer
2016:07:29 13:49:55 alexmiller @mpenet: the whole purpose of :fn is to describe relationships between args and ret
2016:07:29 13:51:21 mpenet got it, so not really good to discern 2 arities of a function like I was doing (I had one "giant" fspec describing both arities/return types)
2016:07:29 13:51:38 alexmiller seems perfectly reasonable to cover each possibility in the :ret (just tell the truth about what it can be)
2016:07:29 13:51:53 alexmiller then in :fn you can declare that the :ret you got matched the :args that were sent
2016:07:29 13:52:13 alexmiller you get conformed values of :args and :ret in :fn, so it should be relatively easy to check
2016:07:29 13:53:01 mpenet Yes I saw this, but wouldn't it be more work for gen down the road having all these branches? I mean vs. (s/or (s/fspec) (s/fspec))
2016:07:29 13:53:17 mpenet I guess it depends what kind of functions we re talking about
2016:07:29 13:53:19 alexmiller it just picks one or the other
2016:07:29 13:53:27 alexmiller that’s why I have a computer :)
2016:07:29 15:00:35 mpenet any teaser of what's coming in the next alpha?
2016:07:29 15:14:13 gfredericks a fully-featured dependent type system
2016:07:29 23:58:41 lvh Is there a pred for large integers?
2016:07:29 23:58:49 lvh Like biginteger or bigint
2016:07:30 00:00:26 lvh I can’t seem to find long? either
2016:07:30 00:01:30 eggsyntax int? will capture all fixed-precision integers. It would still be nice to have a bigint? pred so that the generator specifically produces bigints.
2016:07:30 00:02:04 lvh ah, gotcha
2016:07:30 00:02:06 lvh thanks 🙂
2016:07:30 00:02:11 gfredericks @lvh I don't mind the top level namespace for things that are pretty useful/general
2016:07:30 00:03:34 lvh gfredericks: OK works for me
2016:07:30 01:10:21 alexmiller Doesn't integer? map to that?
2016:07:30 01:10:43 alexmiller Oh you want just bigints
2016:07:30 01:15:03 seancorfield BTW @alexmiller I think the docstring on s/conformer is very confusing about its intent...
2016:07:30 01:15:15 seancorfield "takes a predicate function with the semantics of conform i.e. it should return either a
(possibly converted) value or :clojure.spec/invalid, and returns a
spec that uses it as a predicate/conformer"
2016:07:30 01:17:14 alexmiller Because?
2016:07:30 01:18:37 seancorfield Because the function argument it takes isn’t really a "predicate function"… that makes it sound like it should be something like int?...
2016:07:30 01:19:29 seancorfield …perhaps if it said "takes a conforming function i.e., it should return either a (possibly converted) value or :clojure.spec/invalid"…?
2016:07:30 01:21:32 alexmiller Maybe
2016:07:30 01:24:50 seancorfield (predicate is pretty much otherwise used for functions that return truthy / falsey values in the Spec docs I think?)
2016:07:30 01:26:15 seancorfield & and and take predicates, fdef’s :ret, merge, spec
2016:07:30 01:28:57 seancorfield (although there is mention in a couple of places that some of those "predicate" contexts may also conform the value… so it’s not entirely clear what’s intended)
2016:07:30 01:31:43 seancorfield @lvh @eggsyntax I tested and it looks like you can use (s/and (s/double-in …) (s/conformer bigdec)) if you wanted BigDecimals — and (s/conformer rationalize) if you wanted ratios… and the example above produces big ints
2016:07:30 01:32:45 seancorfield Oh, simpler: (s/exercise bigdec?) — didn’t realize there was a direct generator for that
2016:07:30 01:33:05 seancorfield and (s/exercise ratio?) ...
2016:07:30 01:33:25 seancorfield So bigint? is kind of the only missing one...
2016:07:30 01:35:10 eggsyntax Oh, interesting. I was thinking something roughly like (s/with-gen int? (gen/fmap #(bigint %) int?)) (that's just offhand & may have errors; I don't have a repl at hand).
2016:07:30 01:48:14 seancorfield Using s/conformer seems simpler… but now Alex has made me question whether that’s entirely valid 😸
2016:07:30 01:48:26 seancorfield j/k
2016:07:30 01:50:08 seancorfield We use s/conformer as a way to map input field formats to domain model formats along with specs for input fields. We’re now using specs as part of input validation and as part of our domain model testing.
2016:07:30 01:52:14 eggsyntax Nice. I haven't really played with s/conformer yet (although I'm certainly making use of s/conform).
2016:07:30 17:44:00 seancorfield @dryewo: yes, instrument is to test that functions are called correctly, check is to test that functions are implemented correctly.
2016:07:31 03:00:07 gfredericks that's a nice explanation
2016:07:31 03:15:38 seancorfield (I’m just repeating what Stu said in the latest Cognitect clojure.spec 'cast! 🙂 )
2016:07:31 18:47:27 gfredericks in particular if you're running check while instrumenting with stubs
2016:07:31 18:48:17 gfredericks I don't think there's as straightforward a fix as in CLJ-1949 though
2016:07:31 18:49:07 gfredericks probably the cleanest thing would be a private dynamic var somewhere for registering quickcheck-driven generation fns
2016:07:31 18:49:22 gfredericks it could use gen/generate as the default when you're not actually running a test.check test
2016:07:31 18:54:22 gfredericks shall I make a ticket?
2016:07:31 19:02:15 alexmiller Sure
2016:07:31 19:16:34 gfredericks ACK
2016:07:31 22:43:17 lvh @gfredericks: FYI, I made a bunch of progress on the JSON Schema thing, knocking a few remaining ones down; the really really hard one is references (idk if you’re familiar with JSON schema, but basically you have a base URL and URL-y internal/external references, so you need a way to resolve things like #../../xyzzy/0 or whatever). Dunno if it’s going to be of a size where it’s interesting for spec, but we’ll see
2016:07:31 22:44:10 lvh Are you interested in like, mostly-working-but-missing-stuff impls, or would you rather just pull in an entire impl when it’s done, or rather have it live in its own package
2016:07:31 22:44:38 lvh It’s currently about 250 lines of impl and 220 of test; that’ll probably go up to 300
2016:07:31 22:45:03 lvh it doesn’t have any particularly weird deps, but to do it Right(TM) you need test-chuck’s regex generator
2016:08:01 03:37:53 gfredericks @lvh I was pondering having an "experimental" namespace, so we could probably start it there (c.g.schpec.expermental.json-schema)
2016:08:01 03:38:37 gfredericks schpec depending on chuck is fine and probably inevitable
2016:08:01 09:09:24 rickmoynihan @lvh: @gfredericks: Sorry, just seen the messages above about JSON Schema / spec... curious what you're doing exactly, it sounds interesting and useful!
2016:08:01 09:46:48 mpenet same here, I'd love to see the work in progress
2016:08:01 10:12:53 mpenet I started to dump utils I am using in our projects in https://github.com/mpenet/spex, I might make a couple of PR if there's an agreed "spec utils" project in the works (and interest)
2016:08:01 13:38:53 lvh I’m writing a fn that takes a JSON Schema and produces an equivalent spec
2016:08:01 14:42:39 dominicm @lvh: How possible would the reverse be do you think?
2016:08:01 14:50:00 gfredericks should be possible for a subset of specs presumably
2016:08:01 15:24:30 lvh @dominicm: Trickier, but like gfredericks said possible for some subset of specs; the biggest problem is that you presumably need to statically determine what a particular spec pred means, and then see how much of that you can express in JSON Schema
2016:08:01 15:25:04 lvh it’s not going to be very nice json schema (probably larger, decent amount of duplication) but as long as you treat it like a compilation product and don’t care about bidirectional access that seems fine
2016:08:01 15:25:16 dominicm Yeah, that was my thoughts also.
2016:08:01 15:25:23 gfredericks the fact the predicates are stored as symbols that can't be reliably mapped to vars seems problematic for doing this kind of thing robustly
2016:08:01 15:25:27 dominicm ^^
2016:08:01 15:26:04 dominicm Could you say :foo/bar is this part of JSON schema?
2016:08:01 16:03:27 rickmoynihan a symbol is a namespace qualified value too though right? 'clojure.core/int
2016:08:01 16:06:24 gfredericks Not necessarily
2016:08:01 16:07:06 gfredericks The mapping from symbols to vars is entirely relative to the current namespace setup
2016:08:01 16:08:00 gfredericks You can setup your namespace so that clojure.core/int resolves to something entirely different
2016:08:01 16:08:31 gfredericks But in any case I think spec would only store int? generally
2016:08:01 16:10:53 rickmoynihan surely a macro could resolve it to its bound value though by inspecting ns-map etc... though
2016:08:01 16:33:21 arohner
(s/form (s/spec int?))
clojure.core/int?
2016:08:01 16:34:32 arohner yes, you could set up your namespace so that int? redirects to something else, but spec does store the ns
2016:08:01 16:37:20 eggsyntax @lvh: @seancorfield -- I just had occasion to write a spec for bigints, and was surprised to discover that (int? (bigint 1)) returns false. So I went with:
2016:08:01 16:42:50 eggsyntax Thanks @lvh, I would have missed that.
2016:08:01 16:45:21 seancorfield @eggsyntax: and why doesn’t (s/def ::bigint (s/and integer? (s/conformer bigint))) work for you?
2016:08:01 16:45:49 seancorfield (all I changed from my original suggestion was to use integer? instead of int? there)
2016:08:01 16:46:17 seancorfield I guess the question is should (s/conform ::bigint 1) be 1N or :clojure.spec/invalid?
2016:08:01 16:46:50 eggsyntax Yeah, good question. For my purposes, sure. But I could imagine going the other way in another context.
2016:08:01 16:47:10 eggsyntax The conformer version works too, & is nicer actually, I'll go with that 😄
2016:08:01 16:48:10 eggsyntax I mainly just meant to illustrate that it had to be integer? rather than int?
2016:08:01 17:05:45 seancorfield Your predicate — #(instance? clojure.lang.BigInt %) — is not the same as integer? tho’. Your predicate rejects 1. That was my question.
2016:08:01 17:06:53 seancorfield (and (s/exercise integer?) doesn’t seem to generate bigints in my quick tests)
2016:08:01 17:07:35 seancorfield clojure.spec encourages us to be specific 🙂
2016:08:01 17:56:36 gfredericks w.r.t. generating bigints, I ran into a bug once that didn't surface until the bigints were larger than Double/MAX_VALUE
2016:08:01 17:58:29 gfredericks which makes it interesting to consider how large of a bigint a general bigint generator should test by default
2016:08:01 17:58:49 gfredericks presumably at least bigger than Long/MAX_VALUE
2016:08:01 17:58:53 gfredericks but Double/MAX_VALUE is quite large
2016:08:01 18:02:43 dominicm @gfredericks: I thought some of the automatic tests were around max/min values?
2016:08:01 18:03:00 gfredericks you're asking about default generator behavior?
2016:08:01 18:03:22 gfredericks test.check doesn't have anything like that currently besides favoring small numbers
2016:08:01 18:03:37 gfredericks and nan/infinity
2016:08:01 18:03:57 dominicm Ah right. I was operating based on conference talks. That's unusual.
2016:08:01 18:05:01 gfredericks I'm not sure what the best way to do that is
2016:08:01 18:05:49 gfredericks e.g., if Double/MAX_VALUE should be tested frequently, does that mean the value right below it should be too? or just make that one special? and should it be easy to "shrink" to Double/MAX_VALUE from something more random? or should we always shrink downward?
2016:08:01 18:06:10 glv @gfredericks: I don’t know if you saw the discussion I started a couple of weeks ago about the growth rate on the default spec generators … but the conclusion was that we definitely need integer generators for sizes (that grow slowly and have reasonable bounds) and generators for values used in mathematical functions (that grow rapidly and as large as they can get). Those already exist in test.check (although they’re not labeled/documented for those uses) but you have to jump through more hoops to get sensible bounds in spec.
2016:08:01 18:06:31 gfredericks I think I saw that, yeah
2016:08:01 18:06:37 gfredericks I wish I could just rename a bunch of generators
2016:08:01 18:06:55 gfredericks A docstring overhaul could be good at least
2016:08:01 18:08:29 xcthulhu @gfredericks: You probably weren't considering supporting goog.math.Integers, but if you were you might be interested to know that multiplication is broken for them
https://github.com/google/closure-library/issues/703
2016:08:01 18:09:40 gfredericks xcthulhu: ha, I fixed a couple division bugs in that class a while back
2016:08:01 18:10:03 xcthulhu Oh man thank you I did notice that division works!
2016:08:01 18:12:17 gfredericks I've definitely thought about Integer but I'm not sure how it ought to fit since it's pretty 2nd class in the cljs arithmetic world
2016:08:01 18:12:28 gfredericks could definitely add some stuff in test.chuck for it though
2016:08:01 18:13:10 xcthulhu Well... they don't work... when I need bigintegers I use SJCL and am generally disappointed.
2016:08:01 18:13:23 gfredericks what doesn't work?
2016:08:01 18:13:28 xcthulhu Multiplication
2016:08:01 18:13:38 gfredericks that might be a recent bug
2016:08:01 18:13:44 xcthulhu I ran into that particular bug when I was implementing modular inverses.
2016:08:01 18:13:56 gfredericks which is where I ran into the division bugs
2016:08:01 18:14:15 gfredericks so either I tested on an earlier version w/ the mult bugs or else they're hard to find
2016:08:01 18:15:16 xcthulhu Cool maybe I should port the modular inverse code I wrote since it makes great tests.
2016:08:01 18:15:55 xcthulhu I didn't have time to spelunk goog.math.Integer to figure out the problem.
2016:08:01 18:16:58 gfredericks I'll probably take a look at it; I've already been through their contribution process
2016:08:01 18:21:53 xcthulhu This is a cool library btw. Would you ever want an is-prime? function? Miller-Rabin is rather fast provided you accept the generalized Riemann hypothesis is true.
2016:08:01 18:28:47 gfredericks well you should probably call it probable-prime? then; but I'm not opposed to a namespace for number theory stuff if you find that useful
2016:08:01 19:45:28 glv @gfredericks: Yeah, I think for test.check a documentation update is fine … the right generators are there, it’s just not obvious to a newcomer how different those two use cases are.
2016:08:01 19:45:38 glv … or at least, to this newcomer.
2016:08:01 20:01:13 gfredericks yeah; communicating how sizing works in general is kind of difficult
2016:08:01 21:21:56 pvinis hello. i want to have something like
:nav {:bla 1
:foo [{:ok true
:sure 0}]}
how would i do the s/def on that?
2016:08:01 21:30:47 lvh Is there a way in clojure.spec to specify xor of schemas? JSON Schema has it. Specifically, there’s a way to say that something must match all these schemas, or at least one of them, or exactly one of them (the latter being the xor I’m talking about).
2016:08:01 21:31:10 pvinis also, what happens if i have a key like :ok in some other place, and its a string there? how do i s/def that..?
2016:08:01 21:31:20 lvh I guess I’ll just do it with and and a predicate
2016:08:01 21:32:43 gfredericks @lvh: add that to schpec
2016:08:01 21:32:57 lvh sounds good
2016:08:01 22:37:53 lvh is there a way for a pred that fails to communicate something about why it failed? specifically, i’m implementing xor as described above (aka one-of), I’d like to say the name of the spec that failed the pred
2016:08:01 22:42:46 eggsyntax Specifically the predicate itself? For a spec it's just (s/explain ::the-spec failing-input)
2016:08:01 22:43:33 lvh yeah; the predicate is checking that something is invalid against a particular spec
2016:08:01 22:43:40 lvh and spec doesn’t appear to support negation natively right now
2016:08:01 22:44:03 eggsyntax For a pred you can do (s/explain (s/spec my-pred) failing-data)
2016:08:01 23:26:02 lvh eggsyntax: Sorry, I’m not being very clear. I know how to get spec’s opinion on why the data is bad; I’m trying to improve what s/explain produces when you give it failing data
2016:08:01 23:59:17 eggsyntax @lvh: ah, gotcha. Maybe you can get more detail by splitting it into multiple subspecs? Dunno, just guessing.
2016:08:02 02:39:48 lvh @gfredericks: I don’t know if you’re interested in adding me as a contributor to schpec, but I’d be happy to add e.g. CI and review PRs.
2016:08:02 02:40:46 gfredericks @lvh that'd be cool
2016:08:02 02:53:09 jstokes is there a test runner for specs now? the documentation for fdef references clojure.spec.test/run-tests but I can't seem to find that function
2016:08:02 04:18:42 seancorfield @jstokes: The short answer is "no". Rich has talked about this on the mailing list and in the Cognicast and it sounds like they want clojure.spec to be test tool neutral. You can already use it with clojure.test, Expectations, etc however you want so you can use your existing test runner setup if you want. But clojure.spec is kind of different to regular tests...
2016:08:02 04:19:22 seancorfield ...you may well want to (instrument) your application while running your normal test suite or even while just developing...
2016:08:02 04:20:23 seancorfield ...but generative tests don't always fit in with fast-running "unit" style tests so you probably don't want to just run those in amongst your regular tests.
2016:08:02 13:13:49 jstokes @seancorfield: thanks! that makes a lot of sense
2016:08:02 13:15:38 jstokes i noticed that some of my specs do take a while to run, so probably dont want that included in my normal cider-test workflow
2016:08:02 13:19:35 glv What I’ve always wanted with test.check is a nice way to run the generative tests as part of the usual unit-test suite, but with a much smaller number of tests. And then run them separately with the full deal.
2016:08:02 13:21:02 glv (1000 tests is nicely exhaustive; 25 is a decent sanity check.)
2016:08:02 13:21:28 glv I suspect there’s a way to do that that I haven’t figured out yet.
2016:08:02 14:46:08 jstokes how would you use test.chuck with specs? clojure.spec.test/check returns the check results, not something you can plug in to defspec, right?
2016:08:02 15:03:16 gfredericks you'd probably reimplement something similar that passes num-tests to clojure.spec.test/check
2016:08:02 15:51:08 jstokes ah ok, i didn’t know if that was already in place
2016:08:02 15:51:09 jstokes thanks
2016:08:02 15:52:42 gfredericks something like that could be added to schpec
2016:08:02 16:48:53 lvh I’m writing stuff that produces (s/and …) specs and generates a pred for it; I’d like to make the error message more useful. Right now it just says something silly like:
all-props is a set. It’d be much more useful if it showed the actual set. s/and inspects the form, so I can’t quite figure out how to do that — eval doesn’t quite work because then you get the ugly syntax-quoted form
2016:08:02 16:50:24 lvh how can I communicate the value of all-props for when it fails? Write my own Spec impl?
2016:08:02 22:14:09 gfredericks bsima: def is a special form, you can't exclude it
2016:08:02 22:14:34 bsima mmm ok
2016:08:02 22:54:57 glv In that case, does having it in the exclude clause have any effect?
2016:08:02 23:11:18 gfredericks not that I know of
2016:08:02 23:11:37 gfredericks exclude is silent when the thing doesn't exist
2016:08:03 03:03:20 scriptor right, def doesn't exist in clojure.core, it's part of the base language
2016:08:03 03:22:07 gfredericks You could imagine a not-too-different clojure where special forms are namespaced and excludable
2016:08:03 12:50:18 rickmoynihan I'm curious what people think about the differences/benefits/relationships between:
- Generative Property Based Testing
- Pairwise Testing
In particular is anyone combining the two... It seems to me that you could/should use generative tests to provide instance data within certain equivalence classes of values and then use pairwise testing to generate arguments as test pairs to apply to the function under test.
2016:08:03 13:43:32 borkdude clojure.spec and music generation - has anyone tried it ? 😉
2016:08:03 13:46:46 eggsyntax @borkdude: not I, but I've written some evolutionary music programs, and I think that'd be a really cool approach -- evolve organisms whose phenotype is a spec & generate music from the spec.
2016:08:03 13:47:37 eggsyntax Carin Meier has already some genetic programming w/ spec.
2016:08:03 13:47:52 borkdude @eggsyntax: thanks for the read!
2016:08:03 14:26:33 lopalghost Anyone using spec to parse and validate tabular data?
2016:08:03 15:06:49 mpenet @alexmiller: Should I fill a ticket to have :conform-keys be turned into :conform-keys? (option in map-of), also :distinct-> :distinct?, there are possibly other bool options like these in spec, didn't check all of them
2016:08:03 15:22:17 rickmoynihan lopalghost: not yet - but I totally plan too
2016:08:03 15:22:55 rickmoynihan lopalghost: I think specs and their regexes etc are a natural fit for validating columns/rows of data
2016:08:03 15:35:03 lopalghost @rickmoynihan: I've been using it to read unfriendly excel spreadsheets, it's really helpful
Was wondering what sort of approaches people might be taking
2016:08:03 15:43:37 alexmiller @mpenet you can, not sure on the likelihood of that changing
2016:08:03 15:44:00 rickmoynihan lopalghost: I plan on doing the same thing - as it seems a natural fit... curious to hear about your experiences though
2016:08:03 18:56:20 thegeez is clojure.spec/conform meant to be used in production code (and not only when developing/testing/instrumenting)? For spec/assert it is mentioned explicitly that is should be a no-op and disabled in production, same as with instrumenting and such. Does the same hold for spec/conform? Will the run-time impact be to high to use spec/conform generously everywhere?
2016:08:03 19:02:15 lvh my understanding is that conform is expected at the edges where you take input
2016:08:03 19:02:34 lvh it has the benefit of e.g. automatically classifying multiple options for you (s/or) via a map key
2016:08:03 19:03:02 lvh also, you don’t care about expensive there because a) you’re doing it once b) the alternative is potentially consuming random garbage and having it break somewhere down the road
2016:08:03 19:11:06 thegeez thanks, I hadn't considered the "one time" aspect of using conform. I was thinking of using conform multiple times on the same value but that doesn't work as conform will most likely transform the value
2016:08:03 19:28:46 donaldball For a couple of maps I want to specify, there are some derived values therein which I’d like to specify (and generate) correctly. For example {:a 1 :b 2 :a+b 3}. This is easy to express with (s/and (s/keys …) #(some-predicate)) but of course the generator essentially never passes the such-that filter
2016:08:03 19:29:05 donaldball Has anyone found any good patterns for this situation?
2016:08:03 20:43:13 gfredericks donaldball: running the default generator through gen/fmap would make it easy enough to overwrite key like the one you showed
2016:08:03 20:43:20 gfredericks I don't think clojure.spec makes it easy to modify a generator though
2016:08:03 20:55:40 donaldball I’ve kinda settled on having two specs, one which specifies the map with only independent values, and another which specifies the map including derived values
2016:08:03 21:09:44 gfredericks gen/fmap should help in that case too
2016:08:03 22:28:25 donaldball Yea, it’s not hard to add on, I’m just wondering if anyone else has been writing specs for maps with internal value consistency requirements and, if so, if there’s a declarative approach to expressing them
2016:08:04 02:31:47 misha good morning.
is there a short answer to
are clojure.spec's - a replacement for {:pre ... :post ...} condition-map parameters?
? (assuming sparse usage of the latter)
2016:08:04 02:35:01 gfredericks @misha from my perspective, mostly yes
2016:08:04 02:36:10 misha @gfredericks: thanks
2016:08:04 02:54:46 alexmiller @thegeez you can definitely use conform to destructure values if that’s useful (particularly useful in macros when decoding syntax). Cost is not free of course but will get better.
2016:08:04 02:58:39 alexmiller @gfredericks it’s easy to add a generator with s/with-gen. There are combinations of gen/bind and gen/fmap that can help you work from a model of your data towards generated values. The next (I think) screencast from Stu is going to talk about this more.
2016:08:04 03:00:13 gfredericks @alexmiller: but it's not easy to use with-gen to modify the default generator, unless I'm missing something
2016:08:04 03:03:51 alexmiller It could be easier :)
2016:08:04 03:05:40 alexmiller But if you have just the base spec, you can pull its gen and feed it into fmap
2016:08:04 03:08:54 gfredericks Yep
2016:08:04 16:43:33 calvis is there any easy way to spec a map where the required keywords depend on the values in the map?
> e.g. if (:a m) is 5 then m should also have a :b keyword
I realize this can be done with manually defined predicates and then manually defined generators
2016:08:04 16:46:49 eggsyntax Seems like you could cobble it together with s/or, defining each possible combination separately (&, in your example, presumably two separate specs for the options for :a). But that’s hardly elegant...
2016:08:04 16:47:27 eggsyntax ie an ::a-5 predicate and a general ::a version
2016:08:04 16:48:09 eggsyntax with one alternative containing the former as well as the :b keyword, and the other alternative containing the latter and not containing :b.
2016:08:04 16:48:30 calvis I’m specifically asking for an elegant way, before I go and write a macro that does it
2016:08:04 16:48:50 eggsyntax Yeah, just brainstorming.
2016:08:04 16:49:22 calvis it seems like this would be a pretty common thing to do at least
2016:08:04 18:30:54 gfredericks @bsima: I think java.jdbc does it to be backwards compatible with older versions of clojure
2016:08:04 18:34:10 seancorfield @bsima: FWIW, we’re following a similar pattern at World Singles where our specs are in one namespace and we pull that into other namespaces as needed — but that’s partly because nearly all of our specs so far are very data-focused rather than function-focused.
2016:08:04 18:34:50 seancorfield We haven’t yet decided how that will play out when we’re defining system boundary APIs on top of those specs.
2016:08:04 18:35:41 seancorfield And, yes, java.jdbc needs to support Clojure back to 1.4.0, as @gfredericks says.
2016:08:04 18:37:45 bsima in a recent cognicast rich hickey mentioned using specs as a way to version a codebase, like "if your code matches this set of spec keys then we didn't break anything with this new release" (paraphrasing)
2016:08:04 18:38:17 bsima so, I think that's an interesting way to approach specs, and an argument for separating specs from code
2016:08:04 18:39:17 bsima most of the specs i've written so far have been fdefs, it almost feels like racket's contract system
2016:08:04 19:53:56 alexmiller for another data point, we are currently working on putting clojure.core specs in the namespace clojure.core.specs
2016:08:04 19:54:30 alexmiller not saying that following the pattern is the only way or best way to do it for anyone else’s particular situation though
2016:08:04 21:14:03 rickmoynihan FWIW - I've been experimenting with the specs in a sidecar namespace, but with the fdefs in the foo namespace beneath the functions... not been using spec much yet though - so not sure how it'll work out
2016:08:04 21:40:02 bbrinck I've been placing the fdef right above the function definition. I find that's the most valuable place from the perspective of providing documentation. I like seeing the specs, the arg names, and the docstring all in one place.
2016:08:04 21:41:43 bbrinck but i suspect that specs for common data types will go in a different namespace that can be used in many namespaces
2016:08:04 21:53:18 rickmoynihan you can do that? I'd assumed because fdef takes a symbol for the function its annotating that it would have to come after the definition of the function
2016:08:04 22:03:14 bbrinck @rickmoynihan: yeah it works for me:
2016:08:05 02:08:59 lvh @gfredericks: You saw the PR I put up for schpec a few days ago, right? I’d like to contribute all 3 things I have tickets open for, but I’d prefer to do them sequentially for minimal git exposure
2016:08:05 02:09:44 gfredericks @lvh yep, I just didn't have any free time the last week; I think this weekend is promising
2016:08:05 02:10:05 gfredericks I can probably turn them into a proper release
2016:08:05 02:13:01 lvh sure, no problem 🙂 not complaining, just wanted to make sure you knew it existed
2016:08:05 02:13:12 lvh I will be in Seoul next week with mixed amounts of time available on this
2016:08:05 02:14:16 gfredericks ACK
2016:08:05 10:06:20 kurt-yagram Hey, would it be considered bad practice to put spec/def inside a top-level let?
2016:08:05 10:08:54 mpenet I think so yes, if you need let scoped specs you can use s/spec I think
2016:08:05 10:09:27 mpenet s/def mutates the spec registry so a bit hairy in a let imho
2016:08:05 10:15:59 kurt-yagram alright... thx
2016:08:05 13:36:00 glv Just curious … what would you be defining in the let that the spec would need? I can see maybe putting it in a letfn if you needed a particularly hairy :fn and wanted to compose it from smaller pieces …
2016:08:05 15:47:24 mpenet (s/def ::something (letfn [(yourpred? [x[ ...) (yourgen? [] ...)] (s/spec yourpred? :gen yourgen?)) is nice also for the explain
2016:08:05 15:47:54 mpenet at least it shows something more meaningful than an anonymous fn
2016:08:05 15:57:31 glv Yep, you could put the letfn either place.
2016:08:05 16:08:17 donaldball I’ve been using that pattern like it’s going out of style for my specs of e.g. maps with dependent values
2016:08:05 16:15:14 donaldball Has anyone been able to articulate a reason for or work around the MultiFn errors that often occur when running stest/check?
2016:08:05 16:15:22 donaldball E.g.
CompilerException java.lang.ClassCastException: cider.nrepl.middleware.test$report cannot be cast to clojure.lang.MultiFn, compiling:(clojure/test/check/clojure_test.cljc:95:1)
2016:08:05 16:55:08 gfredericks @donaldball: yes, you just have to explicitly require clojure.test.check.clojure-test
2016:08:05 16:56:45 donaldball Awesome, thanks
2016:08:05 16:59:56 donaldball Alas, I still get that exception
2016:08:05 17:14:16 gfredericks @donaldball: see TCHECK-113
2016:08:05 17:45:40 alexmiller @donaldball I think I added the lein workaround as a comment there to turn off clojure.test monkeypatching
2016:08:05 19:19:44 donaldball FWIW the lein workaround is sufficient to get lein test to not barf, but I needed to explicitly require clojure.test.check.clojure-test in my test namespaces for the check to work in my cider repl
2016:08:05 19:19:46 donaldball ¯\(ツ)/¯
2016:08:05 19:53:21 gfredericks I'll take this as more evidence that a test.check release would be helpful
2016:08:06 00:07:20 lvh Has anyone worked on a datalog spec?
2016:08:06 00:46:48 eggsyntax @lvh: I’ve been doing a spec for our datomic data, but I’m generating specs from our (somewhat customized) schema definition vectors, not from the datalog itself. In retrospect I might have derived them from the datalog in the interest of greater generality.
2016:08:06 00:49:23 eggsyntax We have some custom code that takes things like:
[[:ti/install-type :ti/Location []
"A location in the real world that can be the source or destination of a shipment"
{:id [:uuid :identity "Unique ID for the location"]
:name [:string :fulltext "The name of the loaction"] ...]]
and turns them into vanilla datomic schema defs.
2016:08:06 01:15:22 lvh Yep, makes sense :)
2016:08:06 06:24:21 levitanong Hi all, is there a way to use multi-spec with a multimethod that takes two arguments instead of one? I’d like to be able to dispatch on a key/value outside of the map being mult-spec’d.
Another way of describing the problem is: I have some map of the shape:
:message/payload changes its shape based on :message/action.
2016:08:06 06:27:43 levitanong As far as I can tell, multi-specs only allow you to dispatch on a key/value inside of the data structure.
i.e. it would only be able to do the following:
{:message/uuid uuid
:message/payload
{:payload/action :some-action
;; other data here
}}
This is what I’m currently using, but I think it is more appropriate to place the :action as part of the message, and not the payload.
2016:08:06 20:24:18 seancorfield @levitanong: consider the message as a whole -- although it has the same three keys in every case, the spec for :message/payload would be different in each arm of the multi spec.
2016:08:06 23:07:20 atroche i’ve done the small amount of plumbing required, and what’s left is to experiment with making it look nice
2016:08:07 02:48:21 levitanong @seancorfield: thanks for replying :) so you're saying that I should multi spec the message as a whole? If that is so, then how would I change the spec for the :message/payload for each arm of the multi spec? Is there a way to alias a spec locally? Like if I have two different specs, ::warning-payload and a ::user-joined-payload, how would I be able to assign ::warning-payload to :message/payload inside the (defmulti message :warning ...) body, and likewise for the :user-joined branch?
2016:08:07 02:52:41 seancorfield I haven't looked at the multi-spec stuff. That's just what my intuition said ought to work...
2016:08:07 02:59:41 seancorfield Based on reading the documentation just now, it looks like you could have each method return an s/and of the message spec (with payload just specified as map? perhaps), along with a type-specific predicate for the payload value.
2016:08:07 03:01:08 seancorfield I'm on my phone otherwise I'd crack open a REPL and try to produce a working spec for you.
2016:08:07 03:57:38 levitanong Hmm… Wouldn’t that just end up spec’ing a message against both the message spec and the payload spec?
2016:08:07 03:58:32 levitanong oh wait, I think I see what you’re saying
2016:08:07 05:14:33 seancorfield Yup, that's along the lines of what I was thinking. I don't know how it would hold up for generative testing -- I suspect you'd need to provide s/with-gen based on the ::pin-with-latlng spec -- but it seems reasonable for validation.
2016:08:07 05:52:55 levitanong I haven’t even gotten to generative testing yet 😛
2016:08:07 05:54:38 levitanong I hope rich hickey adds a way to decouple spec map keys from the specs of the keys themselves. I imagine I won’t be alone with this use case.
2016:08:07 05:55:47 levitanong i.e. something like (s/def some-map-spec (s/keys :req [:some/key1 :some/key2] :req-dec {:some/key3 :spec-for-key-3}))
2016:08:07 05:55:51 seancorfield Well, that's why you use namespaced keys -- even for non-qualified keys.
2016:08:07 05:56:39 seancorfield I think that if you have :foo/bar that's a very specific entity in your application, but :bar is not.
2016:08:07 05:56:42 levitanong the predicate with s/valid? works, but it seems hacky
2016:08:07 05:57:13 seancorfield So you can use :quux/bar and :what/bar as different specs for :bar in different contexts
2016:08:07 05:57:30 levitanong hmm. would you say then that I’m being too specific with how i’m namespacing the :message/payload?
2016:08:07 05:58:07 seancorfield If you're using a qualified keyword, it should represent one very specific entity in your domain.
2016:08:07 05:58:37 seancorfield We have :ws.domain.member as a prefix for attributes that belong to "Member" entities.
2016:08:07 05:58:46 seancorfield so things are unique in our domain
2016:08:07 05:59:53 seancorfield :message/payload is not very specific
2016:08:07 06:00:34 levitanong ah
2016:08:07 06:01:18 seancorfield It's interesting... clojure.spec is very opinionated. And it speaks to how the Clojure/core team feel the language should be used.
2016:08:07 06:01:47 seancorfield And if you use it that way, life is easy. If you fight it, life is hard.
2016:08:07 06:01:54 levitanong hahaha
2016:08:07 06:04:14 seancorfield As a guideline, any time I find something feels "hard" in Clojure, I ask here what I'm doing wrong (because it nearly always means I am doing something wrong). And the correct idiom is always easier...
2016:08:07 06:04:58 levitanong Any hint on what I’m doing wrong? 😄
2016:08:07 06:06:26 levitanong Could it be that i’m structuring the shape of my messages wrong?
2016:08:07 06:06:29 seancorfield I don't know that you're doing anything wrong but you might consider whether :message is the right prefix for your keywords... Is that specific enough?
2016:08:07 06:06:51 seancorfield Why is your payload nested?
2016:08:07 06:07:18 seancorfield Having it at the top-level feels a better fit
2016:08:07 06:08:02 seancorfield :message/uuid, :message/action are common and then the other keys would be dependent on the action -- why nest it?
2016:08:07 06:09:50 levitanong The idea is to have it as a generic way to pass information between clients and the server through a websocket. The sender of the message identifies what it would like the recipient to do with the information provided. Sometimes, a payload is not necessary for some specific actions
2016:08:07 06:10:46 seancorfield Is the uuid an inherent part of the data itself? Or is it part of some wrapper concept?
2016:08:07 06:10:59 levitanong Sure, I could bring everything to the top level, but it feels neater to structure it this way. So that everything on the top level is something to do with the message’s metadata: its uuid, who sent it, the timestamp, what action should be done, etc… And then the payload is just a whatever context is needed to perform the action.
2016:08:07 06:11:34 levitanong the UUID is part of the wrapper. It’s a way to facilitate some optimistic update error handling.
2016:08:07 06:12:00 levitanong or some other debugging purpose
2016:08:07 06:12:25 seancorfield If you have a dynamic payload, the discriminant has to be part of that, otherwise it isn't semantically consistent
2016:08:07 06:12:40 seancorfield That's why I suggested considering the message as a whole.
2016:08:07 06:12:40 levitanong So you mean it should be :payload/action?
2016:08:07 06:12:49 levitanong ah, or bringing everything up to the message
2016:08:07 06:13:03 seancorfield If it's really UUID + message then action should be part of the message
2016:08:07 06:13:23 seancorfield It depends on whether the message means something independent of the UUID.
2016:08:07 06:13:52 seancorfield (although maps are open so adding a UUID is OK anyway)
2016:08:07 06:15:00 seancorfield The real issue is that moving action outside payload makes it impossible to analyze payload on its own -- it has no type
2016:08:07 06:15:49 levitanong As an aside:
> If you have a dynamic payload, the discriminant has to be part of that, otherwise it isn't semantically consistent
> The real issue is that moving action outside payload makes it impossible to analyze payload on its own -- it has no type
These seem like very important concepts to learn. Is there a resource where one learns things like this?
2016:08:07 06:16:13 seancorfield I don't know... it just seems... intuitive...?
2016:08:07 06:16:23 levitanong 😁
2016:08:07 06:16:58 seancorfield Not helpful, I know...
2016:08:07 06:17:10 seancorfield ...but what is the atomic element here?
2016:08:07 06:17:56 levitanong I do suppose that the truly important parts here are the :message/action and the contents of :message/payload.
2016:08:07 06:18:29 seancorfield So "payload" is important and it must have an inherent discriminant (i.e., action must be part of it)
2016:08:07 06:18:35 levitanong I see...
2016:08:07 06:18:46 seancorfield And message is uuid + payload
2016:08:07 06:19:12 levitanong I seeeee
2016:08:07 06:19:21 levitanong okay yeah it makes a lot of sense now
2016:08:07 06:19:57 levitanong And with that framework, if I were to rename the keys, I would rename payload to action, and action to type.
2016:08:07 06:21:50 seancorfield Well, "payload" would be a map that contained "action" (as a type / discriminant) and a various keys that depend on the action ... and then it's an open question whether your "message" is uuid + payload map or whether it's a payload map with a :uuid key added
2016:08:07 06:22:03 seancorfield (i.e., nested or not)
2016:08:07 06:23:07 levitanong Okay, I think I have enough information to proceed confidently. 😄
2016:08:07 06:23:20 levitanong Thanks so much, @seancorfield!
2016:08:07 06:24:01 seancorfield Glad that was helpful... sometimes I'm not sure I'm being helpful 🙂
2016:08:07 06:24:58 levitanong I’m sure you always are 😄 cheers! 🍻
2016:08:07 06:25:06 levitanong (helpful, I mean)
2016:08:07 06:25:33 seancorfield I'll raise a Snake Dog IPA to your 🍻🙂
2016:08:07 16:12:50 gfredericks okay I'm going to try to slog through a schpec release today
2016:08:07 16:37:35 lvh @gfredericks: my wife seems bent on getting me to Portage Park (& assoc’d pool) today so you might have a window betwixt merging xor and me putting up weighted-or 😉
2016:08:07 16:39:57 gfredericks ACK
2016:08:07 16:40:13 gfredericks making two releases isn't terrible either
2016:08:07 16:49:50 lvh sure 😄
2016:08:07 21:30:01 gfredericks I'm thinking putting a test runner in schpec
2016:08:07 21:30:24 gfredericks it would allow you to add metadata to functions you want tested, where you could also set up stubs/mocks/whatever
2016:08:07 21:30:39 gfredericks so theoretically you wouldn't need a separate test namespace if you didn't have other tests
2016:08:07 21:30:56 gfredericks is that silly or dumb?
2016:08:08 13:34:17 levitanong Hello, I may have found a typo in the spec guide. http://clojure.org/guides/spec#_spec_ing_functions
> The second :ret predicate takes as input the conformed result of the first predicate and verifies that start < end. The :ret value
> spec is also an integer. Finally, the :fn spec checks that the return value is >= start and < end.
I believe “The second :ret predicate” should be “The second :args predicate”.
2016:08:08 13:52:20 donaldball I like putting my s/fdef before the fn definition, but when I have a :fn argument which needs a reference to the fn itself (e.g. to specify that the fn is idempotent, or maybe inverts on a given arg), it’s a forward reference and thus I must declare the symbol first. Would it be reasonable for the :fn arg to be called with a reference to the function under spec in its map arg?
2016:08:08 14:13:31 levitanong Hurrah! glad to help
2016:08:08 14:15:25 alexmiller @donaldball: what do you mean by “reference to the function” ? do you mean a key that refers to the function? what would the value be?
2016:08:08 14:17:12 donaldball In this case, simply avoiding the necessity of declaring a forward reference to the function being specified, since s/fdef precedes defn
2016:08:08 14:20:12 alexmiller can you give me an example of what that would look like? I don’t think this is something we would do.
2016:08:08 14:24:05 alexmiller but what would your replacement look like?
2016:08:08 14:24:48 donaldball You could add the function-under-spec to the map given to the :fn callback fn
2016:08:08 14:25:48 alexmiller it seems like a smell that you would be calling the function in the :fn spec
2016:08:08 14:26:03 donaldball The other possible use case I was ruminating about the other day would be higher-order fns to specify that e.g. the specified fn is associative, commutative, has a given identity value
2016:08:08 14:26:57 donaldball Maybe these cases shouldn’t be specified here, but a priori it doesn’t seem unreasonable
2016:08:08 14:50:42 glv To me, at least, that starts to seem overspecified, and kind of feels like “the function does what it does”. (I know you mentioned some cases where it would make more sense than that.) I’ve been wrestling with the tension of how exhaustively to specify functions … there’s a point where it starts to feel a bit ridiculous. There’s still a place for prose documentation, and I think there’s also still a place for traditional test.check property tests to deal with some properties.
2016:08:08 14:53:34 gfredericks yeah those things definitely smell a lot more like normal test.check tests
2016:08:08 15:06:12 glv That tension has always existed in contract systems. How do you specify “what the function is supposed to do” without simply restating the (possibly incorrect) code in the function? For some things that’s fairly easy, but for other things it becomes very difficult. In such cases, the typical practice is to include some simple sanity checks, and use traditional testing methods to validate correctness. It seems to me like clojure.spec raises the bar for what makes sense to put in the spec (or contract) … but the bar is still there.
2016:08:08 15:06:57 donaldball So far to summarize that best practices for the :fn arg of s/fdef is strictly to specify some aspects of the relationship between specific args and return values, not properties of the function more broadly?
2016:08:08 15:11:34 glv The rule of thumb I’ve got so far is that I stop trying to make the fdef more thorough when the validation goal starts to overwhelm the usefulness of the spec for internal documentation purposes. (In other words, I don’t want the spec to be so complex that a programmer trying to work in the file has to stop and really think through complex details in the fdef to figure out how the thing is supposed to work.)
2016:08:08 15:12:07 glv I don’t know if that’s the best practice or not, but for something that claims “you get a lot of different uses out of the same thing” that kind of balance strikes me as important.
2016:08:08 15:39:53 alexmiller yes
2016:08:08 15:40:29 alexmiller complicated :fn should (sometimes) also raise questions about whether your function is doing too much
2016:08:09 12:21:12 gfredericks @xcthulhu: I can't resist a good arithmetic bug
2016:08:09 12:24:15 xcthulhu Okay I'm going to have to try to fix one now too. Google Closure doesn't handle UTF-8 properly, so every time I've tried to use it for taking SHA256 of unicode it's been borked.
running into this error, I realize it’s because s/and passes along the conformed value. is there an alternative to s/or or s/and that doesn’t add/pass along conformed information? would rather not have to think about the conformed value of the earlier spec when the spec essentially means “it’s safe to call pos? on it"
2016:08:09 15:59:48 eggsyntax @alexmiller: realizing that there hasn't been an alpha release in nearly a month makes me wonder: would you say that spec has pretty much stabilized at this point? Not that I'm requesting any sort of commitment to that, of course; just curious what your sense is.
2016:08:09 16:19:49 alexmiller no
2016:08:09 16:19:59 alexmiller Rich has just been busy/ooo
2016:08:09 16:20:12 alexmiller We have a ton of stuff queued up :)
2016:08:09 16:20:23 eggsyntax Makes sense, thanks! Looking forward to checking it out 😄
2016:08:09 16:21:33 alexmiller @calvis Rich has suggested having both a flowing and non-flowing version of and and I think it’s likely that will eventually happen (probably and-> for flowing and and for non-flowing, although that changes the current meaning of and)
2016:08:09 16:22:19 calvis @alexmiller: yeah, that would be helpful. can’t really keep track which of my specs are or’s and which aren't
2016:08:09 16:22:44 alexmiller one option here is to use s/conformer in ::basically-a-positive-integer to unwrap
2016:08:10 17:57:22 alexmiller ^^ new screencast on using models to make custom generators with fmap and bind
2016:08:10 18:08:24 eggsyntax Really looking forward to this one 😄
2016:08:10 18:17:38 alexmiller I’ll write a blog that expands on a few things in this too (eventually)
2016:08:10 19:04:49 eggsyntax ^ Good call IMO. Just watched it, and it's great -- especially good for me, because we've been thinking lately about totally restructuring our domain model to be useful for spec as well as other stuff -- but I'm not sure I'd recommend it to someone as their 1st encounter with custom generators.
2016:08:11 12:47:56 rickmoynihan @alexmiller: super useful screencast thanks again!
One thing I've not quite grokked yet is the full relationship between spec and test.check... I understand it's a dev time dependency for the generators... but should you ever be using raw clojure.test generator functions? Or should you use spec as a wrapper or use both?
2016:08:11 12:48:46 rickmoynihan I noticed that spec lazy-loaded test.check underneath too... not sure what the purpose of that is
2016:08:11 13:00:34 alexmiller spec.gen relies on test.check and exposes most of its generator functions
2016:08:11 13:01:31 alexmiller This anything that uses generators will require a dependency on test.check (we expect that to primarily be a test time dependency)
2016:08:11 13:02:42 alexmiller spec.gen wraps those functions to dynamically load them though, allowing you to build generators without incurring a load-time dependency on test.check
2016:08:11 13:03:41 alexmiller So you should always prefer using spec.gen as that lets you use specs in other ways without incurring that dependency
2016:08:11 13:04:32 alexmiller Maybe this itself would be a good blog topic :)
2016:08:11 14:00:55 borkdude What is the reason :ret is not checked by t/instrument?
2016:08:11 15:49:10 gfredericks man we need some kind of canonical link for that question
2016:08:11 15:54:59 glv t/instrument is designed to help you ensure that you’re calling functions correctly, according to spec. Checking that functions (yours or someone else’s) are implemented correctly is a different job, handled by st/check.
2016:08:11 15:56:57 minimal Could pin that ^
2016:08:11 16:01:00 andrewhr not sure how well this will play with our history constraints
2016:08:11 16:01:57 gfredericks amend it with "please re-say&pin this message each morning"
2016:08:11 16:02:13 minimal I see plenty of old things pinned on other channels
2016:08:11 16:02:19 seancorfield @rickmoynihan: we use test.chuck for additional generators and we've taken the same approach of loading it dynamically as a test-only dependency /cc @alexmiller
2016:08:11 16:02:38 glv Yeah, I think pinned stuff sticks around.
2016:08:11 16:03:00 gfredericks @seancorfield: you just generated a bunch of lazy vars?
2016:08:11 16:03:13 andrewhr theres is some documentation update planned regarding this issue @alexmiller? Something the community could help you with? Sorry if the docs were already amended with this point, I didn't re-read them lately
2016:08:11 16:06:39 rickmoynihan alexmiller: thanks for the enlightening answer... And I think it would be another great blog topic 🙂
2016:08:11 16:08:05 rickmoynihan seancorfield: yeah test.chuck is awesome (thanks gfredericks btw) how do you load it as a test time dependency only? Do you mean just through a dev/test profile? Or do you mean you're reusing core.specs lazy loading things somehow.
2016:08:11 16:11:59 ghadi re :ret it in fact says it in the guide -- I think people have a false expectation of the tool to behave in a certain symmetric (for lack of a better term) way. But the real distinction is about development (`instrument` - :ret) vs test (`check` + :ret) time. As you know dev time vs test time is a little blurrier in a REPL.
2016:08:11 16:12:49 ghadi if you think of it (and teach it to people!) in terms of that distinction, it becomes intuitive
2016:08:11 16:23:36 alexmiller @andrewhr sorry, what issue are you referring to? (several things in this channel)
2016:08:11 16:24:35 alexmiller the guide is updated with each alpha so should be in sync
2016:08:11 16:26:11 alexmiller guide discusses it, docstrings I think are up to date. where else should it be?
2016:08:11 16:27:17 andrewhr great! It's a pretty common question and we've not sure if the docs were up to date.. sorry for my laziness in not checking there before asking you
2016:08:11 16:27:18 alexmiller I started working on a reference page for spec which will be kind of somewhere between the docstrings and the guide, eventually I’ll get around to finishing that
2016:08:11 16:28:27 andrewhr looks promising! thanks for working on that 🙂
2016:08:11 16:48:07 seancorfield @rickmoynihan: we wrap stuff we need to use in a function like this
(defn fn-string-from-regex
"Return a function that produces a generator for the given
regular expression string."
[regex]
(fn []
(require '[com.gfredericks.test.chuck.generators :as xgen])
(let [string-from-regex (resolve 'xgen/string-from-regex)]
(string-from-regex regex))))
/cc @gfredericks
2016:08:11 16:49:01 seancorfield In my build.boot file I have
2016:08:11 16:50:12 seancorfield that task is chained in ahead of any testing tasks, to make the libraries available, but scoped so they don’t end up in artifacts (in case a build test accidentally got chained in with a testing task).
2016:08:11 18:07:17 borkdude how do you say in spec: map of keywords and arbitrary values
2016:08:11 18:16:57 borkdude sorry if I'm asking the obvious, but why are there both or and alt? alt is used in sequences, but why isn't or used there?
2016:08:11 18:23:40 alexmiller this is covered briefly in the guide btw
2016:08:11 18:24:03 alexmiller alt describes alternatives that are in the context of a sequence
2016:08:11 18:24:46 alexmiller a spec with alt can thus be reused in different regex op contexts
2016:08:11 18:25:02 alexmiller whereas or represents a spec for a single value
2016:08:11 18:25:39 alexmiller there are situations where either would give you the same result but then there are cases where they would not
2016:08:11 18:26:25 alexmiller it’s better to start from the direction of what you’re spec’ing - if it is syntax in a sequential context, then use alt, otherwise use or
2016:08:11 18:28:25 alexmiller you are describing the structure / syntax of a sequential structure there
2016:08:11 18:28:31 alexmiller so use s/alt
2016:08:11 18:28:43 alexmiller (note that you can’t use s/or here)
2016:08:11 18:29:16 borkdude @alexmiller: thanks for explaining - I tried to find it in the guide, but couldn't understand it as clear as you explain it now
2016:08:11 18:50:45 borkdude @alexmiller: so was the bug that I missed the surrounding s/spec?
2016:08:11 18:51:40 borkdude The guide says: "If you need to spec a nested sequential collection, you must use an explicit call to spec to start a new nested regex context. "
2016:08:11 18:53:38 alexmiller yes
2016:08:11 18:53:49 alexmiller well I mean I’d say that was your error
2016:08:11 18:54:02 alexmiller but it’s a bug that you got a stackoverflow :)
2016:08:11 18:54:14 borkdude garbage in, garbage out 😉
2016:08:11 18:54:15 alexmiller it should have just said invalid
2016:08:11 18:54:27 alexmiller it wasn't garbage though, just not accurate
2016:08:11 19:45:50 stathissideris is there an equivalent of declare but for specs?
2016:08:11 20:39:11 bfabry @stathissideris: you don't need to declare keywords before using them
2016:08:11 21:16:05 rickmoynihan seancorfield: Thanks... at first the require in that function makes me wince... but it's actually a nice trick being able to include the generator in the spec and use the spec in production context without the generator dependency. Nice! xgen/string-from-regex is too nice to use! 🙂
2016:08:11 21:33:47 seancorfield I use runtime require quite a bit. Very useful for conditional dependencies etc.
2016:08:11 21:46:20 donaldball I seem to recall that s/atom-of was bandied about shortly after clojure.spec was released. Is that something we should anticipate in core or has it been discarded as a bad idea?
2016:08:11 21:49:13 donaldball My current use case calls for like an s/promise-of, but I’m curious what the thinking is wrt specs for async values
2016:08:11 22:01:12 alexmiller it was never bandied about by anyone from core :)
2016:08:11 22:01:17 alexmiller I think it’s a bad idea
2016:08:11 22:01:27 alexmiller IMHO
2016:08:12 00:40:19 donaldball Curious if you could elaborate on why. I can see it being an impractical spec to validate outside of narrow test scenarios, but the documentation benefit seems worthwhile even so
2016:08:12 02:10:30 alexmiller because spec is about data and atoms are about state
2016:08:12 02:11:30 alexmiller it makes sense to me to spec the functions used to update an atom though
2016:08:12 02:11:57 alexmiller this is just my own thoughts btw, not sure if rich or stu would agree so take it with that perspective
2016:08:12 06:36:16 borkdude I guess people could use set-validator! with a normal spec? Does anyone ever use it?
2016:08:12 06:41:06 mpenet I used to do exactly that with prismatic Schema. Works well
2016:08:12 11:46:56 alexmiller Yes
2016:08:12 11:54:06 borkdude It's all so... composable
2016:08:12 17:55:29 donaldball Am I… doing something wrong here?
2016:08:12 17:55:33 donaldball
(clojure.spec.gen/vector clojure.spec.gen/char-alphanumeric 5)
AssertionError Assert failed: First arg to vector must be a generator
(generator? generator) clojure.test.check.generators/vector (generators.cljc:446)
2016:08:12 17:58:06 donaldball sorry, I see now, these vars turned into fns, presumably to facilitate lazy-loading
2016:08:15 02:11:44 lvh I’ll probably end up contributing regex-str spec at some point because I gotta write it anyway
2016:08:15 02:19:13 lvh What would be an appropriate name for a spec for a string that matches a regex?
2016:08:15 02:34:13 gfredericks Would re-matches and re-find be weird?
2016:08:15 02:36:41 lvh My first attempt was matches-re?
2016:08:15 02:37:16 lvh I wonder how I’d write the generator for has-re? or whatever; so far I only care about the entire re
2016:08:15 02:37:48 lvh I guess I could generate prefix string, re string, suffix string in parallel and then fmap str/join
2016:08:15 02:45:08 lvh huh, with-gen has a branch for if the spec is a regex
2016:08:15 02:45:41 lvh oh, wait, that means clojure.spec generalized regex, not str regex 🙂
2016:08:15 09:34:41 sandbags Perhaps this is obvious but I just noticed that cljs.spec doesn’t seem to define an assert macro a la clojure.spec
2016:08:15 09:34:59 sandbags seems relatively unlikely to be an oversight so I am wondering what I am missing
2016:08:15 09:38:48 sandbags I guess i can provide my own version of assert*
2016:08:15 10:38:53 brabster If I want to always validate a function's input against an fdef spec, do I need to use the clojure.spec.test namespace? Or does the namespacing suggest that eg. clojure.spec.test/instrument is only for use in development and tests and I shouldn't use it otherwise?
2016:08:15 10:50:36 borkdude @brabster: good question.
2016:08:15 10:52:27 borkdude @brabster: Maybe you can separate out the args spec and refer to it in the fdef - and then call s/valid? in your function (or hook it up with :pre)?
2016:08:15 10:54:13 brabster @borkdude: you're right, I did that but it's quite a lot of extra typing if the individual arg specs aren't really reusable anywhere - and when I use a spec in a precondition the error message doesn't give any details of why the validation failed (just gives the name of the spec)
2016:08:15 10:56:29 borkdude hmm, I see
2016:08:15 10:56:39 brabster plus I seem to frequently end up with what feels like duplication like (s/cat :foo ::foo :bar ::bar :baz ::baz) when I break the specs out - maybe i'm just writing bad clojure or something 🙂
2016:08:15 10:57:24 brabster (tbh I'm struggling a little to work out when/when not to use namespaced keywords too :P)
2016:08:15 10:57:38 borkdude It's quite annoying to get the arguments as a vector that you can validate yourself, isn't it?
2016:08:15 10:58:59 borkdude I get your point. Following the discussion.
2016:08:15 10:59:38 borkdude What I'm also a bit uneasy with is that you have to repeat the argument names and that there's a possible source of duplication error.
2016:08:15 11:01:28 brabster I imagine there are plans to integrate spec more deeply with core stuff like defn so maybe this is a bit painful because it's early days
2016:08:15 11:01:47 borkdude It would be cool if you could write a spec and then the function using the spec:
(defn my-fn ::my-fn-args ...)
Kind of like that.
2016:08:15 11:02:13 brabster Looks good to me
2016:08:15 11:03:02 brabster With the option to disable at runtime if you want to trade off the safety for slightly faster code
2016:08:15 11:03:59 brabster What might be even nicer is combined spec conform and destructuring if that's possible
2016:08:15 11:05:44 brabster Thanks for the input @borkdude, more confident I'm not missing something really obvious now 🙂
2016:08:15 11:06:24 borkdude @brabster: No problem, I'm also just beginning to see the dimension that spec adds to Clojure. Pretty excited about it.
2016:08:15 15:02:46 l0st3d @borkdude: seems really useful ... maybe it should do the conform so you specify something like (defn my-fn [{:keys [destructuring form]} ::my-fn-args] ...) and you could get the destructured pieces in your arguments?
2016:08:15 15:32:37 borkdude I don't know yet, but something with less boilerplate would be awesome
2016:08:15 21:45:30 burbma It seems to think it should be getting a map instead of a seq of maps. Is there a detail I’m missing?
2016:08:15 23:09:19 seancorfield @mmb90: s/* is a regex for zero or more of these specs — so you’re saying name-unicorns takes zero or more arguments, each of which should match the ::unicorn spec.
2016:08:15 23:09:43 seancorfield I think you want s/coll-of for a collection of ::unicorn entities.
2016:08:15 23:20:07 smniel @seancorfield: The s/* is inside of an s/cat pulling out the first argument, so I thought the spec said that name-unicorns takes one argument which is a sequence of zero or more ::unicorns entities.
2016:08:15 23:20:54 seancorfield The s/cat is for the entire :args sequence.
2016:08:15 23:21:31 seancorfield In s/fdef you specify :args as a sequence — as if you always invoked the function with apply.
2016:08:15 23:24:10 smniel Yes. I understood that aspect, but I didn’t understand the way s/cat and s/* were combining to only specify one sequence. I was understanding as if the s/* were wrapped in s/spec which is required to turn a regex-op into a full spec or predicate.
2016:08:15 23:24:18 smniel Does that sound correct?
2016:08:16 00:05:30 alexmiller Yes
2016:08:16 07:33:48 borkdude @arohner: do you have any examples how to use the result of parse-spec?
2016:08:16 13:04:26 sandbags Has anyone come across a spec for Hiccup markup? I’m just getting started and not sure I’m up to trying to figure out if & how to express it.
2016:08:16 13:08:34 sandbags at the moment I am copping out and using vector?🙂
2016:08:16 13:08:38 mpenet start from a tag, then attributes then how they relate to each other (and themselves). should be relatively easy
2016:08:16 13:09:54 sandbags thanks Max but I’m in a hurry and not yet up to the nested spec stuff, if the answer is “no” that’s fine too
2016:08:16 17:42:55 burbma In clojure I can do (gen/generate (s/gen int?)) ;; => 1094 or (gen/generate (s/gen boolean?)) ;; => false. In clojurescript I can do (gen/generate (s/gen int?)) ;; => -308. But in clojurescript I can’t do (gen/generate (s/gen boolean?)) ;; => Error: Unable to construct gen at: [] for: function cljs$core$boolean_QMARK_(x){return (x === true) || (x === false);}]. What’s happening in clj vs cljs to make this difference?
2016:08:16 17:56:07 seancorfield Sounds like no one has written the generator for boolean? in cljs yet.
2016:08:16 17:56:41 seancorfield Are you on the very latest cljs version? If so, file a JIRA ticket.
2016:08:16 18:06:22 burbma I’m on 1.9.216.
2016:08:16 20:33:58 alexmiller agreed, that should work
2016:08:16 20:46:08 seancorfield I see spec’ing of ns has hit master snapshot now @alexmiller ? My build just broke 🙂 This is a good thing — turns out we had (import …) in a test namespace instead of (:import …)
2016:08:16 20:52:42 seancorfield Oh, and a require instead of :require in another file...
2016:08:16 20:55:14 alexmiller yep
2016:08:16 20:55:29 alexmiller that’s definitely the most common kind of failure I found in ns in the wild
2016:08:16 20:57:10 alexmiller there are a number of popular libraries that currently fail with the new specs - I’ve filed PRs on all the ones I found and they’ve all been merged, but afaik most do not yet have new released versions
2016:08:16 20:57:20 alexmiller I’ll have some more guidance on some of this after we release an alpha
2016:08:16 21:00:37 seancorfield Wow, I’m running into more and more of those ns bugs!
2016:08:16 21:03:29 seancorfield Well, this is why we always do multi-version testing against master and our current selected release version!
2016:08:16 21:09:58 seancorfield We had a missing : in six files in all.
2016:08:16 21:10:26 alexmiller well I found instances in both Clojure and ClojureScript too so you’re in good company :)
2016:08:17 10:56:03 stathissideris @seancorfield: (require instead of (:require also breaks tools.namespace
2016:08:17 10:56:13 stathissideris or makes it unreliable
2016:08:17 12:06:20 alexmiller I actually found a case of it in an old version of tools.namespace too (long since fixed)
2016:08:17 14:00:17 ikitommi Hi. Back trying to integrate spec into http api-libs for (border) runtime validation, but need to figure out how to do selective conforming based on runtime parameters (which is needed to do this right). Wrote a small gist of how this is done with Schema - https://gist.github.com/ikitommi/17602f0d08f754f89a4c6a029d8dd47e. Any clues how to do this with spec? Writing conformers that read dynamic vars? Wait for the next alpha? 😉
2016:08:17 16:58:22 sparkofreason Short of writing a custom generator, is there a way to limit the depth of a generated tree from a recursive spec?
2016:08:17 17:01:09 gfredericks @dave.dixon why do you need to limit it?
2016:08:17 17:01:22 gfredericks there might be a var for that actually
2016:08:17 17:01:32 gfredericks but I'm still curious why that need comes up
2016:08:17 17:04:47 sparkofreason @gfredericks: I'm getting stack overflow exceptions. Actually not sure if that's the real issue anyway, since I only get the when I wrap the spec in with-gen to try and limit the number of entries.
2016:08:17 17:10:09 sparkofreason @gfredericks: Come to think of it, they way I'm specifying the tree probably isn't right, since there's an additional restriction on the leaves. Guessing that means I need to write my own generator anyway.
2016:08:17 17:13:54 gfredericks test.check has a helper for recursive generators
2016:08:17 17:14:02 gfredericks you have to be careful with sizing though :/
2016:08:17 17:14:14 gfredericks sizing and recursion have a weird relationship
2016:08:17 17:29:35 sparkofreason @gfredericks: using clojure.spec/*recursion-limit*. Sometimes I get answer from exercise, sometimes a stack overflow.
2016:08:17 17:36:45 gfredericks the *recursion-limit* var and the test.check recursion helper are mutually exclusive
2016:08:17 17:37:03 gfredericks the former only applies to the built-in generator clojure.spec gives you for for recursive specs
2016:08:17 17:37:20 gfredericks I need to write some documentation on sizing in test.check
2016:08:17 18:36:01 ikitommi related to my runtime-coercion question. did a spike about using dynamic var to define the environment for the conformations:
* specs define a special “type” predicate as a first predicate. Allows type-based conformations & allows easy generation of api-docs (we are doing the spec->swagger, https://github.com/metosin/ring-swagger/issues/95)
* type predicates know how to conform value in different environment (should be recursive): string, json & edn formats get all different treatment.
Using a dynamic var sounds like a bad idea, but coudn’t figure out any other way to do this. Any comments?
2016:08:18 08:47:13 magnetophon I have a spec for a sorted set of positive integers in a wide range (say 0 to 1000000). I want those to be keys in a map. how do I spec/def that map? (I'm pretty new to clojure, so I hope I'm using the right terms here)
2016:08:18 08:48:19 magnetophon In the normal use-case, there will only be up to a few dozen keys in the map.
2016:08:18 12:30:43 nha (Not sure if this is the right place to ask) When using clojure.spec with ClojureScript, is it able to figure out which keys are being used in order to minimize the build size ?
2016:08:18 12:52:23 alexmiller @magnetophon: does map-of satisfy?
2016:08:18 13:10:39 alexmiller @nha I don’t know, but I would guess not. @dnolen would know.
2016:08:18 13:14:21 dnolen @nha Dead Code Elimination (DCE) is purely a Google Closure thing
2016:08:18 13:14:36 dnolen still eliminating keys is hardly going to save you anything at all - so I wouldn’t be all that worried about it
2016:08:18 13:15:26 dnolen keywords are subject to constant optimization, i.e. single allocation and global usage
2016:08:18 13:15:29 dnolen this also aids DCE
2016:08:18 13:15:48 alexmiller I think probably the question is whether the spec definition and registration for an unused key can be eliminated
2016:08:18 13:16:13 nha It was unclear, yes that is what I meant.
2016:08:18 13:17:32 dnolen ah ok not currently, since it’s stored dynamically in an atom
2016:08:18 13:19:23 nha Ah I see. It may not be an intended use, I just spec'ed some server-side functions and would like to reuse some of these for validation/coercion on the client side.
2016:08:18 13:19:43 dnolen @nha I don’t see what that has to do with your question
2016:08:18 13:20:52 nha Well embedding the validation/coercions fns for my DB (for instance) is not something I would like to have on my frontend code.
2016:08:18 13:21:05 dnolen so split those apart
2016:08:18 13:21:10 dnolen we’re not going to automate any of that stuff for you
2016:08:18 13:21:20 nha Ok separate namespaces then.
2016:08:18 13:22:08 nha Get it, thanks for your quick answers 🙂
2016:08:18 13:22:13 dnolen np
2016:08:18 15:46:47 magnetophon @alexmiller it partly does, but can spec ensure that each element in the set has one in the map, and the other way around?
2016:08:18 15:48:45 magnetophon @alexmiller maybe I should be using other data structures altogether. I'm learning closure while designing this thing.
2016:08:18 15:57:46 alexmiller to create a spec that makes assertions about a set and a map together, you will either need to spec a data structure that combines them or if you are passing them both as args together, that spec could be on the fdef args of that function
2016:08:18 15:57:56 alexmiller both are possible, I’m just not sure what you’re doing from the description
2016:08:18 16:05:46 magnetophon @alexmiller Thanks! Do you have an example of the combined data structure way of doing it? The guide explains how to combine structures with s/keys, but I didn't see how to assert anything about the combination.
2016:08:18 16:06:36 alexmiller You're talking about both a map and a set so those would have to be combined into something else to talk about them together
2016:08:18 16:06:51 alexmiller Or maybe I totally misunderstand what you mean
2016:08:18 16:07:14 alexmiller You can combine arbitrary specs with s/and
2016:08:18 16:09:06 magnetophon @alexmiller I'll push some code and link you to it. one sec.
2016:08:18 17:31:33 alexmiller That question only makes sense in some context where both are present
2016:08:18 17:32:41 alexmiller Either both parts of the same data value or both args in a call to the same function
2016:08:18 17:33:57 magnetophon @alexmiller you mean, I need to define a data-structure (or function, but let's go with DS for now) that combines both, right?
2016:08:18 17:39:49 alexmiller Where is the code that's going to use these things together? What does that need? It feels like you are doing things that are disconnected from the actual problem you are solving. Specs don't exist in a vacuum.
2016:08:18 17:43:41 magnetophon There is no code yet. I'm brainstorming on how to model my problem by writing specs for the needed data structures. I'm learning clojure at the same time. Am I going about this the wrong way?
2016:08:18 17:47:20 magnetophon I have a proof of concept working. now I want to add a ton of features, but I'm trying to do so in an organized way, that won't lead to too many re-writes.
2016:08:18 17:52:44 magnetophon I thought if I have the data-structures and the states of the FSM that will be the brain of the thing, the rest of the code will be relatively easy.
2016:08:18 17:53:15 magnetophon @alexmiller Does that make sense?
2016:08:18 18:03:43 magnetophon I'm manipulating audio on disk into repeating loops in RAM. The data-structure in question is supposed to represent where in the disk-file each part of the RAM loop came from. I thought I'd use a sorted set for the indexes in the loop where I switch to another part of the source file, so that when I want to replace part of the audio with something else, I can easily find out which nodes to replace. The map contains, for each section, the rest of the data needed to construct the actual audio (so IOW: where does the audio come from for this section, how do I crossfade it, etc). I hope I'm making myself clear.
2016:08:18 18:19:12 seancorfield @magnetophon: It may help to think about whether the "sorted set of indexes" is just an implementation detail of you algorithm or whether it’s an inherently important data structure in its own right?
2016:08:18 18:19:59 seancorfield In other words, do you derive the set from the main data structure "internally", just for that piece of work, or are you passing it around "externally" along with the main data structure?
2016:08:18 18:21:00 seancorfield If the answer is "internal", then I don’t think you need to spec it, although you may spec functions that get passed both the set and the main DS (per Alex’s comment about spec’ing via fdef).
2016:08:18 18:21:53 seancorfield If the answer is "external", then the best solution is to pass around a higher level DS that contains both the sorted set and the "main" DS as elements — and have the spec for that higher level DS (since then it can check that the set element is consistent with the other DS element).
2016:08:18 18:21:57 seancorfield Does that help?
2016:08:18 18:30:40 alexmiller +1 :)
2016:08:18 18:34:14 magnetophon @seancorfield thanks. Needed some time to process that. I think the sorted set is an implementation detail. I might not even need it at all.
2016:08:18 18:37:01 magnetophon I thought I'd need it for performance, but I think I won't: I will be adding and removing indexes regularly, and for deciding which ones to remove or update I'll have to look at the value of the indexes. In practice there will be at most a few hundred indexes, so It'll be quick enough to go over them.
2016:08:18 18:38:20 magnetophon @alexmiller and @seancorfield Many thanks for your time and insight!
2016:08:18 19:56:35 bbrinck I've got a variable length vector (some elements are required, some are optional). I'm using cat to create a spec to check it, which works well. But when I want to generate examples, cat generates lists. Is there a way spec a vector this way?
2016:08:18 20:00:50 bbrinck I can achieve what I want with with-gen, but I believe it requires that I always have two specs - one for the list, as defined with cat and then another to wrap the default generator
2016:08:18 20:11:10 alexmiller Yeah, this is a common problem (regex syntax in vector) and a reasonable workaround
2016:08:18 20:11:30 alexmiller We have talked about creating vcat for this particular case
2016:08:18 20:22:18 bbrinck @alexmiller Glad to know I'm not totally missing something 😄. Thanks for the help. vcat would be very useful in my application.
2016:08:18 20:36:02 seancorfield Can’t you just use coll-of with :into [] for that?
2016:08:18 20:36:50 seancorfield Ah, you’re talking about heterogeneous vectors, not homogeneous...
2016:08:18 21:28:44 alexmiller Yeah. Comes up a lot in macro syntax.
2016:08:19 04:30:40 lvh What’s the standard tactic for taming optionally recursive data structures? E.g. I have a foo that may also contain a foo under ::child; sampling the generator takes a long time because it generates huge foos
2016:08:19 13:06:50 gfredericks huge foos because of the recursion?
2016:08:19 13:18:00 lvh Yep.
2016:08:19 13:19:28 lvh Realistically the number of recursions for real data is like, uh, 1. Most applications won’t do the recursion beyond that 1 level, but some might. Generating “deep” trees is fine and a useful test, but I’d prefer to have it generate wider trees instead (because now generation is taking a while). I’m not super sure yet if it’s actually taking a while or if it’s just CIDER’s repl view that’s super slow or both.
2016:08:19 16:55:40 sparkofreason Having a similar issue, trying to control size of a tree. I don't think *recursion-limit* is working for me. Exercising the spec below sometimes gives results that violate the recursion limit, or even generate a stack overflow.
2016:08:19 17:05:39 sparkofreason Seems a lot better behaved if I redefine as follows. Not sure if this is expected:
2016:08:19 17:12:39 sparkofreason At least I'm not getting stack overflow errors with this case, though trees appear to sometimes be fairly deep regardless of *recursion-limit*, but maybe that's what the docs mean by it being a "soft limit".
2016:08:19 18:34:08 bbrinck What do I need to require to use exercise-fn? With clojurescript 1.9.225 and test.check 0.9.0 I get the following error: #object[Error Error: Var clojure.test.check.generators/->Generator does not exist, clojure.test.check.generators never required]. This is in a namespace where I've required [cljs.spec :as s :include-macros true], [cljs.spec.test :as st :include-macros true] and [clojure.test.check.generators :as gen :include-macros true].
2016:08:19 23:02:28 lvh alexmiller: I have not; messing with a global dynamic var seems inappropriate, I want to affect this specific recursiveness
2016:08:19 23:21:18 alexmiller @lvh do it in a bindings
2016:08:20 05:18:34 kahunamoore I just posted a question to the Clojure ML - has anyone had any issues with danielz/system or stuartsierra/component when used with the latest alpha11?
2016:08:20 05:21:24 kahunamoore I’m using lein (not boot) if that makes a difference… I started with a fresh closp template generated code which ran fine. Then I ran lein ancient upgrade on the project (using :allow-all) and now I’m getting errors related to (require vs (:require … afaik…
2016:08:20 05:55:58 alexmiller I replied on list - there are bugs in system, I have filed PRs, and they have been merged. There are also upstream problems in monger, which system pulls in.
2016:08:20 14:11:24 eraserhd Is there a way to enable fdef return checking?
2016:08:20 14:11:32 eraserhd I thought there was, but can't find it.
2016:08:20 15:24:31 gfredericks @eraserhd: w.r.t. instrumentation, no -- the advice is to use generative testing to test the impl of functions
2016:08:20 15:30:06 eraserhd Ah, OK.
2016:08:20 15:30:37 eraserhd I'm kind of using clojure.spec for debugging, though I guess I could do that.
2016:08:20 15:31:27 gfredericks @eraserhd probably not hard to write a helper aimed at your use case, and you could throw it into https://github.com/gfredericks/schpec if you want
2016:08:20 15:32:14 eraserhd Hrmm
2016:08:20 15:34:19 eraserhd I have a weird tree data structure made out of vectors and numbers that has positional information in it. I did this mostly because I've written too much C. In Clojure, it's neither actually efficient, nor semantic. (You have to compute offsets for information.)
2016:08:20 15:34:59 eraserhd I'm using clojure.spec to migrate it to maps.
2016:08:20 15:36:05 eraserhd I was happy to discover I can actually specify the whole thing with spec. Though the conformed values aren't useful.
2016:08:20 15:49:22 eraserhd (It was a terrible terrible idea.)
2016:08:20 19:29:12 lvh @gfredericks: Is there anything I can do to help https://github.com/gfredericks/schpec/pull/10 land? My first thought was “contribute review time”, but the two other open PRs seem to be blocked on upstream responding to feedback.
2016:08:20 19:33:51 gfredericks @lvh I have some minor concerns but I think they're all improvements that can be made later without breaking anything. Go ahead and merge it, I'll release later today
2016:08:20 19:34:05 lvh cool, thanks!
2016:08:20 19:34:21 gfredericks Sorry I had mistakenly thought everything was blocked on others
2016:08:20 19:35:28 lvh no worries, I’ve maintained open sores projects, I get stuff falling through the cracks 🙂
2016:08:20 22:14:37 gfredericks @lvh schpec 0.1.0 exists now
2016:08:20 22:15:18 lvh @gfredericks: awesome, thanks; I'll ship more features your way
2016:08:21 04:01:30 lvh I’m trying to run stest/check for my entire coebase. It’s failing, but I can’t tell which thing is failing. All I get is:
ExceptionInfo Couldn't satisfy such-that predicate after 100 tries. clojure.core/ex-info (core.clj:4725)
2016:08:21 04:02:13 lvh It’s also not clear how I run this as part of clojure.test; puting it in the toplevel of a ns doesn’t work (`lein test` doesn’t pick it up) — wrapping it in a deftest makes lein test see that ns as a test ns, but doesn’t seem to run the actual checking (no exception)
2016:08:21 04:02:23 lvh I only get the exception when manually evaling stest/check in a repl
2016:08:21 05:24:34 seancorfield The generative testing behind check is / should be very different from your unit testing with clojure.test...
2016:08:21 05:25:52 seancorfield ... you should see something in the stacktrace explaining which identifies which spec's generator is unable to satisfy the predicate ...
2016:08:21 05:41:21 seancorfield ... as you're writing specs, you probably want to exercise them to make sure the generators are working as expected: that would help you find problems like the one above as you created each spec. Not sure whether that's helpful advice @lvh?
2016:08:21 09:12:35 robert-stuttaford @lvh that particular error means that your generator can't produce something that matches your spec. i had this when i tried to s/and two s/keys together; it would generate for the first keys and never find anything that satisfied the second keys
2016:08:21 11:39:34 lvh robert-stuttaford; yep; I know what the error means, but I have more than 1 spec
2016:08:21 11:40:07 lvh @seancorfield: I’ve manually arrowed it down to a particular spec; and gen/sample + s/gen works fine
2016:08:21 11:41:35 lvh @seancorfield: I’ve used test.check before: is the generative testing behind check different enough to not want to run it as part of lein test? Because that’s how I run my normal generative tests
2016:08:21 11:41:59 lvh Also, judging by the Guide, just running all the tests for all the namespaces is an explicitly supported feature
2016:08:21 12:14:10 lvh Is there any intention to make named? publicly available as a predicate such that s/and can maybe take advantage of if? Ideally also not-named 🙂
2016:08:21 13:34:05 alexmiller Use ident?
2016:08:21 13:34:41 alexmiller That was added to core (named? really needs to be replaced in that code)
2016:08:21 13:35:26 alexmiller And #(not (ident? %))
2016:08:21 13:35:29 alexmiller :)
2016:08:21 14:19:33 gfredericks oooh found another thing that clojure allowed before specs were added: (fn a/b [])
2016:08:21 14:36:32 alexmiller Yeah, I fixed a bunch of those, nearly always in a sloppy macro
2016:08:21 14:37:11 alexmiller algo.monads is actually broken on alpha11 due to a similar problem but with defn
2016:08:21 17:03:03 seancorfield @lvh I'm basing those comments on what Rich and others have said about generative testing not being part of "normal" (fast) unit testing.
2016:08:21 17:05:57 seancorfield Like you, in the past, what little generative testing we've done has been okay in our regular Expectations-based test suite. But with the much more extensive use of generative testing in spec.test/check that is not a realistic approach. Unit tests should give near-immediate feedback and take "seconds" to run overall. Generative testing takes much longer so it needs a different approach.
2016:08:21 18:03:03 lvh Sure, that makes sense; I still want to run the tests as part of a test suite even if it isn't the fast, rapid-feedback one though :)
2016:08:21 21:25:52 lvh I’m seeing a lot of failures with stest/check like:
6. { :failure clojure.lang.ExceptionInfo: No fn to spec #:clojure.spec{:failure :no-fn}, :sym clojure.core/ns, :spec
defn and ns not having fns seems like not my problem though — I’m guessing the real solution is to explicitly specify namespaces?
2016:08:21 21:37:09 alexmiller Looks buggy to me
2016:08:21 22:29:31 lvh @alexmiller: I’m guessing that means “you should go file a bug?” 🙂
2016:08:21 22:30:05 lvh Also probably something for core.typed to stop using versions of core.contracts that pull in old versions of core.unified that no longer work 🙂
2016:08:21 23:42:39 atroche has anyone been working on a macro that combines defn and fdef, in a similar way to schema’s defnk?
2016:08:22 02:49:38 lvh atroche: nope but that sounds like a good idea 🙂
2016:08:22 02:50:05 lvh any plans for a with macro for instrument/unstrument? That seems like an obvious thing to want 🙂
2016:08:22 02:50:18 lvh I’m also wondering how to use check in a test suite
doesn’t do anything useful (doesn’t fail when it’s supposed to)
2016:08:22 02:53:03 atroche does check have a return value you can make assertions on?
2016:08:22 02:53:33 atroche standard clojure.test is assertions i mean
2016:08:22 03:00:42 alexmiller check returns results that you need to assert on
2016:08:22 03:01:24 alexmiller @lvh yes bug, and yes on core.typed
2016:08:22 03:01:56 alexmiller @lvh the with instrument is interesting
2016:08:22 03:02:21 lvh Maybe I just haven’t seen the other use cases; it just seems obvious that I’d use it in a deftest
2016:08:22 03:03:31 alexmiller You mean on instrument ? It's useful just to turn on instrumented specs at the repl
2016:08:22 03:13:03 lvh Hm, OK, that makes sense
2016:08:22 03:13:23 lvh I filed that issue, I’ll file another under… core.typed I guess
2016:08:22 03:17:50 lvh Done 🙂
2016:08:22 04:42:05 seancorfield @lvh Have you seen how clojure.java.jdbc deals with instrument in the test namespace?
2016:08:22 04:52:36 seancorfield For stest/check, you could look at stest/summarize-results which prints a nice summary of the success/failure and produces a hash map you can easily assert about (using is, for example).
2016:08:22 04:54:14 seancorfield We use Expectations, and with test.check we (expect (more-of {:keys [...]} ...) (property-based checks go here))
2016:08:22 05:05:57 seancorfield (or you can (map stest/abbrev-result (stest/check 'my-ns/foo)) and make assertions on each of those results)
2016:08:22 12:07:40 brabster I'm having a bit of pain with namespaced keywords - I have code that defines specs as namespaced keywords and expects to see those keywords used in calls to its functions - but it's super easy for a client to accidentally typo a new keyword into my namespace instead of referring to one that I've defined, and I can't catch the mistake if the keyword I've defined is optional
2016:08:22 12:09:08 brabster so I defined a map key as ::s3/bucket-name, and in the client code mistyped ::s3/ucket-name - it seems like a really obvious dumb error but I'm not sure I can do anything to catch it without starting to custom-code validation?
2016:08:22 12:10:46 brabster is there anything I can do in clojure to prevent keywords being defined into my namespace from outside (i.e. making that an error) or in spec to say that only the keys I've defined are valid - or have I misused something?
2016:08:22 12:27:55 donaldball For the latter, (s/and (s/keys …) (s/map-of #{allowed-keys} any?)) is a possibility
2016:08:22 12:45:34 eraserhd @atroche I've always wanted a macro like this. Preferably allowing for with-test as well.
2016:08:22 13:33:22 brabster @donaldball nice, hadn't thought of that - only problem being the need to duplicate union of :req and :opt into it to achieve the effect... but helpful, thanks!
2016:08:22 13:34:19 donaldball Could probably wrap it up in a nice macro
2016:08:22 13:36:02 brabster yeah - it doesn't seem like it would be a terrible idea to just have a flag or something on s/keys that made any keys that aren't in the union file
2016:08:22 13:38:14 brabster I get the argument for the leniency about composing functions to build up maps from the rationale, but it seems like there's a case for the final step in such a process where I need to assert that I've produced a valid result.
2016:08:22 14:41:28 bbrinck @lvh @seancorfield Re: unit/generative tests, with my existing test.check tests, I've had luck using something like times (https://github.com/gfredericks/test.chuck#times) to run a small percentage of generative tests in my normal, fast feedback loop and then run the full amount less often.
2016:08:22 14:42:16 bbrinck I haven't yet tried that approach with the clojure.test generative testing, but it's been a useful technique so far
2016:08:22 18:49:46 jstokes im trying to come up with a way to define a spec for a map, and also a spec for key/value pairs in that map
2016:08:22 18:50:52 jstokes it also seems like something like this doesn’t work, which makes me think I’m headed in the wrong direction
2016:08:22 18:53:21 angusiguess You could define an entry as
(s/or :a ::a :b ::b :c ::c)
2016:08:22 18:54:14 angusiguess That would get you the values only, you might have to do something with tuples.
2016:08:22 18:55:24 angusiguess
(s/tuple #{::a} int?)
2016:08:22 18:55:54 angusiguess Might start to get a little redundant
2016:08:22 21:21:44 alexmiller Entries are collections and maps can be treated as collections of entries
2016:08:22 21:22:56 alexmiller tuple is probably the best match for an entry
2016:08:22 21:24:01 alexmiller So you can spec a map as a coll-of an or of tuples
2016:08:22 21:24:44 alexmiller If you do so, you should also merge it with s/keys to get validation of values
2016:08:23 03:10:46 amacdougall I've been following http://clojure.org/guides/spec, and trying to apply it to some maze-building code I'm toying with. The core data structure is a grid (a two-dimensional vector) of cells (x, y, and a set of open exit directions). I have some specs which seem entirely reasonable to me:
And then I have this pretty basic function, with a pretty basic fspec:
(defn grid-contains?
"True if the supplied x and y coordinates fall within the grid boundaries."
[grid x y]
(and (<= 0 y) (<= 0 x)
(< y (count grid))
(< x (count (nth grid y)))))
(spec/fdef grid-contains?
:args (spec/cat :grid ::grid :x ::x :y ::y)
:ret boolean?)
...I use 350% CPU for fifteen minutes and counting. What gives? Is there some unbounded complexity in my specs? Or is test.check just being very thorough?
2016:08:23 03:19:24 amacdougall (spec/exercise ::grid) does fine, so I want to think that Clojure is able to generate correct test data without any issues. ¯\(ツ)/¯
2016:08:23 03:20:41 amacdougall Final note: when I first ran stest/check, it hit null errors which revealed bugs (yay?). Once I fixed those bugs, I ended up in this situation.
2016:08:23 03:58:33 lvh Can I use spec to spec out a protocol? Just fdef the protocol’s method separately??
2016:08:23 04:23:48 seancorfield @lvh I seem to recall a JIRA issue around protocol functions, but it may be only for instrument...?
2016:08:23 06:58:08 brabster @amacdougall my first thought is maybe your spec is generating enormous grids under test which is chewing up cpu walking around it
2016:08:23 07:13:13 brabster @amacdougall just added a print to your fn and ran (stest/check `grid-contains? {:num-tests 1}), getting values like 32174406 809896496
-13491 -1
-3980 -2856
193086473 -53862035
2016:08:23 07:15:57 brabster Would guess it's the (nth grid y) that's trying to walk along really long linked lists or something - out of time now but hope that helps!
2016:08:23 11:20:33 magnetophon what's wrong with the above?
2016:08:23 11:22:08 magnetophon the generator works, the s/def works if I comment out the with-gen parts, but the whole fails with: "Couldn't satisfy such-that predicate after 100 tries."
2016:08:23 11:24:42 minimal sort turns maps into lists of map entries
2016:08:23 11:26:27 magnetophon @minimal thanks.
2016:08:23 11:26:54 magnetophon what would be the proper way?
2016:08:23 11:32:34 minimal are you trying to gen a sorted-map? @magnetophon
2016:08:23 11:32:45 magnetophon yes
2016:08:23 11:33:55 minimal i think this works (gen/sample (gen/fmap (fn [a] (apply sorted-map (flatten (sort a)))) (gen/map (s/gen int? ) (s/gen int?))))
2016:08:23 12:42:46 magnetophon the weird thing is: when I comment out the :src-index, it works. when I leave that in, but uncomment :length, it also works.
2016:08:23 14:22:51 amacdougall @brabster: Thanks for the pointer—maybe if I refine my spec to declare that the grid should never be more than, say, 10000x10000, and also specify that it must be made of vectors (for faster indexed access), test.check will be a bit less ambitious.
On the other hand, should we expect such a system to generate arbitrarily large collections up to the literal limit of the integer type? I certainly see the benefit, but if a single test takes such a long time to run, how could you run a whole suite without a Bitcoin-mining level of resources?
2016:08:23 14:29:09 amacdougall Last night before giving up, I did change the grid spec to include :kind vector?:
2016:08:23 14:33:03 amacdougall To put it more generally: should developers expect to hand-tune specs to keep generative testing from trying to simulate the entire possible range of inputs? On reflection, the answer is yes: that's what generative testing is for.
But in practice, generative testing on strings generates a bunch of garbage strings, it doesn't generate million-character strings. So why is this test trying to generate a 193,086,473-column grid?
...end of musing.
2016:08:23 14:48:39 magnetophon @minimal ah, found the problem: flatten flattens recursively, so when I have an uneven number of elements I get an error.
2016:08:23 14:49:18 minimal cool, sorry was afk for a bit
2016:08:23 14:50:51 magnetophon np, you've been very helpful.
2016:08:23 14:51:04 minimal np
2016:08:23 15:08:21 amacdougall I'm afraid that setting explicit bounds on my specs doesn't make a difference. I updated the relevant specs to limit both coordinates and grid dimensions:
Perhaps there's something I could do to work around this, but I think for now I'll skip the generative testing.
2016:08:23 15:14:06 amacdougall The other main use of fdef is to instrument a namespace during development/test, which forces spec checking of all function calls, right? That seems like it would still be quite beneficial even without full-spectrum generative testing. 🌈
2016:08:23 16:59:06 raymcdermott Hi guys, I want to do some checking of JSON data with spec, seems like it will be OK - any gotchas?
2016:08:23 17:37:47 magnetophon @minimal I found a solution ^^
2016:08:23 17:40:15 magnetophon when I s/conform some ok values it works as expected. when I try to conform values with a duplicate key, I don't get an error saying that, but instead I get:
Unmatched delimiter: )
2016:08:23 17:41:00 magnetophon should I report that as a bug in clojure, or am I doing it wrong?
2016:08:23 17:54:52 bbrinck @amacdougall Keep in mind that instrument will only check :args, not :ret or :fn specs
2016:08:23 17:57:37 bfabry @magnetophon you should be getting an exception from the reader creating a literal map with duplicate keys
2016:08:23 17:57:45 minimal @magnetophon you can’t have map literals with duplicate keys anyway. The unmatched delimiter must be a symptom of that
2016:08:23 18:01:01 bfabry I mean.. you also can't have maps with duplicate keys either. if you try and construct one one of the keyval pairs will disappear
2016:08:23 18:01:29 magnetophon @bfabry @minimal thanks. It's just that I recently read the clojure spec page, and wanted to try out the promise of usefull error messages.
2016:08:23 18:03:15 bfabry the thing is spec is never coming into play when you're trying to detect duplicate keys in a map. the map data structure just doesn't support that, and the clojure reader also doesn't support that. so it gets chucked out way before spec ever checks it
2016:08:23 18:04:15 magnetophon I was hoping to get a nice sorry, the keys are not unique message from spec. oh well... 🙂
2016:08:23 18:04:40 bfabry I'm kind of surprised the reader didn't give you that error
2016:08:23 18:04:43 bfabry because that's what I get
2016:08:23 18:07:05 magnetophon @bfabry yeah I get that too, but not when I do it in spec
2016:08:23 18:07:29 bfabry ok but you're not doing it "in spec"
2016:08:23 18:07:54 bfabry the error happens pre spec
2016:08:23 18:08:41 bfabry you literally cannot create a map with duplicate keys, so there's no way for spec to look at a map that has duplicate keys and give you an error about it
2016:08:23 18:09:43 magnetophon @bfabry I mean I don't get a usefull error from the reader when the faulty map is inside a conform or explain function.
2016:08:23 18:11:10 bfabry it's just because you've blown up inside the reader, so there's unconsumed input
2016:08:23 18:11:28 minimal Once the reader fails, all bets are off and it gets more errors
2016:08:23 18:12:09 magnetophon @minimal when I run that line, I get the ) error.
2016:08:23 18:12:37 bfabry something about your environment is eating the first error, I guess
2016:08:23 18:12:52 minimal Yeah
2016:08:23 18:13:39 minimal It’s a distracting error but the duplicate key error is the useful one. So it's confusing if it’s missing from your output
2016:08:23 18:14:16 magnetophon @bfabry @minimal ah, I was looking in the wrong place. both errors are there. sorry for the noise.
2016:08:23 18:15:04 bfabry np
2016:08:23 18:15:37 minimal no worries.
2016:08:23 18:19:46 magnetophon What is meant by
By default map-of will validate but not conform keys because conformed keys might create key duplicates that would cause entries in the map to be overridden. If conformed keys are desired, pass the option `:conform-keys true'.
in the spec guide?
2016:08:23 18:21:57 magnetophon afaik conform means to add metadata like the keys of a map. What do they mean by conforming keys? And why would that create duplicates?
2016:08:23 18:28:13 bfabry conforming means modifying the data to fit the spec
2016:08:23 18:45:15 alexmiller and the reason not to do that automatically is that when you are altering the keys of the map, you might end up producing the same key and effectively erasing some of your transformed input, so we err on the side of safety
2016:08:23 18:56:08 magnetophon thanks. so it only makes sense if you have a branch in your key spec, and you want to know which branch was taken?
2016:08:23 18:56:27 magnetophon that is not clear (to me) from the guide.
2016:08:23 21:56:19 alexmiller It only matters if your key spec conforms to something other than the original value
2016:08:23 21:57:04 alexmiller Given the common map keys - strings, longs, keywords, symbols - those are all likely to conform the same and it's a nonissue
2016:08:23 21:57:40 alexmiller So in most cases you can ignore this entirely
2016:08:24 06:42:22 tgk I’m sure my googling skills are just sub-par but I can’t seem to find an obvious way of running (clojure.spec.test/test) from lein e.g. as part CI integration
2016:08:24 06:42:44 tgk Maybe everyone has switched to boot while I wasn’t looking 😉
2016:08:24 07:01:28 tgk The approach I’m taking now is to have a spec-test namespace loaded in dev environments and run -main using lein run -n spec-test/-main (or an alias) and have -main run clojure.spec.test/check and analyse the result. I guess this might be the obvious way of doing this.
2016:08:24 07:54:05 magnetophon Is it expected behavior that s/unform doesn't descend into vectors? IOW:
gives an error because the outer conform gets passed :test-val plus the actual value.
2016:08:24 08:08:43 magnetophon @alexmiller I just noticed your response. Thanks, all clear now.
2016:08:25 12:11:07 nicola Hi all, i think this question already was answered, but - how to restrict collection of keys in map, i.e. no extra keys?
2016:08:25 12:18:55 tgk @nicola: You could write a predicate to cover that case and do (s/and (s/keys …) pred)
2016:08:25 12:19:30 tgk I assume there isn’t a standard way of doing it to be liberal in what you should be able to receive as data
2016:08:25 12:19:40 tgk (seems to be the Clojure philosophy)
2016:08:25 12:32:57 nicola I've already read this thread - https://clojurians-log.clojureverse.org/clojure-spec/2016-06-09.html🙂 We want to use clojure.spec instead of json schema and we have many cases of miss-typed keys, so validating this would be helpful. :closed true would be nice, but let's start from custom pred.
2016:08:25 12:39:52 nicola Then another question: s/and report errors of first failed predicate, could i compose two predicates - so both be applied, even if first failed?
2016:08:25 13:25:43 nicola thx @minimal , i've got it. It's not problem to write extra-keys predicate, but force validation and reporting both extra-keys and s/keys
2016:08:25 13:26:22 nicola i.e. error-greedy s/and
2016:08:25 13:26:35 minimal yeah could be useful
2016:08:25 13:51:31 xcthulhu At one point someone here was trying to convert specs to BNF grammars - what came of that?
2016:08:25 13:54:34 xcthulhu That could be incredibly valuable - it would be very useful if you could compile a spec into a .proto file, or even C++ code for a JNI interface.
2016:08:25 14:01:38 potetm Has anyone had experience setting up clojure.spec.test/check to run automatically (e.g. via lein test)?
2016:08:25 14:03:16 potetm I know you can wrap it in a (deftest foo (testing ...)), but the output of an is failure isn't as verbose as the output of check.
2016:08:25 19:05:21 sattvik Yes, I have, and it has been really useful. It’s just we’re trying to figure out how to use instrument and check together for some functions that may not be specced or only partially specced.
2016:08:25 19:07:03 sattvik The videos on ClojureTV have also been useful, but a more thorough example that shows how to use some of the arguments to instrument to deal with things that might otherwise be considered part of a “test fixture”.
2016:08:25 19:09:40 alexmiller I think Stu might have a screencast in mind for it, but not sure
2016:08:25 19:10:17 sparkofreason I've been getting this exception a lot lately when running check: "Additional data must be non-nil." Any thoughts on why?
2016:08:25 19:11:57 alexmiller that shouldn’t ever happen :)
2016:08:25 19:12:34 alexmiller it basically means that conform failed, but explain didn’t find any problems so ex-info was created with a nil map
2016:08:25 19:13:06 alexmiller unfortunately this exception happens while throwing the useful exception so it’s hard to diagnose
2016:08:25 19:13:19 alexmiller but it’s definitely a bug whatever it is
2016:08:25 19:13:25 alexmiller a bug in Clojure that is
2016:08:25 19:13:43 alexmiller and if you can figure out what it is, I’d love a jira on it
2016:08:25 19:28:23 alexmiller it’s not too hard to hack Clojure itself to dump more info when this happens if you have the interest in doing so
2016:08:25 20:19:05 sparkofreason If I can come up with a consistent minimal repo, I'll file a ticket.
2016:08:25 20:22:03 alexmiller thx
2016:08:25 20:41:16 sparkofreason I'm going to venture a guess, though, based on some recent debugging, which is that it seems to occur when I have a function that returns another spec'ed function, and the returned function throws an exception while spec is checking it. Just a guess, will try to see if that stick consistently when I have a little time.
2016:08:25 20:59:48 alexmiller hmm, well I guess that’s possible too (although I’d put that one on you :)
2016:08:25 21:20:12 alexmiller ok, I’ll put it on Apache :)
2016:08:25 21:36:17 shaun-mahood @alexmiller: Any expectation of previous spec knowledge for your Conj workshop, or can we come in blind?
2016:08:25 21:36:27 alexmiller nope
2016:08:25 21:36:42 alexmiller I will expect basic Clojure knowledge :)
2016:08:25 21:38:28 shaun-mahood Perfect, I should be able to handle that 🙂
2016:08:25 21:39:02 alexmiller for sure
2016:08:25 21:54:24 sebastianpoeplau This might be a stupid question, but can anyone give me a hint how to spec the following function:
(defn my-apply [f & args]
(apply f args))
(This is a simplified version of what I actually have.)
My problem is in expressing that (count args) has to be equal to the number of parameters that f takes...
2016:08:26 00:38:34 gfredericks That sounds hard.
2016:08:26 02:11:39 alexmiller the hard part is that you don’t know how many params f takes
2016:08:26 02:12:42 alexmiller I don’t know that it’s profitable to spec such a thing at that level
2016:08:26 05:53:06 rymndhng is there a way to take a map schema of un-namespaced keys, and promote them to namespace qualified ones? -- I'm imagining something like:
2016:08:26 08:42:52 alexmiller this is kind of an aside and I assume you did this really for readability, but you’ll never see ::id printed - autoresolved kws are only read, never printed
but you could certainly use a conformer to do this
2016:08:26 14:27:08 vikeri This has probably already been asked, but can I somehow pprint the error messages thrown by (spec.test/instument)? They are pretty much impossible to read for what I’m doing right now.
2016:08:26 17:54:44 alexmiller Not yet
2016:08:26 19:39:19 sveri Hi, seems I missed a few updates and instrument-all is not in clojure.spec anymore. What is the idiomatic approach to enable the specs during tests with the latest alpha release?
2016:08:26 19:45:04 sveri @alexmiller Ah, there it is and without arguments it instruments the ns, right?
2016:08:26 19:45:24 alexmiller it instruments everything in the registry
2016:08:26 19:45:38 alexmiller when you pass it no args
2016:08:26 19:45:40 sveri Even better, thank you very much
2016:08:26 19:45:53 alexmiller same thing for unstrument and check
2016:08:26 20:32:02 fenton (cross-posted from clojurescript...but no answer there): do others find it difficult to spec functions that use core-async for backend comms? This seems where stubbing would be very helpful, that is to stub out backend service calls. However since core-async based functions only return channels, not request responses...how do other people tackle this issue?
2016:08:26 20:42:32 dnolen @fenton there’s no story for that yet
2016:08:26 20:43:02 fenton @dnolen ok...
2016:08:26 20:45:11 fenton ok, i'll nut it over with some clojure devs here at our Vancouver, Canada, meetup...see what we come up with.
2016:08:26 21:26:04 fenton maybe this came up before, but wondering if this is being thought about. There is some overlap in clojure.spec'ing and datomic schema spec'ing. Could these be unified?
2016:08:26 21:53:45 alexmiller eventually I expect that spec will be more intimately involved in both core.async and Datomic
2016:08:26 21:54:01 alexmiller but that’s a ways off
2016:08:26 21:58:20 fenton @alexmiller cool. glad to hear that is the direction tho! 🙂
2016:08:26 22:38:23 mjhamrick Is there a better way to do the following? I'm figuring the registry-ref is private for a reason, but I couldn't come up with another way to temporarily override the value of a spec. My usecase is that during runtime, it is okay for a spec to be nilable, but not during test time. Looking for any suggestions for this. I did try writing the specs separately, but that ended up with (quite a bit) of duplication since the spec I need to redef is nested a few levels down.
(defmacro with-spec-redefs
[& forms]
`(let [previous-registry# (s/registry)]
(let [result# (do
2016:08:27 15:58:07 sattvik @fenton I believe exercise returns tuples of [val conformed-val].
2016:08:27 16:02:34 sattvik @mjhamrick I think the way to do that is via instrument. When you instrument, you can pass something like (stest/instrument some-ns/some-fn {:spec {`some-ns/some-fn (s/fspec …)}})`
2016:08:28 14:30:45 lvh I’m trying to use regex ops and running into some problems. I have two data structures naturally represented by a regex, the main one is of the shape: a*c, in my map data structure in order to get the item at that location, the get-in path is something like (ab)*c — it seems that the problem is that nested regex operators “merge”; so now for a fn that takes a*c and returns (ab)*c, I can’t specify its :args as (s/cat :a ::a) — does that make sense?
2016:08:28 14:32:19 lvh Looks like that’s intentional; found it in the guide 🙂
> When regex ops are combined, they describe a single sequence. If you need to spec a nested sequential collection, you must use an explicit call to spec to start a new nested regex context. For example to describe a sequence like [:names ["a" "b"] :nums [1 2 3]], you need nested regular expressions to describe the inner sequential data:
2016:08:28 14:50:20 alexmiller Yes
2016:08:28 14:50:26 alexmiller That's intended
2016:08:28 14:51:53 lvh Makes sense — you’d want to be able to parse both, and break them out 🙂
2016:08:28 16:36:22 sveri Hi, how do I spec functions with optional arguments?
2016:08:28 20:34:51 lvh sveri: usually you use regex ops to describe the args as they would be given to apply
2016:08:28 20:34:58 lvh so .e.g you can use spec/?
2016:08:28 22:07:18 potetm How would you spec 2 arguments where one arg's validity depends on the other arg.
2016:08:28 22:08:12 potetm For example: (fn [i-am-a-map i-am-a-key-in-the-map] ...)
2016:08:28 22:31:10 gfredericks potetm: you can use s/and
2016:08:28 22:32:14 potetm @gfredericks perfect; just figured that out 🙂
2016:08:29 01:03:51 potetm next question: is there an ad hoc way to supply a :gen for function :args to stest/check?
2016:08:29 01:05:54 potetm I see you can s/def the args, then supply a gen. I'm not opposed to the notion that you must name something to use it elsewhere (even if that elsewhere is a test), but I thought there might be some sugar out there.
2016:08:29 01:06:24 sattvik I think you can do it with instrument.
2016:08:29 09:16:00 sveri @lvh Thanks, I just took ? It works as expected 🙂
2016:08:29 13:17:00 lvh Is there any chance of a with-instrument macro, and/or a clojure.test + clojure.spec.test.check deftest macro a la defspec with test.check? (Too many similarly named words; I feel like I screwed that up)
2016:08:29 13:34:47 sattvik We just started really looking into clojure.spec last week, and while we are still figuring out how to best leverage it, I definitely agree that a clojure.test integration and a with-instrument macro sound like useful additions.
2016:08:29 14:58:35 sveri Hm, I am not sure, but, so far I would want to have everything instrumented during my tests
2016:08:29 14:58:49 sveri Thats the first thing I do in my tests, turn on instrumentation
2016:08:29 15:08:53 sattvik Yes, in general, you want to have everything instrumented. However, if you are using stubs or overriding specs, you may only want to do that within the scope of a few checks.
2016:08:29 16:30:10 xcthulhu Hey! I'm trying to cryptographically sign data structures in Clojure/ClojureScript. The approach I am taking is to serialize my structures into a canonical form - namely recursively sorting the keywords of every hash-map in the structure, prior to serializing with transit.
(1) Is there already something that does this for me?
2016:08:29 16:30:27 xcthulhu (2) Maybe this isn't the best practice, is there a better way?
2016:08:29 16:31:09 xcthulhu (3) If there isn't a better way, how do you enforce that a data structure is canonical in spec?
2016:08:29 16:36:21 lvh @sveri +1 on also wanting instrumentation with stubs/overrides as sattvik mentioned
2016:08:29 16:36:44 lvh @xcthulhu Hello I am a cryptographer
2016:08:29 16:36:52 lvh Solving a similar problem, turns out
2016:08:29 16:37:23 lvh Fortunately I don’t particularly care about canonicalization or deterministic signatures
2016:08:29 16:37:37 lvh I’m personally using nippy, but I don’t think that has any particular canonical form guarantees
2016:08:29 16:38:03 lvh @xcthulhu How do you keep the hash-map structure in the serialized form whilst being sorted?
2016:08:29 16:39:20 xcthulhu Serialized form comes after the thing is canonical. Yeah, I've gone to pretty extreme lengths to keep all of my signatures deterministic.
2016:08:29 16:40:10 xcthulhu I'm doing ECC in ClojureScript and I don't believe I can get a reasonable secure RNG without resorting to RFC 6979
2016:08:29 16:40:19 lvh Sure; when you say canonicalizing a data structure the first things that come to mind are ASN.1 C(X)ER
2016:08:29 16:40:43 xcthulhu Yeah... I've implemented a fragment.
2016:08:29 16:40:56 xcthulhu I could go whole-hog. I was thinking of just using transit
2016:08:29 16:41:13 lvh Fortunately RFC 6979 + ed25519 is how you’d want to do it anyway regardless of how much you care about that determinism 🙂
2016:08:29 16:41:29 lvh How do you ship the ClojureScript?
2016:08:29 16:42:12 lvh (The devil’s advocate argument against JS crypto usually goes by how you ship the source to begin with)
2016:08:29 16:42:31 lvh Oh; OK — why SECP256k1?
2016:08:29 16:42:53 xcthulhu Because I started this when I was doing consulting for digital currency and it's what everyone uses because of BitCoin
2016:08:29 16:42:58 lvh Either way this is not a #spec discussion anymore so maybe we should move to #crypto 🙂
2016:08:29 16:43:00 xcthulhu I should switch it to ed25519
2016:08:29 19:57:49 lfn3 @fenton A thing I’ve done previously with schema is wrap a channel with a validation to check anything put into the channel conforms to the schema. This causes exceptions to be thrown where you’re putting the unexpected value onto the channel which is really useful. Sure you could do the same thing with spec if you wanted to.
2016:08:29 19:58:24 lfn3 Is there a way to attach a doc-string to a spec?
2016:08:29 19:58:50 fenton @ifn3 that sounds like a great use of specs
2016:08:29 22:09:17 emrehan Hello all. I'm stuck at defining a map with undefined keys with clojure.spec.
For instance, I'm defining a map with keyword keys and string values. I can define it as below with Schema library:
(require '[schema.core :as schema])
(def Data {schema/Keyword schema/Str})
(schema/validate Data {:x "str"})
> {:x "str"}
(schema/validate Data {:x 0})
> ExceptionInfo Value does not match schema: {:x (not (matches-some-precondition? 0))} schema.core/validator/fn--8583 (core.clj:155)
2016:08:29 22:28:32 bfabry @ag not sure what you're saying, could you give a (s/valid ::account-name ...) => true example? also is accounts known at spec definition time?
2016:08:29 22:45:11 ag so when I do (gen/generate (s/gen ::foo-or-bar-or-baz)) it would give me one of the maps of accounts vector
2016:08:29 22:47:06 bfabry assuming accounts is known at spec definition time, and accounts is reasonably small, then (s/def ::foo-or-bar-or-baz (set accounts))
2016:08:29 22:59:14 emrehan Is there a nice way to combine to clojure.spec definitions? I have a definition that must conform two defined schemas and I defined it as below:
2016:08:29 23:01:02 bfabry so (s/and ::spec1 ::spec2) is fine
2016:08:29 23:02:55 bfabry that's actually not that clear from the docstring is it? it just says "spec-forms"
2016:08:29 23:09:12 emrehan I didn’t look at the spec but I tried that form. It didn’t work, maybe I’m missing something.
2016:08:29 23:10:50 bfabry if ::spec1 conforms then the conformed value will be passed to ::spec2 instead of the original value
2016:08:29 23:11:02 bfabry maybe that's your issue?
2016:08:29 23:13:18 bfabry if it is you could (s/and any? ::spec1 ::spec2) I suppose
2016:08:29 23:15:22 emrehan thanks for the tip. this behaviour is not obvious though. let me try that
2016:08:29 23:17:20 bfabry I think s/and conforming with the first spec has tripped a few people up so the docstrings are probably being worked I imagine. iirc the guide does cover it though
2016:08:29 23:19:48 ag is there equivalent of clojure.test.check.generators/let? in clojure.spec?
2016:08:29 23:25:48 emrehan Here’s a simplified example
2016:08:29 23:28:54 emrehan It seems to me that s/and passes the return value of (s/conform ::a-map {:id 1 :n 1 :s "dsa"}) to the second conform. However, the when s/or is conformed the return value is different from the input.
2016:08:29 23:31:45 bfabry yes, conforming means changing the value to explain how it is true given the spec, ie if the spec has a branch
2016:08:29 23:33:19 bfabry I think maybe s/and has changed so it successively conforms the values now
2016:08:29 23:37:01 bfabry anyway, you have options. if you want to preserve the conforming behaviour, I would just re-order your predicates so that ::with-id comes first. if you want to do away with it then creating a new predicate with #(s/valid? ::a-map %) just around the spec that you don't want to conform anymore seems sensible
2016:08:29 23:38:19 bfabry OH
2016:08:29 23:38:26 bfabry or you could use the new merge function
2016:08:29 23:38:35 bfabry which is specifically for map specs
2016:08:29 23:44:04 bfabry I think something named def should always be at the top level
2016:08:29 23:44:40 bfabry and in that particular case I don't think it will work because keys is a macro
2016:08:29 23:45:30 bfabry so the keys macro is passed the form '[(s/def ::bar string?)] instead of [::bar]
2016:08:29 23:46:48 alexmiller @bfabry s/and has always flowed conformed values. We may change it to not flow and add s/and-> still as both are sometimes useful
2016:08:29 23:47:13 bfabry ah, cheers, not sure where I ever got the idea that only the first did
2016:08:29 23:52:10 alexmiller @ag s/keys always requires registered keys - that's part of the design that is talked about at http://clojure.org/about/spec
2016:08:29 23:52:56 ag ok… thanks..
2016:08:29 23:56:00 ag how do I create a spec for uuid? this doesn’t seem to work:
(s/def ::id g/uuid)
2016:08:29 23:57:45 ag I don’t want to use java.util.UUID/randomUUID I may have to make it later work on cljs side too
2016:08:29 23:59:18 bfabry @ag there's a uuid? predicate in core
2016:08:29 23:59:57 ag oh, ok… thanks!
2016:08:30 00:13:30 ag oh no.. I don’t think I know how to solve it with s/keys… the problem where I need to have a spec for a map where values picked up from a vector and associated values are together
2016:08:30 00:16:40 ag so this is how I am generating a map with test.check.generators:
2016:08:30 00:28:52 ag that’s the thing, account-names are predefined. and types are predefined. I need to have a map with bunch of keys along with name and type where name comes from that vector and type is the associated type (from the same vector)
2016:08:30 00:35:39 bfabry ok, honestly it's strange enough that there's probably not a way to express it and still get generators so you'll probably need to specify the generator like you did above, and do the spec portion separate
2016:08:30 00:39:21 bfabry I'd probably write the spec as
2016:08:30 00:49:13 alexmiller @ag it sounds like you are expressing what have been called hybrid maps
2016:08:30 00:49:46 alexmiller Which you can spec with a merge of keys and every of tuples of key-value pairs
2016:08:30 00:49:59 alexmiller I've given some examples here in the past
2016:08:30 00:51:40 ag Yeah I’m watching Stu’s screencast “customizing generators”, looking into gen/bind, gen/tuple etc.
2016:08:30 01:25:50 ag so why for simple predicates generator won’t work:
(s/def ::more-than-five #(< 5 %))
(s/conform ::more-than-five 6)
(s/exercise ::more-than-five)
clojure.lang.ExceptionInfo: Unable to construct gen at: [] for: :user/more-than-five
2016:08:30 01:26:16 gfredericks @ag that would be a hard thing to support in general
2016:08:30 01:28:13 gfredericks if you do (s/and integer? #(< 5 %)) it will at least make an attempt
2016:08:30 01:28:33 gfredericks because clojure.spec has a registry of generators for some of the built-in predicates
2016:08:30 01:29:19 ag oh, ok… thanks!
2016:08:30 01:30:25 gfredericks np
2016:08:30 01:47:18 ag once again, how do I create s/def with a custom generator? I think I saw it somewhere?
2016:08:30 02:03:18 sattvik I beleive it is (s/def ::foo (s/with-gen spec gen))
2016:08:30 02:22:50 ag @sattvik Thanks!
2016:08:30 07:24:24 ag erhm… how do I refer to a spec s/deffed in other namespace? I am trying to put spec in .cljc file, is that possible at all?
2016:08:30 07:45:55 ag I think I found it… so if spec is in foo.clj then require [foo :as f] … (s/exercise ::f/my-spec
2016:08:30 07:46:34 ag still can’t figure out how to refer to spec (and use it without having to prefix it with namespace)
2016:08:30 10:25:17 emrehan thanks all; clojure.spec/merge is exactly what I was looking for. I even read the whole API but I was quite sleepy.
2016:08:30 10:26:29 emrehan How can I contribure to clojure.spec? docs/dev/testing/benchmarking?
2016:08:30 12:16:16 jannis Hi. Using the latest ClojureScript version, I'd like to get all checkable syms (with cljs.spec.test/checkable-syms, filter them and call clojure.spec.test/check on each of them individually). However, if I do something like (doseq [sym syms] (st/check sym)) it tells me sym is unresolvable. This works fine in Clojure but in ClojureScript, am I supposed to pass in something else than symbols?
2016:08:30 15:57:19 spinningtopsofdoom I'm looking to make a spec that validates nested maps (e.g.` {:value {:foo :bar :key :value}}`).
2016:08:30 15:57:44 spinningtopsofdoom So far I'm only able to get one level of nesting via spec/keys
2016:08:30 15:59:43 sattvik You can use two different specs: one for the inner map and then use it in the outer map.
2016:08:30 16:00:07 sattvik There are other options, but I think that’s one of the easiest.
2016:08:30 16:03:27 spinningtopsofdoom I also have the need to have different specs for value (e.g. http://clojure.org/guides/spec#_multi_spec) . For example {:value {:foo :bar}} and {:value {:key :value}}
2016:08:30 16:05:05 sattvik Well, in that case, I think you could do with more specs:
2016:08:30 17:12:42 ag generates: ("pn7zcHi1WDk830ZltM5L5Yl82E0Ul”)
2016:08:30 17:12:49 ag and not a map {:id “somenonsense”} ?
2016:08:30 17:14:50 djwhitt s/cat specifies a sequence. ":id" in that code is specifying a label rather than a map key
2016:08:30 17:15:48 ag so what should I use instead, if I don’t want to use s/keys?
2016:08:30 17:17:51 bfabry you should use s/keys
2016:08:30 17:18:02 djwhitt hmm... I'm not sure (just learning spec myself). any reason you can't use s/keys?
2016:08:30 17:19:14 ag because it doesn’t allow “inline specs” e.g. s/keys (s/def :id string?)
2016:08:30 17:20:57 djwhitt I think spec is intentionally trying to steer people away from inline specs like that
2016:08:30 17:21:36 djwhitt you could make a separate spec for the key and use req-un to allow unnamespaced keys if that's what you're looking for
2016:08:30 17:21:51 bfabry @ag giving every spec a fully qualified name is one of the core design goals of spec, things are going to get difficult if you try not to follow it
2016:08:30 17:22:46 ag mkay
2016:08:30 17:34:26 ag guys, I need a good spec for values of milliseconds after the Unix epoch. i.e.:`clj-time.coerce/to-long`
2016:08:30 18:16:18 alexmiller should really use int-in if you’re defining a spec (not just the predicate int-in?), as then you will get generator support
2016:08:30 18:18:38 alexmiller and there is a new fn for extracting the ms - inst-ms in core
2016:08:30 18:20:58 alexmiller so I would say (s/int-in 0 (inst-ms #inst "2100-01-01"))
2016:08:30 18:23:25 alexmiller assuming you’re cool with the year 2100 problem
2016:08:30 18:24:44 alexmiller inst-ms calls into the Inst protocol, which is extended to both Date and (if you use JDK 8), Instant. Could also be extended to a JodaTime Instant.
2016:08:30 18:27:15 ag I don’t understand why this is failing:
2016:08:30 18:28:16 sattvik You may want to use :req-un instead.
2016:08:30 18:28:26 alexmiller :: will autoresolve and fully qualify
2016:08:30 18:28:33 alexmiller oh you mean in the data
2016:08:30 18:28:33 alexmiller yes
2016:08:30 18:28:41 alexmiller sorry! :)
2016:08:30 18:29:14 ag oh, ok… thanks a lot!
2016:08:30 18:35:59 ag Sorry for bugging you with bunch of noob questions. Trying to solve real problem with spec. If I don’t get this right sooner, would be asked to stop my experiments.
2016:08:30 18:36:14 alexmiller noob questions are good
2016:08:30 18:36:41 alexmiller I asked Rich a lot of noob questions at the beginning too :)
2016:08:30 18:37:36 ag is there a way to pass original value (I dunno of spec) to custom-generator function, when it’s created with s/with-gen?
2016:08:30 18:38:02 ag something like (gen/generate (s/gen (s/with-gen ::account-balance-updated #(gen-account-balance-updated %))))
2016:08:30 18:39:06 ag I guess I can refer to ::account-balance-updated from inside the gen-account-balance-updated, yet thinking if there’s more “generic” way
2016:08:30 18:41:20 ag eh… I guess I can just pass it as is
2016:08:30 18:41:23 ag nevermind
2016:08:30 18:41:58 sattvik Well, you can do something like use gen/fmap or gen/bind.
2016:08:30 18:43:16 sattvik Though, I am not sure that really answers your question…
2016:08:30 18:47:47 alexmiller @ag in short, no
2016:08:30 18:48:02 alexmiller but you can define both the spec and the spec-with-custom-gen to do so
2016:08:30 18:48:09 ag so I have a spec for a map structure, I need to use custom generator function where I would generate all the fields of that map, except of few selected, those should come from a predefined variable
2016:08:30 18:49:59 alexmiller it’s easiest to use gen/fmap and source it with (s/gen (s/keys ::a ::b)), then in the function just merge with the constant map
2016:08:30 18:50:09 alexmiller where ::a and ::b are the variable parts
2016:08:30 18:56:08 alexmiller @eraserhd what’s the point of that gen/tuple?
2016:08:30 18:56:11 sattvik Could it be that the tuple is adding an extra layer of being a collection, i.e. instead of returning [task] it is returning [[task]]?
2016:08:30 18:56:37 alexmiller seems like you just need vector in that case
2016:08:30 18:58:17 alexmiller or you might just want to fmap with vector inside the fn
2016:08:30 18:58:50 eraserhd @alexmiller Er, we found out that spec conforms the generator's results to the spec.
2016:08:30 18:59:12 eraserhd We had something with multiple values in the tuple, but deleted code until we found out where there was a problem.
2016:08:30 18:59:21 eraserhd (except that we didn't know that part)
2016:08:30 19:02:52 alexmiller yes, spec does not trust the generator
2016:08:30 19:03:22 alexmiller even custom gens must produce values valid for the spec
2016:08:30 19:23:05 sattvik Hmmm… so what’s the best way to create fixtures (for lack of a better term) with spec? This could range anywhere from setting up an environment within which to run the tests to testing that the output from the function under test matches a value generated from a replaced/stubbed function.
check seems really useful for relatively pure functions, but having to orchestrate a lot via instrument seems clunky. Does anyone have any recommendations?
2016:08:30 19:32:06 alexmiller if you want fixtures and assertions, use a testing framework which has them ?
2016:08:30 19:32:25 alexmiller that is, wrap a clojure.test around instrument+check
2016:08:30 19:32:47 alexmiller instrument could go in a fixture too if it applies broadly
2016:08:30 19:36:33 sattvik Yes, I’ve done a bit of that. I have gotten it to work, but I was a bit unsatisfied with the result. I felt that the end result was a bit clunky and harder to understand/maintain compared to just using a traditional deftest.
2016:08:30 19:37:31 sattvik I will continue to experiment with it, but I was curious to see if anyone else had any experience with non-trivial testing using spec.
2016:08:30 19:38:48 alexmiller what was unsatisfying?
2016:08:30 19:49:45 sattvik Well, I am still a bit of a spec beginner (have been using it for less than a week), so there are things I may be missing. Some things that could help include:
1. More documentation around instrument, especially with examples of using the various options.
2. It would be nice to be able to call instrument with just options rather than (instrument (instrumentable-syms) opts).
3. The lack of a with-instrumentation macro makes setting up ‘fixtures’ harder. It was a simple one to write, but it would be nice to have it baked in.
4. Overall, specifying things through instrument doesn’t read as well. When this sort of thing is written out long-hand in code, it is easier to understand how the flow works. Instrumentation is sort of like defining callbacks before you need to use them, which removes some of the context from them.
4. The last one it is a little bit harder to describe. Instrumenting is very good about modifying things that will happen during the execution of the function, but sometimes I want to do something around the execution of the function. I can do that by specifying a test function and running spec on that, but it would be nice if there was a more direct way of doing that with spec.
2016:08:30 19:58:50 alexmiller 1. for sure :) more will be coming eventually.
2. did you find clojure.spec.test/enumerate-namespace? it helps with building sym lists.
3. there is a ticket for that that I was just looking at. there are questions - feel free to weigh in. http://dev.clojure.org/jira/browse/CLJ-2015
4. I can see what you mean, prob worth seeing more examples
4. that’s interesting. sounds like a check fixture? I assume you meant “running check”, not “running spec” above
2016:08:30 19:59:34 gfredericks woah CLJ tickets have exactly caught up with the calendar year
2016:08:30 20:00:26 alexmiller now if we can just slow down to 1 per year, that will always be true
2016:08:30 20:01:10 gfredericks sounds doable
2016:08:30 20:01:13 alexmiller thx
2016:08:30 20:08:14 sattvik Regarding (4), you are right. I set up a method to handle (is (checks? foo/bar {…}))` where the last two arguments are just used to invoke stest/check. I first started by putting my fixture in a separately defined test function that invoked the the function to test. It works, but adds a bit of noise (also there is not a way to automatically inherit the function spec of the original). In a case where the fixture is transparent, i.e. takes the same args/returns the same result, I was able to write a macro that too the var name’s symbol and a higher-order function that served as the test fixture. That macro dereferenced the var giving the old value a local binding, and with-redefed the var to invoke the fixture (which took the old binding as an argument and returned a new function that behaved like the old one plus the fixture logic).
2016:08:30 20:08:42 sattvik I had to do with-redef because I couldn’t :replace the function under test.
2016:08:30 20:14:17 sattvik That technique works when the fixture is transparent, but perhaps I want to create a fixture that has a different signature than the function I am testing. For example, I might be testing that a function will eventually invoke a stubbed function that will return generated data. I want to check that the return value of the function under test satisfies a predicate that is partly dependent on the data that comes from the stubbed function, not just the inputs.
2016:08:30 20:15:17 sattvik This is all doable right now by creating such a function and properly specifying the test function and instrumenting the third-party function.
2016:08:30 20:15:52 sattvik This is all doable right now by creating such a function and properly specifying the test function and instrumenting the stubbed function.
2016:08:30 20:16:48 sattvik I suppose it might make more sense with an example…
2016:08:30 22:08:14 ag how can I create generate based on predefined vector? let’s say I have a vector [{:name “Anna”}{:name “David}] ..etc, I need a generator that creates vector with the same amount of elements, with added fields let’s say :age and :height?
2016:08:30 22:15:33 ag I guess I’ll just wrap things into gen/return and for
2016:08:30 22:30:45 gfredericks @ag so you want a generator that completes the maps?
2016:08:30 22:31:34 gfredericks You could use gen/vector with a fixed length and gen/hash-map
2016:08:30 22:40:39 ag I need to create a bunch of accounts with some predefined fields and some randomly generated fields, and need to create bunch of other structures associated
2016:08:30 22:44:27 gfredericks (gen/fmap #(map merge % predefined) (gen/vector (gen/hash-map ...) (count predefined))) @ag would something like ← that work?
2016:08:30 22:50:30 ag lemme try… thanks!
2016:08:31 00:12:32 ag errr… so I have spec like this:
for uuid vals. A bit bulky, I couldn’t find anything better. my problem now, if another spec includes it it always generates same uuids. How do I fix that?
2016:08:31 12:43:14 sveri @martinklepsch thanks, so basically its: 1. validate input with spec 2. try to parse result and display some useful text
2016:08:31 12:44:31 martinklepsch right. Do all of that on a form level and do some basic dirty? tracking
2016:08:31 12:45:00 sveri Ok, thank you very much, I will try some stuff 🙂
2016:08:31 12:58:55 alexmiller Rich checked in a round of perf improvements to master yesterday which fixed some hot spots
2016:08:31 14:22:52 mschmele When I try to run check on a function that I’ve spec’d with fdef, it fails with the cause “Unable to construct gen at: [] for : clojure.spec$…"
2016:08:31 14:22:57 mschmele am I missing something obvious?
2016:08:31 14:26:39 mschmele Code is up there for anybody who wants to read through it
2016:08:31 14:29:42 minimal @mschmele you should (generally) use the predicates on their own instead of wrapping in an anonymous function. Otherwise you won.t get generators for the functions that have them.
2016:08:31 14:30:14 minimal e.g. instead of (s/def ::id #(string? %)) use (s/def ::id string?)
2016:08:31 14:31:31 mschmele Okay, I just saw an example that was formatted that way and it seemed to work. Changing that now
2016:08:31 14:31:48 minimal also you should just use the specs inside fdef, no need for assert or valid which will probably give strange results
2016:08:31 14:32:34 mschmele so something like
2016:08:31 14:32:46 minimal and you need to use s/cat for arg lists
2016:08:31 14:32:49 mschmele
:args [::accounts ::id]
2016:08:31 14:33:03 minimal no
2016:08:31 14:33:53 minimal more like :args (s/cat :acc ::accounts :id ::id)
2016:08:31 22:30:37 bfabry @sattvik doing it that way makes the spec validate that the keyword is short, which you may not want, and also makes for slow generators
2016:08:31 22:40:04 sattvik Well, if the idea is to accept any keyword but only generate short ones during testing, then supplying the generator at check time is probably what you want. test.check has things like c.t.c.generators/resize that can fix the generator to a manageable size, e.g. 10.
2016:08:31 22:40:32 bfabry this is true
2016:08:31 22:40:35 sattvik Unfortunately, that’s not one of the functions clojure.spec mirrors.
2016:09:01 00:26:11 sattvik Curious. When working with higher-order functions, instrumentation will result in the execution of a function argument repeatedly. Which, isn’t great when the function has side effects.
2016:09:01 00:29:12 gfredericks you probably need to stub things?
2016:09:01 00:32:42 sattvik Eh… it’s kind of a weird situation. I have a function that performs some assertions for testing. It is invoked by the function under test. Under normal circumstances, my function with assertions is only invoked once per test. However, as the function under test has its parameters specified, the conformance checker ends up running my test function scores of times.
2016:09:01 00:34:49 sattvik I think one workaround is to instrument the function under test with less demanding specification (something like fn?). In the end, I just use a mutable cell to hold the data I want to check and have my test code update that cell. I can count on the last invocation of the function to be the one I care about.
2016:09:01 00:35:15 gfredericks you also get to override specs
2016:09:01 00:35:18 gfredericks so maybe you could test it in two ways
2016:09:01 00:35:40 gfredericks once where you have the function doing the assertion and you make sure it gets called only once, and another test without that where the spec gets exercised
2016:09:01 00:36:54 sattvik Well, doesn’t overriding specs only be done when using check? I am not doing that. I am only instrumenting for input argument validation.
2016:09:01 00:39:07 gfredericks that might be true
2016:09:01 01:40:14 seancorfield Is there any sort of standard pattern for spec’ing a function that curries its arguments?
(defn my-fn
([a] (fn [b] (my-fn a b)))
([a b] … a … b …))
2016:09:01 01:41:01 seancorfield Specifically, how to handle the fact that :ret depends on whether you provided one or two arguments...
2016:09:01 01:45:48 sattvik Hmm… good question. My guess is that you would use alt for the different return types and then an fn to tie them together to the passed args. I’m not sure if there is a better way, though.
2016:09:01 02:17:00 seancorfield @gfredericks all of a sudden, I’m getting this exception from test.check — any pointers? clojure.lang.Compiler$CompilerException: java.lang.ClassCastException: clojure.lang.AFunction$1 cannot be cast to clojure.lang.MultiFn, compiling:(clojure/test/check/clojure_test.cljc:95:1)
2016:09:01 02:19:43 seancorfield @sattvik yeah… that feels kinda ugly tho’… I’m hoping there’s a cleaner way...
2016:09:01 02:23:21 sattvik Hmm… I wonder if it is possible to do (s/or :unary (s/fspec …) :binary (s/fspec …))
2016:09:01 02:28:38 gfredericks @seancorfield that smells like this one thing hold on
2016:09:01 02:30:32 seancorfield Perfect! Thanks… I’ll add that Leiningen setting!
2016:09:01 02:31:49 seancorfield (I’m expanding the clojure.spec coverage for java.jdbc and ran into that for the first time)
2016:09:01 02:32:22 gfredericks this is all my fault for not having a new test.check release ready yet
2016:09:01 04:49:29 seancorfield Don't sweat it. test.check is already great!
2016:09:01 04:52:49 seancorfield Making progress with spec'ing java.jdbc. Some good, some frustrating simple_smile
2016:09:01 07:26:09 sveri Hi, I have trouble specing & args. What I want is (f [a1 a2 & ax]) where ax can be zero or more (s/*) and it must have an even number of elements.
This is what I thought should work: (s/* (s/and any? #(even? (count %))))
But it does not, instead it works for (f 1 2 3) but not for (f 1 2 3 4)
2016:09:01 08:00:28 sveri Is it because spec takes the args vector apart and checks every element of the vector?
2016:09:01 12:43:25 alexmiller The s/and there is in the spot for each element of the arg vector so that doesn't make sense
2016:09:01 12:48:05 alexmiller Or actually since you require at least 2 args that should be + not *
2016:09:01 12:49:10 alexmiller That's probably preferred, esp if you can give better names to x1 and x2
2016:09:01 12:49:14 sveri Zero / 2 / 4 / ...args are fine
2016:09:01 12:50:16 sveri I just wonder, if I look at: (s/ (s/cat :x1 any? :x2 any?)) I would assume it should have two arguments, not more, not less. But, I think its the s/ around it, that makes it work for more inputs, true?
2016:09:01 12:53:19 alexmiller Yes - it's just like regex
2016:09:01 12:53:31 alexmiller It's a repetition of pairs
2016:09:01 12:54:45 alexmiller @sattvik: you can override specs in instrument to use a simpler spec
2016:09:01 12:55:47 alexmiller @seancorfield re currying, you can use a :fn spec if you need to make an assertion about the relationship between :args and :ret
2016:09:01 13:01:06 sveri @alexmiller Thank you very much 🙂
2016:09:01 13:30:07 mschmele Is it generally considered bad practice to allow functions to have nilable return values?
2016:09:01 13:30:42 mschmele It makes having functions that pass spec tests quite a bit easier, but I feel like it kinda defeats the purpose to some extent
2016:09:01 13:49:22 sattvik @mschmele It’s perfectly reasonable, if nil means something like ‘not found’ or ‘empty’.
2016:09:01 13:50:49 sattvik @alexmiller That’s true. It’s just that I was a little surprised (but perhaps I shouldn’t have been) that conforming a function argument involved repeated invocations of the function.
2016:09:01 13:51:38 mschmele That’s what I’ve been using it for in simpler functions like get-from-id for example. I guess my real question (and I probably should have been more specific), applies to more complex functions like transfer which would take two accounts that can’t be nil
2016:09:01 13:52:11 mschmele Despite having validation in the function, I’m having trouble getting it to pass spec check
2016:09:01 16:09:08 seancorfield @alexmiller Yeah, I know I can deal with currying via :fn but that still makes for a fairly complex spec — Has there been any discussion of making it easier to spec multi-arity functions?
2016:09:01 16:19:46 alexmiller Not in regards to this
2016:09:01 16:20:50 alexmiller My experience has been that it is comparatively rare for the ret spec to rely on the args arity
2016:09:01 16:21:28 alexmiller And when it does :fn is the way to talk about that constraint
2016:09:01 16:25:42 seancorfield Well, with a curried function, the :ret will either be a function (of the remaining argument(s)) or a result...
2016:09:01 16:26:53 seancorfield So
(defn foo
([a] (fn [b] (foo a b)))
([a b] (* a (inc b))))
=> :ret would be int? in the two arg case but some fspec in the one arg case
2016:09:01 16:27:24 seancorfield If fspec / fdef supported multi-arity, this could be neater...
2016:09:01 16:30:45 alexmiller Well I'd say curried functions are not idiomatic in Clojure :)
2016:09:01 16:30:59 seancorfield 😝
2016:09:01 16:31:34 alexmiller But the main place I've run into this is with the core seq functions with transducer arity
2016:09:01 16:31:38 seancorfield What about multi-arity functions in general?
2016:09:01 16:32:22 alexmiller Usually most arities call into a canonical arity and have the same ret spec
2016:09:01 16:32:23 seancorfield They’re still pretty messy to spec out, even if :ret is fixed.
2016:09:01 16:32:44 alexmiller I have not found multiple arities at all difficult to spec
2016:09:01 16:32:59 alexmiller They're often quite easy to talk about in regex
2016:09:01 16:34:31 seancorfield As long as the arities all extend the base case in order, yes… but there’s quite a bit of real world code out there which doesn’t follow that model (or is that also non-idiomatic?).
2016:09:01 16:36:02 alexmiller Even if not in same order, regex can easily describe with ?
2016:09:01 16:37:02 alexmiller I'm not discounting what you're saying, but I have not found that to be an issue in my own experience
2016:09:01 16:38:11 alexmiller In general, I am more commonly surprised at how well regex specs work for args
2016:09:01 16:41:03 robert-stuttaford you guys are having a ball, aren't you. can't wait to get stuck in myself!
2016:09:01 16:41:18 mschmele @sattvik copied from above, because I forgot to @ you earlier 😝
That’s what I’ve been using it for in simpler functions like get-from-id for example. I guess my real question (and I probably should have been more specific), applies to more complex functions like transfer which would take two accounts that can’t be nil
Despite having validation in the function, I’m having trouble getting it to pass spec check
2016:09:01 16:43:47 seancorfield @alexmiller Fair enough. I held off spec’ing some of the multi-arity java.jdbc stuff at first since it didn’t seem "easy" (although it may yet prove to be "simple"). I’ll take another run at it soon (next week probably) and report back.
2016:09:01 16:46:48 seancorfield @alexmiller My last Q for the morning (I promise!): is there an idiomatic way to spec something that is treated as truthy / falsey, when it might not be strictly true or false. I tried #{true false nil} before realizing won’t work facepalm and (s/nilable #{true false}) "works" but seems a bit … I guess that treating an any? argument as a pseudo-boolean is probably a bit sketchy but ...
2016:09:01 17:03:01 alexmiller I don’t think (s/nilable #{true false}) works either for false (for same reason as the set)
2016:09:01 17:04:57 alexmiller so you need some kind of fn to do it so pick your favorite function that matches those 3 values :)
2016:09:01 17:05:52 alexmiller I guess (s/nilable boolean?) works
2016:09:01 17:06:03 alexmiller I’ll go with that :)
2016:09:01 17:09:51 alexmiller no set with falsey values in it is going to be useful :)
2016:09:01 17:10:42 alexmiller s/nilable has been made significantly better performing this week from Rich’s commits in master btw
2016:09:01 17:12:00 seancorfield Ah, yes, of course false won’t work either. facepalm again 🙂
2016:09:01 17:17:14 seancorfield (it hadn’t failed in testing yet but…)
2016:09:01 17:19:15 seancorfield The specific case is for ::as-arrays? in java.jdbc where it’s intended to be true / false or :cols-as-is but in reality nil is acceptable and common when passing defaulted options around (and, of course, it really accepts any? and just treats it as a boolean).
2016:09:01 17:20:50 sattvik For the multi-arity stuff, the following doesn’t work, but it might be nice if it did:
2016:09:01 18:37:56 alexmiller I think Rich would say about :ret here that it should simply state the truth - it can either be a number or a function
2016:09:01 18:38:43 alexmiller and :fn can add an arg-dependent constraint
2016:09:01 18:39:24 alexmiller I think it’s unlikely we would extend to either of the two suggestions above
2016:09:01 18:40:03 alexmiller this kind of dependent constraint is the whole reason to have :fn
2016:09:01 18:41:48 alexmiller if you felt the need, I think you could create a macro that automatically created a spec for a curried fn
2016:09:01 18:52:52 seancorfield Hmm, yeah, that might well be worth doing… I’ll have a think about that...
2016:09:01 18:54:31 seancorfield A variant of fdef for which you give :args, :ret, and :fn -- and then also an indication of which curried variants you need… and then it automatically generates the fdef spec from that...
2016:09:01 18:54:39 alexmiller I guess you’d need to specify the shape of every curried result
2016:09:01 18:55:07 seancorfield That would be the hard part since you can’t use s/? for all those args, only for the last one.
2016:09:01 18:56:20 alexmiller you can stack em up
2016:09:01 18:57:28 seancorfield (s/? (s/cat …))
2016:09:01 18:58:05 alexmiller yeah, it’s gross looking - you’d want to generate it out of an s/cat or an s/cat with noted optional parts or something
2016:09:01 18:58:36 alexmiller I don’t think curried fns are common enough to do any of this :) but if you’re looking for a puzzle to play with …
2016:09:01 18:59:44 seancorfield I tend to curry functions quite a bit — to avoid partial all over the place — but it is almost always currying just a two arg function.
2016:09:01 19:02:40 alexmiller well maybe that’s a simplifier
2016:09:01 19:02:57 alexmiller the shape I had above could be made generic
2016:09:01 19:06:28 seancorfield I may just remove the currying in java.jdbc since I’d be shocked if anyone actually leverages it in client code (given your comment about it being non-idiomatic). java.jdbc doesn’t use the curried form of as-sql-name internally and I don’t know why anyone would externally. And I wouldn’t expect the non-curried form of quoted to be used by anyone either (the one-argument form is actually the common, useful arity).
2016:09:01 19:06:46 seancorfield I hadn’t thought hard about either of those until I sat down to try to spec them out.
2016:09:01 19:10:41 seancorfield clojure.spec definitely makes you question your design choices 🙂
2016:09:01 19:32:25 alexmiller indeed
2016:09:01 19:33:42 patrkris Any ideas on how to organize a project's specs? For instance, if I have a domain model for my application that describes a customer entity, would I then create the spec in a com.example.customer namespace and use ::name for defining a spec for the customer's name? Or would I benefit from centralizing specs in a dedicated namespace, and in that spell out the fully namespaced keyword, i.e. com.example.customer/name?
2016:09:01 19:35:30 alexmiller I’d say both are fine :)
2016:09:01 19:38:37 patrkris Yeah. That's what I thought you'd say. 😉 But then I am imagining scenarios where a customer means different things in different contexts. It may be one thing in the domain model and another thing in a HTTP request handler. So it might be okay to have :com.example.customer/name and :com.example.resources.customer/name? Does that look wrong?
2016:09:01 19:46:28 alexmiller you can alias specs (s/def :com.example.customer/name :com.example.resources.customer/name)
2016:09:01 19:46:38 alexmiller so you can have both
2016:09:01 19:47:02 alexmiller has to be done explicitly of course so ymmv
2016:09:01 19:47:44 alexmiller we’ve talked about a version of s/keys that would separate the map keys from the specs rather than requiring them to be the same keyword, not sure if that will pan out
2016:09:01 20:31:00 manderson +1 to separating map keys from specs ^^^
2016:09:01 20:55:54 alexmiller the spec part would still be a qualified keyword (not inline specs), just to be clear
2016:09:01 20:56:39 alexmiller this was considered as an alternative to :req-un too
2016:09:01 21:05:53 seancorfield It took me a while to internalize that :req-un could have :my.foo/bar as a spec for the key :bar independent of the namespace you define the keys spec in.
2016:09:01 21:07:00 seancorfield (Because I didn't read the docs closely enough apparently)
2016:09:01 21:08:11 seancorfield So I'm not sure why you'd want to separate the specs from the keys at this point?
2016:09:01 21:08:29 seancorfield (Or am I still misunderstanding the issue?)
2016:09:02 01:57:45 seancorfield It rebinds the report function
2016:09:02 01:58:16 gfredericks do you get to choose an alternate version of the clojure-maven-plugin if you want?
2016:09:02 01:59:13 seancorfield I could override it for my project I think, yes (this is for java.jdbc)
2016:09:02 01:59:31 seancorfield Is there a version that is compatible with test.check’s clojure_test stuff?
2016:09:02 01:59:34 gfredericks alternately, any tactic you can think of that lets you require test.check.clojure-test prior to that line running will fix it
2016:09:02 01:59:41 gfredericks no I was just imagining forking it :)
2016:09:02 01:59:48 seancorfield Hahaha… ok...
2016:09:02 01:59:58 gfredericks so a user.clj could work if you can keep it out of the release jar
2016:09:02 02:00:17 seancorfield Would that run with Maven?
2016:09:02 02:00:25 gfredericks it runs when clojure boots up
2016:09:02 02:00:41 gfredericks user.clj is a pretty reliably way to slip something in before just about anything else happens
2016:09:02 02:01:33 seancorfield Pretty sure that doesn’t work with Boot? (more an FYI but…)
2016:09:02 02:01:44 gfredericks why not?
2016:09:02 02:03:52 seancorfield I don’t remember… but it was discussed in #boot a while back...
2016:09:02 02:04:18 gfredericks probably something about their magical space age classloader thing
2016:09:02 02:05:15 seancorfield And how do you get it to be loaded for Maven running Clojure?
2016:09:02 02:05:30 gfredericks @seancorfield heck for that matter adding a (:require clojure.test.check.clojure-test) to any of your namespaces should also work
2016:09:02 02:06:03 gfredericks putting the user.clj on the classpath means that clojure.core reads it at then end of its loading
2016:09:02 02:06:23 gfredericks e.g. src/test/resources/user.clj or whatever
2016:09:02 02:09:50 seancorfield Ah, yeah, that worked...
2016:09:02 02:15:27 talios @seancorfield you can define/give your own test runner script to clojure-maven-plugin if you need to
2016:09:02 02:16:12 seancorfield Dynamically requireing that namespace when my test namespace loads seems to do the trick — and I can remove the Leiningen monkey-patch setting as well.
Depending on the type, the value needs to be different, e.g., for type-date, the value should be a date. I know there are multimethods that can, and probably should, be used, but I just can't get it right. What comes at the <???>, so I can have different predicates for value depending on the value of type?
2016:09:02 15:12:27 alexmiller have you looked at s/multi-spec?
2016:09:02 15:19:30 spinningtopsofdoom I ran into this problem two days ago you want to make N :value specs. (e..g.`(s/def :my/value <???>)`, (s/def :other/value <???>), (s/def :one.more/value <???>) and then dispatch those :value's with a multimethod
2016:09:02 15:20:14 kurt-yagram yeah, I'm looking at it. but it seems to be somewhat different. So, I could do:
So I day value is required for "type-date". But that still doesn't solve the problem. I must be missing something...
2016:09:02 15:21:49 kurt-yagram @spinningtopsofdoom Allright, and I don't need to care about the namespaces? - I mean, the map contains just 'value', no namespaces. I'll give it a few tries.
2016:09:02 15:21:54 alexmiller instead of using ::value, you could define many :foo1/value :foo2/value :foo3/value specs
2016:09:02 15:22:01 kurt-yagram 🙂
2016:09:02 15:22:07 alexmiller each defmethod would use a different one in :req-un
2016:09:02 15:22:21 kurt-yagram oh, allright
2016:09:02 15:25:43 spinningtopsofdoom @alexmiller is there any movement on having spec/keys take a map of keywords and specs (e.g.)
So that you don't have to have registered specs for spec/keys
2016:09:02 15:25:55 alexmiller nothing yet
2016:09:02 15:26:07 alexmiller well, it will never take inline specs
2016:09:02 15:27:16 alexmiller we might possibly loosen the constraint between key name and spec name, but it’s part of the design that s/keys doesn’t use inline specs and I don’t expect that to change
2016:09:02 15:27:39 alexmiller the idea is to encourage defining semantics for attributes
2016:09:02 15:29:28 spinningtopsofdoom The constraint between key name and spec name is what I would like to loosen. So then my example would be
2016:09:02 15:30:11 alexmiller yeah, something like that has been mentioned, but I do not know whether we’ll end up doing it or not
2016:09:02 15:32:18 spinningtopsofdoom Well I'll make a Jira ticket for that if it's not already there, then. Thanks for the clarification.
2016:09:02 15:35:42 alexmiller I do not know of a jira ticket for this
2016:09:02 15:40:30 gerstree Hi everybody, very new to specs, but have already a good part implemented. Now I need a little help on something that is probably very easy, but I'm stuck. I have a def that verifies a url to be a s3 url (s/def ::s3-url #(str/starts-with? % "s3"). Using it to check a single argument works. Now I have a function with 2 arguments (from-url and to-url) of which 1 needs to conform to that spec. Can anyone hint me for the solution.
2016:09:02 15:44:37 gerstree This is what I have:
(s/def ::valid-url is-valid-url?)
(s/def ::s3-url #(str/starts-with? % "s3"))
(defn sync
"sync an s3 folder with a local folder, this works both ways"
[from-url to-url])
(s/fdef sync
:args (s/and (s/cat :from-url is-valid-url? :to-url is-valid-url?)
??? this is where I get lost ???))
2016:09:02 15:49:28 alexmiller something like that
2016:09:02 15:50:25 gerstree The function will be called either (sync "" "") or (sync "" "")
2016:09:02 15:51:53 gerstree What I am trying to spec is that either url is an s3 url
2016:09:02 15:51:55 alexmiller right so you could have something like:
(s/def ::file-url #(str/starts-with? % “file://“))
(s/def ::s3-url #(str/starts-with? % “s3://“))
(s/def ::aws-url (s/or :file ::file-url :s3 ::s3-url))
… then use ::aws-url in the sync fdef
2016:09:02 15:52:08 alexmiller or whatever is appropriate
2016:09:02 15:52:33 alexmiller oh, you want a constraint across the args!
2016:09:02 15:52:48 gerstree Yes, is that beyond what spec is meant for?
2016:09:02 15:53:27 alexmiller No, that's fine!
2016:09:02 15:54:03 alexmiller You can do it with s/and like you were or you can do that in the :fn spec too
2016:09:02 15:54:45 alexmiller fn is used for constraints between args and ret or also across args
2016:09:02 15:55:13 alexmiller Only the args spec is checked in instrumentation though so that might be what you want
2016:09:02 15:55:59 gerstree Ah, been reading about it all day and understood the args / ret, but of course that works for across args as well.
2016:09:02 15:56:30 alexmiller Yeah fn gets the conformed version of both args and ret
2016:09:02 15:56:42 gerstree That might be the easiest way to accomplish it.
2016:09:02 16:16:17 gerstree Also read about 10 times that stest/instrument only does :args, now I know that is true 😉
2016:09:02 16:30:01 gerstree I have the :fn version working. I'm still curious about a solution in the :args part, where I got stuck to begin with. That way, if people use my sync function they can simply (stest/instrument `sync) and work from there.
2016:09:02 18:07:59 alexmiller or it’s a great opportunity to use juxt :)
2016:09:02 18:08:09 alexmiller (defmulti m (juxt ::type ::op))
2016:09:02 18:09:53 kurt-yagram oh, cool... thx!
2016:09:02 18:10:39 alexmiller and then the retag value is not valid either - should either be a tag or a fn of generated value and dispatch-tag
2016:09:02 22:34:37 lvh huh; are instrumentation exceptions supposed to escape clojure.test assertions?
2016:09:02 22:35:06 lvh oh, wait, never mind; only some of them are outside assertions 🙂
2016:09:03 00:44:16 ag can anyone tell me ETA of 1.9 stable? I already have a piece of code that depends on clojure.spec, unfortunately my team did not approve my PR (bunch of alpha-phobics), that makes it a bit inconvenient for me personally.
2016:09:03 00:45:19 alexmiller No eta
2016:09:03 00:47:22 ag not even approximation? before Christmas, 2nd quarter next year etc.?
2016:09:03 00:47:52 alexmiller I'm sure we will try to hit some milestone before the conj but may just be beta (feature freeze)
2016:09:03 00:48:25 alexmiller We have a couple other things in flight that we haven't even talked about yet
2016:09:03 00:49:20 alexmiller We are not currently working towards any concrete deadline though
2016:09:03 00:50:46 ag yeah, ok… I am so excited though. Spec turns out to be a massive feature of the platform. I really like it.
2016:09:03 19:10:05 ag @wagjo yeah, I switched to backport. Worked as a charm.
2016:09:03 22:34:08 lvh Are you generally expected to add enough info to s/fdef such that running check on that fn works? Is that considered the “default” state? I’m writing reasonably gnarly generators to make that work, because I have two args with related keys (and if you picked args at random, you’d almost ceratinly end up with garbage)
2016:09:03 22:47:00 lvh Hm. I wonder if it’s OK for an fdef’s :args to have a generator that uses the fn being fdef’d (a lot of functions can clearly be run with a “base case” if you will; e.g. if you have something that conjs a bunch of stuff together, you might conj a bit ahead of time and assert that it doesn’t throw away previously conj’d things)
2016:09:03 22:47:25 lvh I think so, but with the apparently suggested pattern of fdefing before the defn itself, you run into a compile error; so I’m wondering if I should just declare it and be done with it or what 🙂
2016:09:03 23:40:32 lvh FYI, got a weird error when running clojure.spec.test/check + deftest through CIDER (and only CIDER): https://github.com/clojure-emacs/cider/issues/1841
Might be interesting for upstream to look into since just because CIDER ran into it (or rather; I only ran into it with CIDER) doesn’t mean it’s necessarily CIDER-specific 🙂
2016:09:04 01:52:50 hiredman it likely is, cider is likely redefing the report function in clojure test with function that is not a multimethod
2016:09:04 02:54:13 alexmiller Yes this is the lein monkey patching
2016:09:04 02:54:46 alexmiller There are issues in both lein and test.check about it and it's fixed in test.check master
2016:09:04 02:55:01 alexmiller You can disable lein monkey patch to avoid
2016:09:04 02:56:15 alexmiller Well I guess it could be a cider variant of the same problem, not sure
2016:09:04 03:14:21 gfredericks somebody filed a bug report a few months back, I think to test.chuck, wherein I convinced them that cider was monkeypatching the wrong way
2016:09:04 03:16:07 gfredericks no nevermind
2016:09:04 03:16:25 gfredericks it was probably fireplace too
2016:09:04 17:53:37 flyboarder Hello, just a quick question with the new spec system. Where do I place my specs? In the same namespace as the things they specify are declared? And how do I go about making them compatible with older versions of clojure? I am imagining my lib being used in 1.7 and 1.8 as well.
2016:09:04 18:03:08 flyboarder Ah I see there are back ports for spec
2016:09:04 19:21:51 gfredericks Other libs have put the specs in a separate ns so users on older versions can ignore them
2016:09:04 20:42:59 lvh meh; stest/check now basically looks like it hangs forever and I don’t know which thing it’s choking on
2016:09:04 20:43:11 lvh it’d be nice if there was a debug mode or something where it told me what it was working on
2016:09:04 20:47:39 lvh It’s definitely helpful in finding bugs though 🙂
2016:09:04 20:48:05 lvh also in making my laptop not very suitable for holding on a lap 😄
2016:09:05 09:57:58 ikitommi @alexmiller did you consider using Records instead of just doing reify for the Spec protocol? For example and-spec-impl could return clojure.spec.AndSpec record? Would it have negative effect on performance? Records might be easier to understand and adding extensions would be easier. Trying to cover the JSON Schema generation for the core Specs. Extending records with a new protocol would be easy (the way we are doing the same with Schema).
2016:09:05 10:02:26 ikitommi Pushed out a experimental version of the dynamic conformations (based on runtime data). Would like to get feedback on that, now using dynamic var as the conform doesn’t take any optional parameters to do this. https://github.com/metosin/spec-tools
2016:09:05 11:59:30 misha greetings, is there an equivalent to clojure.spec/def, for defining many specs? like:
2016:09:05 12:21:31 misha @madstap 🙂
is there any popular use case, of using return value of s/def?
2016:09:06 17:03:58 noprompt are there any explanations of how to properly implement clojure.spec.Spec?
2016:09:06 17:13:33 noprompt i ask because i'm interested in integrating a small parser combinator library i wrote with clojure.spec so i can hook into to the conform, explain, etc. goodness without having to do it manually post-hoc.
2016:09:06 17:18:13 noprompt it seems like a useful thing since, on occasion, one might be interested in validating/conforming nested (or unnested) textual data.
2016:09:06 17:44:19 alexmiller @noprompt for the moment, we’re considering all that to be implementation details
2016:09:06 17:44:33 alexmiller and subject to change without notice
2016:09:06 17:45:21 alexmiller in general, spec is designed to primarily be extensible via predicates (at the bottom) and wrapping with macros (at the top - particularly s/conformer and s/& are tools for this)
2016:09:06 17:50:02 noprompt @alexmiller i see. well, i'll continue to hold out then for things to stabilize. i've at least familiarized myself a bit more with the internals (implementing Spec myself) so that can't be a bad thing. is there something similar to conformer for explain? that would be nice in my case because the parser combinators do produce failure data of where in the string the parse failed which could be appended to the path.
2016:09:06 17:51:55 noprompt tl;dr explainer?
2016:09:06 18:00:49 alexmiller no, not currently
2016:09:06 18:01:09 alexmiller although in next alpha, the explain-out fn is a dynvar you can swap
2016:09:06 18:01:32 alexmiller that’s a bigger hammer though
2016:09:06 18:03:10 noprompt yeah. that sounds like too big of a hammer. some way to implement an explain for a pred would be nice. i think i can get by with conformer for now. i hope in the future the protocols will stabilize for this kind of thing.
2016:09:07 00:25:44 lvh +1; by the way; I have nothing to contribute to that other than “I can explain a problem better than my consumers can read Clojure” 😄
2016:09:07 01:19:35 ag how do I create an enum spec? Value that conforms that is in a given set?
2016:09:07 01:20:12 sparkofreason It's just the set itself.
2016:09:07 01:20:59 ag yeah but oh… I got it… (gen/generate (s/gen (s/and #{:foo :boo})))
2016:09:07 01:21:35 ag I was using s/exercise its output confused me
2016:09:07 01:22:09 ag thanks
2016:09:07 02:20:24 bbrinck @ag I think (gen/generate (s/gen #{:foo :boo})) is equivalent
2016:09:07 15:18:53 jannis Out of curiosity: Is there a way to say "in this s/cat spec, please omit this item and this item from the map resulting from s/conform"? One example (s/def ::myspec (s/cat :first-operand number? :_ #{:plus :minus} :second-operand number?)) -> (s/conform ::myspec [15 :plus 10]]) -> {:first-operand 15 :second-operand 10} (with the :_ key indicating "please don't include this in the result")?
2016:09:07 15:20:12 jannis I'm asking because I'm parsing a small language with clojure.spec and there are some parts in the language that I don't care about later on. And the simpler the output of s/conform the better - in this case.
2016:09:07 15:26:00 alexmiller there is no way to automatically omit syntactic elements like that, but you can use a conformer to get the same effect
2016:09:07 15:28:04 alexmiller you can put the conformer inside the spec too of course, just wrapping here as an example
2016:09:07 15:29:02 jannis Nice 🙂
2016:09:07 15:29:10 jannis That's pretty awesome.
2016:09:07 15:29:55 jannis I assume I can't have multiple :_s in an s/cat?
2016:09:07 15:31:04 alexmiller it probably wouldn’t throw an error, but I haven’t tried it
2016:09:07 15:31:40 alexmiller they’d probably all just override the prior as it parsed - you wouldn’t be able to unform that but doesn’t seem like you care about that anyways
2016:09:07 15:32:09 alexmiller (`conformer` can also take an unform function to support both directions)
2016:09:07 15:33:37 jannis Yep, later :_s override earlier occurences in the resulting map.
2016:09:07 15:34:18 jannis Cool. I'm not sure I'm going to use this but it could make my life easier.
2016:09:07 16:02:29 joshg Is there an idiomatic way to define :args and :ret when defining a function without a separate fdef declaration? It would be easy to write a macro to do this, but is it considered bad practice?
2016:09:07 16:03:21 joshg (similar to schema’s s/defn)
2016:09:07 16:04:59 alexmiller spec does not provide this (and does not intend to)
2016:09:07 16:05:06 alexmiller but you’re welcome to do so :)
2016:09:07 16:06:33 joshg thanks
2016:09:07 16:09:10 joshg @alexmiller I’m curious why spec :args and :ret are not defined in defn like pre and postconditions?
2016:09:07 16:14:41 alexmiller Several reasons
2016:09:07 16:15:25 alexmiller Rich talks about the notion of an independent registry at http://clojure.org/about/spec
2016:09:07 16:16:13 alexmiller That is, we don't need to hang everything off vars (and there are downsides to doing so, that affect bytecode size and startup time)
2016:09:07 16:16:48 alexmiller Also, from the purposes of API evolution, it is useful for the specs to be independent from the vars
2016:09:07 16:17:45 joshg Got it. Thanks for the explanation.
2016:09:07 16:19:41 joshg the “API evolution” sound interesting. I assume that’s yet to be announced?
2016:09:07 16:27:04 alexmiller Rich talked about some of that on the Cognicast episode he did
2016:09:07 16:28:06 joshg when he was talking about Postel's Law and API versioning?
2016:09:07 16:28:40 alexmiller But the idea is that if the specs are independent from the functions you can talk (programmatically) about whether one api subsumed another
2016:09:07 21:44:14 alexmiller in particular, Rich did a big perf pass and improved times of most things
2016:09:07 22:20:03 noprompt @alexmiller have you talked to or has rich mentioned anything by the way of the explainer concept we talked about yesterday? it'd be really nice to have something to pair with conformer to produce explanation data for custom conformers!
2016:09:07 23:08:42 alexmiller No
2016:09:08 02:36:53 bbloom i just want to say: i’ve been writing javascript with flowtype for the past ~2 weeks. It’s a very nice type system and it’s very well implemented…. and I absolutely hate it. I miss clojure 😞
2016:09:08 02:51:43 seancorfield @bbloom No, no, you must be mistaken. Type systems are awesome 🙂
2016:09:08 02:52:32 seancorfield I’m finding new things to use clojure.spec for almost every day, I have to say…
2016:09:08 02:54:46 bbloom i think i’ve hit more false positives about NPEs from flowtype in two weeks than i’ve had ACTUAL NPEs in a year of Go
2016:09:08 02:59:25 seancorfield So it’s telling you "This will likely NPE" but the code wouldn’t?
2016:09:08 03:03:47 bbloom basically it says something like “potentially null or undefined” b/c it’s doing a map lookup or something, but i know with 100% confidence by construction that value is in the map
2016:09:08 03:04:06 bbloom or similar
2016:09:08 03:05:00 bbloom you wind up having to employ strategies all over the place:
2016:09:08 03:05:09 bbloom 1) making tons of extra structure types for every possible state of things
2016:09:08 03:05:52 bbloom and 2) using invariant(whatever, ‘whatever may not be null’) which basically is just a glorified manual throw npe
2016:09:08 03:36:00 seancorfield Must admit, when we were using core.typed at work, that was one of the problems we had: nil was contagious and then we kept getting complaints about code not accepting nil (when we knew nil wouldn't flow to that point),
2016:09:08 03:37:45 shaun-mahood @seancorfield: Have you run into anything like that with spec yet?
2016:09:08 05:38:40 seancorfield @shaun-mahood no, but that’s because spec isn’t "contagious" in the same way that type systems tend to be: spec’ing one piece of code doesn’t ripple out into other pieces of code.
2016:09:08 05:42:25 shaun-mahood @seancorfield: That's awesome - I ran into similar issues with schema in a very limited manner and it really bothered me. I know you'll put spec through a lot more than I ever will so I've been using your notes and comments about as sort of my real-world expectations.
2016:09:08 05:45:28 seancorfield I'm surprised you hit that with Schema since it's really just runtime assertions.
2016:09:08 06:22:08 shaun-mahood Well, it's more likely that I did something stupid or am remembering different problems then - I really didn't put a whole ton of effort into figuring out exactly what was going on and it was quite a while ago. I probably just didn't know what I was doing as well as I though I did :)
2016:09:08 13:27:51 jannis Is it a deliberate difference between Clojure and ClojureScript that ClojureScript's clojure.spec.test/check only accepts a literal collection of syms instead of a code-generated value (e.g. (let [syms (filter in-my-namespaces? (st/checkable-syms))] (st/check syms)) will not work as it raises a Unable to resolve symbol: syms error.
2016:09:08 14:24:59 jeroenvandijk Question about multi-spec, is it meant/possible to use with something else than a map? I cannot find examples and after a lot of trying it seems the multi-spec implementation doesn’t call the multi-method like one would expect
2016:09:08 14:32:34 jeroenvandijk ah ok, i had a clear moment. I was a victim of multimethod + multi-spec reloading. It is possible with the right dispatch function
2016:09:08 16:47:44 jeroenvandijk @tmtwd yep, we use it. Not in something we deployed in production (yet). But I have used it quite a lot now and it works well
2016:09:08 16:48:11 Tim hm
2016:09:08 16:49:19 jeroenvandijk Is anybody doing things with clojure.spec/unform at the moment? I asked something on the mailinglist about using it for data migrations, but no answer yet
2016:09:08 17:04:27 alexmiller not many from my impression (given the number of bugs I am aware of but no one has mentioned yet :)
2016:09:08 17:04:48 alexmiller data migration is one interesting use case for it though
2016:09:08 17:07:16 jeroenvandijk thanks, for confirming the use case 🙂 I’ll see if I can make it work
2016:09:08 18:21:26 dmarjenburgh Hello. With multi-spec, is there a way to generate values conforming to only a specific dispatch-value? For example, in the :event/event example in http://clojure.org/guides/spec, the generator generates all events. What is the best way to generate events of only a certain type?
2016:09:08 18:42:16 bfabry @dmarjenburgh you can get the spec (and hence the generator) for a specific dispatch value by just invoking the multi-method, (event-type {:event/type :event/search})
2016:09:09 15:20:08 otfrom alexmiller: that's great. Means I'll use it to conform stuff in my sparkling spark jobs. :-D Thx!
2016:09:09 15:20:45 alexmiller generally, it’s faster than schema, so if schema was fast enough for you, it should be sufficient
2016:09:09 15:24:05 otfrom alexmiller: that was my alternative so I'm very happy. :-D
2016:09:09 15:32:55 seancorfield I'm going to have to study the commit(s) that are responsible for that massive speed up -- very curious about how that was achieved! 😸
2016:09:09 15:34:11 alexmiller lot of changes. one change of possible importance is that a spec using another registered spec will “compile” in that definition, so changes to the upstream spec (during dev) now require the downstream spec to be reloaded
2016:09:09 15:34:20 alexmiller so it’s a little less dynamic than prior
2016:09:09 15:34:37 alexmiller this does not affect recursive specs as delays are used
2016:09:09 15:52:30 jannis I'm hitting odd behavior with clojure.spec and I'm not sure it's me or a bug in clojure.spec. Here's the example: https://gist.github.com/Jannis/f23a2ecf350b401745d190b02bbb619c
What I would expect is no surrounding [] around [:nested ...] in the second case. It seems that ::link-value spec is clashing with the ::query spec. If I change ::link-value to e.g. (s/tuple symbol? keyword?) I don't get the extract []. The odd thing is that despite the "clash", [:link ...] never appears in the output... any ideas?
2016:09:09 15:59:05 alexmiller @otfrom :type isn’t a thing
2016:09:09 15:59:10 alexmiller it’s :kind
2016:09:09 16:00:39 alexmiller if only you had the spec specs, it would tell you that :)
2016:09:09 16:06:24 alexmiller @jannis so the one you’re asking about is
2016:09:09 16:07:53 alexmiller the outermost [ ] is from ::query
2016:09:09 16:08:10 alexmiller the [ ] around :single and :nested are due to the s/alt tagging
2016:09:09 16:08:26 otfrom @alexmiller thx. Mostly just flailing around in the library trying to learn it (and getting dumb things wrong) 😉
2016:09:09 16:12:53 jannis @alexmiller The one I'm asking about is [[:single [:simple a]] [[:nested ...]]] - with the [[:nested ..]] instead of [:nested ..].
2016:09:09 16:16:00 jannis What's puzzling me there is that the other two examples don't have the extra [] around [:nested ...] - and that I can make it disappear in the second case if I change the :link-value spec that should have no impact here.
2016:09:09 16:16:35 jannis (This is Clojure 1.9.0-alpha11 by the way)
2016:09:09 16:17:20 alexmiller oh, this reminds me of something
2016:09:09 16:18:39 alexmiller that example in the comment also has something similar
2016:09:09 16:19:12 alexmiller I’ve been looking at some regex conform results that seem off to me - although most of those relate to use of s/?
2016:09:09 16:20:12 jannis The output certainly looks similar 😉
2016:09:09 16:25:13 jannis In this case I think s/conform may not know which of [::query :single :link] and [::query :nested :children] to pick. It decides on :nested instead of :single but surrounds it with the extra [] for some reason.
2016:09:09 16:26:23 jannis Could that be what happens in case of ambiguity?
2016:09:09 16:30:37 alexmiller I don’t think that’s what’s happening - I think it has to do with the guts of the regex derivatives and when it decides to create the nested context
2016:09:09 16:36:37 jannis You know the internals, I don't. 😉 Is there anything I can do to help investigate the problem (if it is one at all)?
2016:09:09 17:06:06 alexmiller no, I’m looking at some of this stuff already
2016:09:09 17:06:23 alexmiller if you wanted to boil it down to a simpler repro and file a jira, that would be helpful
2016:09:09 17:06:42 alexmiller just to make sure that it does get addressed and not just “in my head” :)
2016:09:09 20:55:55 fenton how do I spec that a map should look like: {:db/id <something>}. I don't own the db namespace so am not sure how to spec for that.
2016:09:09 20:56:39 bfabry @fenton you don't need to own a namespace to refer to it
2016:09:11 00:02:37 ghufran I’m going through the spec tutorial on http://clojure.org . Is there any difference between using spec/or and using spec/alt ? It seems to work the same for the example from the tutorial:
`
(s/def ::config-alt (s/*
(s/cat :prop string?
:val (s/alt :s string?
:b boolean?))))
;;=> :testspec.core/config-alt
(s/conform ::config-alt [ "-server" "foo"
"-verbose" true
"-user" "joe"])
;;=> [{:prop "-server", :val [:s "foo"]} {:prop "-verbose", :val [:b true]} {:prop "-user", :val [:s "joe"]}]
(s/def ::config-or (s/*
(s/cat :prop string?
:val (s/or :s string?
:b boolean?))))
;;=> :testspec.core/config-or
(s/conform ::config-or [ "-server" "foo"
"-verbose" true
"-user" "joe"])
;;=> [{:prop "-server", :val [:s "foo"]} {:prop "-verbose", :val [:b true]} {:prop "-user", :val [:s "joe"]}]`
2016:09:11 00:14:16 alexmiller Yes (although there are situations where they can yield the same result)
2016:09:11 00:15:02 alexmiller in this particular example we are describing the structure of a sequential thing and regex ops are being used to specify that structure
2016:09:11 00:15:29 alexmiller s/alt is the proper choice here as it is the regex op version
2016:09:11 00:16:06 alexmiller it matters in the context where it is used - when used inside another regex op, it does not start a new nested context, rather it matches elements out of the current context
2016:09:11 00:16:23 alexmiller however s/or is effectively opaquely matching a value
2016:09:11 00:16:42 alexmiller so an example where alt works but or does not is:
2016:09:12 12:24:01 jmglov Sorry for asking such a basic question, but what is the recommended way to test spec'd functions in unit tests (i.e. by running lein test)?
2016:09:12 12:26:16 jmglov But I'm not really sure how to get c.s.test/check to hook into the (deftest ... (checking ...)) style.
2016:09:12 12:26:28 jmglov RTFM links welcome. 😉
2016:09:12 12:29:23 otfrom jmglov: I'd be happy with figuring out that workflow too
2016:09:12 12:32:09 tgk Yeah, I was puzzled with this as well. I ended up writing a very small namespace for it and creating a lein alias for running the specs
2016:09:12 12:32:55 jmglov Using enumerate-namespace?
2016:09:12 12:34:40 tgk The file is only on dev path by the way 🙂
2016:09:12 12:37:49 jmglov Interesting. I didn't realise that test/check had a zero-arity form. Really useful!
2016:09:12 12:38:36 jmglov Does it actually find all specs in all namespaces in your classpath, or something?
2016:09:12 12:39:13 jmglov I don't see you specifying any namespaces under test, or requiring them in.
2016:09:12 12:39:16 tgk Yes, as long as they have been evaluated
2016:09:12 12:39:24 jmglov Wow.
2016:09:12 12:39:34 tgk Ah yes, that’s where refresh comes into the picture 🙂
2016:09:12 12:39:42 jmglov Would your approach actually run all the specs for dependencies as well?
2016:09:12 12:40:02 jmglov Or just stuff in your src?
2016:09:12 12:40:15 jmglov That's nifty!
2016:09:12 12:41:19 jmglov All the same, it would be great to find an approach that would allow me to drop spec generative tests into my standard clojure.test files.
2016:09:12 12:41:46 jmglov I haven't found anything with Google, but stemming is really fighting me on this one. 😉
2016:09:12 12:49:00 tgk Hmm yes, I don’t think specs for dependencies would be run. I can’t see how they would 🙂
2016:09:12 12:49:15 jmglov That's good. 🙂
2016:09:12 12:49:17 tgk I found it very hard to find anyone who’d hooked it into tests
2016:09:12 12:49:40 jmglov So refresh simply evals everything in your source directories?
2016:09:12 12:54:26 alexmiller using spec.test/check or spec.test/instrument will pick up any spec’ed fns that have been loaded and added to the registry, so it depends completely on what code you’ve loaded
2016:09:12 12:55:29 jmglov Thanks, Alex!
2016:09:12 12:55:52 jmglov Also, thanks for the spec Guide. I finished reading it, and it is really excellent!
2016:09:12 12:56:58 alexmiller @bret I personally would prefer your first spec (although looks like you missing the kw namespaces on the map-of and you probably want s/merge instead of s/and)
2016:09:12 12:57:47 jmglov OK, switching gears for a second, I'm obviously doing something silly, but I'm not sure what. I'm trying to spec out the input coming in from some JSON, and I have some code like this:
2016:09:12 13:01:05 jmglov Can anyone shed light on what I'm doing wrong?
2016:09:12 13:01:42 alexmiller @jeroenvandijk yes, please file a jira. this is where we are hurting for an s/vcat which Rich and I have talked about several times.
2016:09:12 13:05:36 alexmiller @jmglov it looks to me like your :fn spec is wrong and should be comparing count of each thing?
2016:09:12 13:06:26 jmglov doh
2016:09:12 13:06:51 jmglov @alexmiller Of course you are right. Thanks for pointing out my idiocy! 🙂
2016:09:12 13:07:14 alexmiller well I wouldn’t go that far. :) fwiw, I’ve done the same.
2016:09:12 13:10:05 jmglov Oh, so much better! Now summarize-results is showing me a bug in my code or spec. 🙂
2016:09:12 13:11:27 bret @alexmiller I guess my point/observation is that the second one is not allowed at all. That was puzzling at first. I've missed s/merge all this time. I’ll look at that and see if that changes anything.
2016:09:12 13:16:31 jeroenvandijk @alexmiller thank, will do
2016:09:12 13:17:24 alexmiller @bret not sure what you mean by your point/observation, sorry
2016:09:12 13:27:34 bret @alexmiller I probably shouldn't start writing at midnight on Sunday night. :)
I guess it boils down to, is the reason this works
(def rks [::k1 ::k2])
=> #'onenine.core/rks
(s/keys :req-un rks)
CompilerException java.lang.IllegalArgumentException: Don't know how to create ISeq from: clojure.lang.Symbol, compiling:(/Users/brety/dev/personal/onenine/src/onenine/core.clj:69:1)
merely a consequence of the s/keys macro implementation or is this intended to not be valid?
2016:09:12 13:27:59 alexmiller well, both
2016:09:12 13:28:16 alexmiller s/keys is a macro and expect a concrete list of keys
2016:09:12 13:28:23 alexmiller so that’s as intended
2016:09:12 13:28:57 alexmiller and the reason most of the spec creating fns are macros is to capture forms for reporting
2016:09:12 13:29:20 jmglov Does anyone know how to make test/check work on private functions? I've tried #'full.namespace/foo, but I get java.lang.IllegalArgumentException: Don't know how to create ISeq from: clojure.lang.Var.
2016:09:12 13:29:46 jmglov Leaving off the #' tells me the function is not public, which I already know. 😉
2016:09:12 13:30:27 alexmiller we might in the future have a fn entry point for s/keys with the caveat that you may lose some of the explain reporting capability
2016:09:12 13:30:44 bret That is what I suspected (the reporting aspect) but wanted to confirm.
2016:09:12 13:31:45 alexmiller @jmglov I don’t think that was considered in the design (and I’m not sure whether it should be)
2016:09:12 13:32:10 jmglov Yeah, I'm a bit naughty, really.
2016:09:12 13:32:51 lvh On a related note: I find myself regularly wanting validation of runtime-defined specs, where I learn e.g. the structure of the JSON in this particular REST API at runtime. This is for presumably obvious reasons, not very convenient right now.
2016:09:12 13:32:58 jmglov This is always a tough call. I want to use private functions to communicate to my clients that they are not part of the interface, but I also want to be able to unit test them. 😕
2016:09:12 13:33:08 lvh (Everything ends up being eval’d, which I guess is fine?)
2016:09:12 13:33:33 bret It would be nice in some case, I think, to be able to define keys once and combine them in different ways ways when building specs. At least, as I think about repeating information in the spec(s) and trying to reduce that.
2016:09:12 13:33:58 bret But I don’t have enough time in spec to be sure.
2016:09:12 13:34:02 jmglov I might just go the foo.bar.internal route, where everything in the internal ns is public and can be tested, but is pretty clearly not for client consumption.
2016:09:12 13:34:27 alexmiller @lvh yeah, I understand that as a use case, not sure how common that will be in general though
2016:09:12 13:34:32 jmglov Using a namespace-level docstring to warn off potential troublemakers, of course.
2016:09:12 13:34:41 alexmiller yeah
2016:09:12 13:35:17 alexmiller @bret well that’s exactly the point of having the registry
2016:09:12 13:40:17 lvh alexmiller: If you give Clojure programmers a feature it seems like a matter of time before they’ll try to express as much of it as data, and then it’s not a long stretch until that data isn’t available at compile time 😉
2016:09:12 13:40:33 lvh I might be able to get around it and just move more stuff into compile-time-land
2016:09:12 13:41:34 alexmiller specs are data in s-expr form
2016:09:12 13:42:41 alexmiller we haven’t released it yet, but I have a spec for spec forms
2016:09:12 13:43:16 alexmiller (which revealed a lot of bugs in s/form :)
2016:09:12 13:45:20 lvh nice; I would very much like that
2016:09:12 13:45:40 lvh since a hypothetical awful person might want to construct specs at runtime and have better feedback about why they don’t work 😉
2016:09:12 13:46:10 alexmiller oh, I don’t think you’re awful :)
2016:09:12 13:46:39 alexmiller it’s reasonable
2016:09:12 13:46:49 alexmiller just not the primary use case we were working to support
2016:09:12 13:47:44 bret @alexmiller Ok, this will help me.
;; I want to check that an input map's keys are valid
;; where the keys are unqualified, some required, some optional,
;; and not allow keys outside that set.
; This is straight forward
(s/def :in/data (s/and
(s/keys :req-un [:in/id] :opt-un [:in/more])
(s/map-of #{:id :more} nil)))
; but I'm (kind of) repeating information.
;
; If I write
(def req-keys [:in/id])
(def opt-keys [:in/more])
(defn unk
"Returns ns unqualified keys for (possibly) qualified ones."
[& ks]
(map #(-> % name keyword) ks))
(s/def :in2/data (s/and
(s/keys :req-un req-keys :opt-un opt-keys)
(s/map-of (set (apply unk (concat req-keys opt-keys))) nil)))
; I have not repeated the key values but s/key doesn't allow it.
What is the proper way to write the spec where I not repeating information? I didn’t initially see a way to piece it together from ‘smaller’ specs since the args in map-of is really just a set used as a predicate.
2016:09:12 13:48:08 bret I could be missing something fundamental.
2016:09:12 13:49:22 alexmiller I’d say generally that Rich believes in open maps and that’s why this is not a feature provided out of the box
2016:09:12 13:49:35 alexmiller and that I think your first example is what I would do if I was doing it
2016:09:12 13:49:56 alexmiller (although nil is not a valid spec there - you want any?)
2016:09:12 13:51:14 alexmiller and I would use s/merge instead of s/and
2016:09:12 13:51:45 alexmiller which I think would gen better
2016:09:12 13:53:41 bret So, s/merge can be used for combining more than s/keys (`s/map-of` in this case)?
2016:09:12 14:00:05 alexmiller s/merge is used to combine (union) map specs
2016:09:12 14:00:38 alexmiller it differs in not flowing conformed results like s/and and also in being better at gen'ing
2016:09:12 14:00:50 bret I get the open map approach and generally like it. One thought I had, that really relates to the reporting aspect, is that s/keys supports ‘and’/‘or’ combinations of key vectors. So, keeping the form used for reporting as close to literal boolean expressions of literal key vectors is not a bad thing. Ok, thanks, this helps.
2016:09:12 14:22:56 rickmoynihan what's the best way to spec that something satisfies? a protocol?
2016:09:12 14:27:11 rickmoynihan obviously you can just use the (partial satisifies? Protocol) predicate... but is there a way for implementers to hook in and extend the generator to generate types that satisfy it? I could imagine that if implementers spec'd their constructing functions, you could get this almost for free.
2016:09:12 14:37:39 alexmiller nothing built-in
2016:09:12 14:54:39 jmglov Here's another fun one. Using the fixed version of the same spec as previously, I can use stest/check on it in my REPL:
2016:09:12 15:00:30 jmglov To be clear, if I REPL into my kpcs.product-catalog.internal-test and run the code above, it works. If I run lein test, I get the exception.
2016:09:12 15:00:47 jmglov I don't see what should be different between the two.
2016:09:12 15:09:16 alexmiller that’s a problem with lein test’s monkeypatching
2016:09:12 15:09:30 alexmiller it’s fixed in (not yet released) next version of test.check
2016:09:12 15:09:39 alexmiller but you can disable lein monkeypatching to fix
2016:09:12 15:28:43 jmglov I'll make a checking function out of it and throw it in a test lib. The output is decent enough with the humane-test-output plugin. 🙂
2016:09:12 15:29:26 jmglov If it's useful, you're welcome. Otherwise, I'm sorry for such a disgusting kludge! 😉
2016:09:12 15:48:27 mike_ananev Hi there! Sorry for very dumb question. How to define spec for function with variable args?
2016:09:12 15:56:00 bahulneel Hi guys, I've just started using clojure.spec and was wondering how to use clojure.spec.test/check with clojure.test
2016:09:12 15:56:46 bahulneel sorry, missed the msg from @jmglov
2016:09:12 16:38:00 alexmiller @mike1452 (s/fdef myf :args (s/cat :map-params (s/? map?))) will take both 0 and 1 (map) arg
2016:09:12 16:38:12 alexmiller you can replace map? with something more specific too of course
2016:09:12 17:20:46 bfabry it seems awkward that core/defn has a special syntax for arity dispatch but spec/fdef doesn't provide one for speccing/testing
2016:09:12 17:21:40 bfabry doesn't really affect me as I never use arity dispatch but I could see it sucking if you previously used it a lot
2016:09:12 17:25:44 alexmiller they are doing different things. most multi-arity functions share param definitions across arities and merging them works very nicely for this in most cases.
2016:09:12 17:28:12 bfabry sure, for the :args, but it doesn't make the :fn for say map less readable?
2016:09:12 17:45:15 alexmiller sure, although I think that’s an unusual case
2016:09:12 17:55:14 bfabry true. I guess the common case would be reduce. smaller arity providing default value
2016:09:12 18:57:10 ikitommi Is there a reason why if-let and when-let can’t return :clojure.spec/invalid?
2016:09:12 19:01:22 hiredman well that is problematic
2016:09:12 19:04:38 hiredman (if-let [a 1] '::s/invalid) works if you need a work around
2016:09:12 19:05:42 ikitommi @hiredman cool, thanks. didn’t know that works too.
2016:09:12 20:26:58 alexmiller @ikitommi heh, that’s fun
2016:09:12 20:28:01 alexmiller there’s actually a ticket related to this
2016:09:12 21:07:10 seancorfield I added the if-let example above to that ticket. I suspect people will run into this in more and more situations as they try to write conformers.
2016:09:12 21:13:59 alexmiller and as there are more spec’ed things
2016:09:12 21:21:52 ag I need a spec that would generate vectors of values taking random elements from a predefined list, e.g: [:foo :bar][:foo][:baz :bar] []… etc.
2016:09:12 21:27:33 alexmiller you can also use the other options on coll-of to set :min-count, :max-count, :count constraints on the spec or :gen-max to cap what the generator will produce
2016:09:13 06:55:13 jmglov @bahulneel I'm working on upgrading a production project to Clojure 1.9 and spec right now, so I'll keep this channel informed of any new stuff I figure out with the clojure.test integration.
2016:09:13 06:56:05 jmglov Given that clojure.spec.test/check takes a quoted symbol, I shouldn't even need a macro to accomplish what I want. A plain 'ol function should do the trick. 🙂
2016:09:13 10:00:26 bahulneel @jmglov thanks, I'll keep my ear to the ground
2016:09:13 11:18:49 jmglov I'm having trouble getting clojure.spec.test/check to run a non-default number of tests. From reading the docs and the code, it looks like I should be able to do this:
2016:09:13 18:16:44 alexmiller just to be sure, that’s the same thing @uwo said above right?
2016:09:13 18:16:54 uwo yes
2016:09:13 18:17:09 alexmiller ok, I’m agreeing that’s wrong if it wasn’t clear :)
2016:09:13 18:19:44 esp1 yup just pointing out that it’s required now. in previous alphas :ret was actually not required on fspec, but in alpha12 that restriction is enforced
2016:09:13 18:20:52 jannis Is there a way to refer to a spec that is defined later in the same file? E.g. (s/def ::foo (s/with-gen ::bar ...)) ... (s/def ::bar ...) ?
2016:09:13 18:22:00 jannis I may be getting an unspecified behavior, where sometimes the spec is found and sometimes it isn't, resulting in conform returning a tagged, conformed value and sometimes returning the untagged original value.
2016:09:13 18:22:17 jannis Granted, the second spec is more complex than in this example but the first is pretty much that.
2016:09:13 18:29:48 darwin clojurescript related question, I’m working on a library which uses clojure.spec to describe a data structure using s/*, the data structure can be used statically during compilation in macros, or during runtime in cljs.
all is nice and shiny on clj side[1], but on cljs side, I want native js arrays to be treated as collections for the purpose of speccing, but unfortunatly s/* does not treat them such. Ended up writing custom predicates for native array case[2], just wondering if there is a better way to teach s/* to walk native js arrays the same way as cljs vectors.
[1] https://github.com/binaryage/cljs-oops/blob/master/src/lib/oops/sdefs.clj
[2] https://github.com/binaryage/cljs-oops/blob/master/src/lib/oops/sdefs.cljs
2016:09:13 18:33:26 jannis Funny. (s/def ::foo (s/with-gen (s/and ::bar) ..)) makes it work at all times.
2016:09:13 18:37:07 alexmiller @jannis I don’t know of any issue with that. however, one of the perf changes in alpha12 means that one spec gets compiled into another’s definition (via delay) so when you change (re s/def) a spec at the repl, downstream specs need to be re-s/def’ed as well
2016:09:13 18:37:17 alexmiller is there any chance you are seeing this behavior at the repl?
2016:09:13 18:37:51 alexmiller maybe when you made it “work” you just caused a new s/def that was needed
2016:09:13 18:38:40 alexmiller @esp1 I don’t know of any change that would have altered this behavior in alpha12, but that’s possible (or that it changed at some point during the alphas)
2016:09:13 18:40:37 esp1 actually previously i was using tonsky’s future-spec port of spec to 1.8, so possibly it was something specific to that, not sure
2016:09:13 18:44:34 jannis @alexmiller: If I load the namespace that these specs are in I get it in the REPL. If I reload the same namespace, it disappears. And I get it when running tests outside the REPL (using boot test).
2016:09:13 18:45:48 jannis This is alpha11 though. I can't use alpha12 at the moment because it fails requiring DataScript due to an invalid ns expression in it.
2016:09:13 18:50:36 alexmiller I can’t even run the first line of that with a bare alpha12 repl, not sure about earlier
2016:09:13 18:51:35 alexmiller I realize this is slimmed down, but I don’t think you should expect to with-gen over a spec that doesn’t exist yet
2016:09:13 18:52:54 jannis These specs are for a language that is recursive (think: Om Next query expressions). How would I include the recursive nature in the spec other than refering to the top-level spec in a part of it?
2016:09:13 18:53:25 jannis The top-level spec needs the sub-spec, the sub-spec needs the top-level spec. Hmm...
2016:09:13 19:04:28 alexmiller yeah, that’s fine
2016:09:13 19:04:35 alexmiller I’m talking about with-gen in particular
2016:09:13 19:05:11 alexmiller you can provide gen overrides separately too of course. gen with recursion is generally hard.
2016:09:13 19:05:37 jannis How do I do it separately?
2016:09:13 19:16:00 alexmiller fns like instrument take an override map
2016:09:13 19:16:09 alexmiller from name (or path) to generator
2016:09:13 19:31:01 jannis Ah, yes. So that would also work with clojure.spec.test/check. I'd prefer to have all generators defined along with the actual spec so it's as reusable as possible (e.g. in combination with fdef args in functions other people may write). I'll see what the best solution is. Thanks!
2016:09:13 19:37:59 bhagany I'm trying to understand why nested calls s/cat don't operate the way I expect - to illustrate:
In my actual situation, I'm trying to reuse the inner s/cat as both the return value of a fn, and an argument to another fn. Also, it does work the way I expect if I use (s/tuple number? number? number?), but I'd like to be able to conform it and get a map. Am I missing something?
2016:09:13 19:43:12 bhagany I'd read it, but I think I have not yet internalized that cat is a regex operator. thanks again
2016:09:13 20:05:45 alexmiller any set of nested regex ops (`cat`, alt, ?, +, *, &, keys*) describes the structure of a single sequential context
2016:09:13 20:06:49 alexmiller which is just like regex on a string
2016:09:13 20:07:15 alexmiller strings are different in that a single character in a string can’t also be a string (but that is true of elements in a Clojure sequential collection)
2016:09:13 22:15:28 gfredericks E.g. gen/vector-distinct-by
2016:09:13 22:21:14 gfredericks @jmglov: https://github.com/gfredericks/schpec exists in case you don't want to bother managing a whole library for that function
2016:09:13 23:07:47 alexmiller @gfredericks nice
2016:09:13 23:40:22 wilkes Are there any conventions around where to put specs? In the namespace, in a special whatever.spec namespace, etc..
2016:09:13 23:46:26 seancorfield @wilkes "it depends" is the answer to that.
2016:09:13 23:46:33 wilkes 🙂
2016:09:13 23:47:22 seancorfield At World Singles, our specs are very data centric so they mostly live in their own namespace and we pull them into other namespaces as needed, and then our tests pull them in to instrument / check the code.
2016:09:13 23:48:11 seancorfield In clojure.java.jdbc all the fdef’s are in a separate namespace so the code still works with pre-1.9.0 versions of Clojure.
2016:09:13 23:49:17 seancorfield But if you know you’re only going to use 1.9.0 onward and you’re mostly spec’ing functions, rather than data, I’d imagine it would be convenient to put fdef’s next to their defn’s in the same namespace.
2016:09:13 23:53:08 wilkes @seancorfield Thanks, that helps. I was expecting this to be at least in flux until people have spent some time with specs in their projects.
2016:09:13 23:56:48 alexmiller Clojure itself is putting specs in clojure.core.specs
2016:09:14 00:17:08 hiredman it seems like that could end up mixing testing code in with, production code (for lack of a better description)
2016:09:14 00:17:35 hiredman is the idea that people will do similar lazying loading kind of stuff like clojure.spec does for test.check?
2016:09:14 00:24:19 seancorfield "mixing testing code in with production code" … how? It’s not the case with how WS is using spec at the moment.
2016:09:14 00:25:46 seancorfield Our spec namespaces have just specs in them — which our (production) code uses for explicit validation and conforming of data at each "system" boundary (input; domain; persistence).
2016:09:14 00:26:35 seancorfield In addition, our test code can pull in the specs and instrument and/or check code. Any test-specific code would live in the test (expectations) namespace, not the main (production) namespace.
2016:09:14 00:27:24 seancorfield (it’s certainly possible I’ve missed some subtlety here but…)
2016:09:14 00:27:50 hiredman do you have specs with custom generators?
2016:09:14 00:30:24 hiredman or, I guess I should say, do you define your own predicates with generators? it seems like that would depend at least on test.check, so you would either need to do lazy loading or have a runtime dependency on test.check
2016:09:14 00:31:01 seancorfield Ah, gotcha...
2016:09:14 00:31:30 hiredman I haven't used spec in anger, so I dunno how it plays out, just something I wonder about
2016:09:14 00:32:16 bfabry @hiredman you can specify the generators in the spec.test/check or instrument call, instead of with the spec itself
2016:09:14 00:32:25 seancorfield We lazy-load via the functions that return generators (and do so in a separate ns):
;; copyright (c) 2016 world singles llc
(ns ws.spec.generators
"Helpers for our specs that need custom generators.
Dynamically loads the namespaces as needed so we can compiler
this without test dependencies.")
(defn fn-string-from-regex
"Return a function that produces a generator for the given
regular expression string."
[regex]
(fn []
(require '[com.gfredericks.test.chuck.generators :as xgen])
(let [string-from-regex (resolve 'xgen/string-from-regex)]
(string-from-regex regex))))
2016:09:14 00:32:51 seancorfield Ooh, typo. compiler instead of compile...
2016:09:14 00:32:54 hiredman ah
2016:09:14 00:35:10 seancorfield We have a few with-gens that leverage stuff in clojure.spec or code that’s already available in production, otherwise we defer to our ws.spec.generators "proxy" namespace.
2016:09:14 00:36:06 seancorfield But, yeah, ensuring those sort of test artifacts only get loaded when we’re testing was something we had to navigate at first...
2016:09:14 00:36:39 seancorfield In our build.boot we have
2016:09:14 00:37:16 seancorfield and that’s only used by our test tasks
2016:09:14 00:39:52 bnoguchi I have been having trouble spec’ing an function arg to another function where the former can be either a function with arity 0 or a function with arity 1 (both not variadic functions). Any ideas?
(s/explain (s/fspec :args (s/alt :arity-0 (s/cat) :arity-1 (s/cat :arg1 any?)) :ret any?) (fn []))
; val: (nil) fails predicate: (apply fn), Wrong number of args (1) passed to: user/eval16844/fn--16845
2016:09:14 00:44:43 bfabry @bnoguchi your spec seems to be working. you said the function should accept 1 or 0, your function only accepts 1
2016:09:14 00:45:04 hiredman I think you need to pull the alt out
2016:09:14 00:45:17 bfabry @bnoguchi fwiw there's a briefer way to describe that spec: (s/cat :arg1 (s/? any?))
2016:09:14 00:46:06 bnoguchi @bfabry It’s written to alternatively accept 1 or 0.
2016:09:14 00:46:27 bfabry @bnoguchi your spec is, but the anonymous function (fn [] ) only accepts 0 arguments
2016:09:14 00:56:35 hiredman there we go, of course it needs to be s/or not s/alt
2016:09:14 00:56:47 hiredman (outside of matching the arglist
2016:09:14 00:56:49 hiredman )
2016:09:14 00:57:39 bfabry oh I'm sorry you wanted to spec either one or the other, not a function that expects both. in that case yes you need two fspec's (two different fiunction specs) not one fspec (a single function that does two things)
2016:09:14 11:53:50 jetzajac Hello here! I’m trying to use spec with a DataScript app. And I want to validate several different things:
1) tx-data in a map form
2) results of a pull
3) entities
In these 3 flavours of data keywords may have different sorts of values. like it’s ok to have an int as a value of ref attr in tx-data, not in a pull. It’s ok to miss some attrs in pull result, not in a tx-data with negative id (creating a new entity, we have a notion of required attrs).
And the question is if we can have different spaces of specs? like separate registries and somehow refer them in spec and make it narrow down to this space. Say, space of tx-data and pulls that have to be validated differently. Not sure if I described the demand clear, bit hope I could find somebody who does these kind of things or be clarified why it’s impossible or is a bad idea. Thanx.
2016:09:14 12:04:47 jetzajac not sure if I like the idea to have one (private) atom for registry. why not a dynamic binding?
2016:09:14 12:16:53 jmglov @bahulneel @otfrom This is what I'm using in tests so far:
2016:09:14 12:29:56 jmglov I see that @esp1 pointed out that :ret is required for fdef now, and I'm trying to spec a function for which the return value is effectively Unit.
2016:09:14 12:30:16 jmglov So I don't care what it returns, only that spec is happy with it. 😉
2016:09:14 12:31:02 bahulneel @jmglov What about a predicate like (constantly true)
2016:09:14 12:35:20 jmglov Probably shouldn't call it any?, though. 😉
2016:09:14 12:35:51 jmglov
user> any?
#function[clojure.core/any?]
2016:09:14 12:35:52 bahulneel yeah, that was more for illustration, best to make it explicit and not hide behind a name
2016:09:14 12:36:24 jmglov I actually like naming it. I think that makes the intention quite clear. I'll probably just call it unit.
2016:09:14 12:36:29 jmglov Clear enough to me.
2016:09:14 12:36:34 bahulneel sure
2016:09:14 12:38:23 jmglov Thanks for that! It's a nice, simple solution.
2016:09:14 12:41:31 bahulneel @jmglov no problem
2016:09:14 12:42:24 bahulneel @jmglov any plans to throw that testing code into a little library to house test helpers?
2016:09:14 12:43:13 jmglov @bahulneel I put it in its own namespace, but I'm not planning to lib it up, no.
2016:09:14 12:43:33 jmglov Not sure the Clojure community needs yet another library of utility functions. 😉
2016:09:14 12:44:30 jmglov I figure best practises will emerge as more people start mixing specs into their tests.
2016:09:14 12:44:57 bahulneel fair enough. Maybe a GIST would do the trick, at least then it's updatable and discoverable
2016:09:14 12:45:01 jmglov I'll stick it in a Gist.
2016:09:14 12:45:13 jmglov Haha, great minds think alike.
2016:09:14 12:45:19 jmglov I'll do that.
2016:09:14 12:47:21 bahulneel also if you put a namespace declaration at the top then, in theory, you could include it as a git submodule in your source tree
2016:09:14 14:52:01 bahulneel @jmglov I noticed that you need to require [clojure.test.check.clojure-test] so that it's loded as some useful multimethods are added
2016:09:14 16:01:21 kkruit Is there any reason to not use instrument for validation if the behavior I want is for the function to throw an error if the data passed is invalid?
2016:09:14 16:01:51 kkruit (in production)
2016:09:14 16:02:42 alexmiller no (as long as it meets your perf concerns)
2016:09:14 16:04:25 kkruit I'm thinking it wouldn't be less performant than doing a verify?, throw, and explain, would it?
2016:09:14 16:06:04 kkruit I can run some tests...
2016:09:14 16:09:10 alexmiller with alpha12, it’s pretty fast but test it and see
2016:09:14 16:09:50 kkruit Will do. Thanks!
2016:09:14 16:20:12 kkruit and by verify? i meant valid?...
2016:09:14 21:40:47 kkruit So I'm looking at using spec in place of schema for a rest api and am running into a question about the best way to handle something. I have something like this:
2016:09:14 21:43:33 kkruit I like using :ns/map-key in combination with :req-un like that but I don't like how long the namespace is/could get any thoughts? Am i shoehorning :ns/map-key somewhere where it shouldn't go?
2016:09:14 21:44:31 arohner @kkruit you can alias namespaces using :require
2016:09:14 21:48:41 arohner that would look for an unaliased namespace named ‘c, rather than resolving the alias ‘c
2016:09:14 21:48:43 arohner np
2016:09:14 22:44:25 stuartmitchell Is this a known issue
(def common-attributes [::id ::type ::from-date ::to-date ::style])
(s/def ::common
;; defines the common parts of an activity
(s/keys :req-un common-attributes))
Gives this compiler error
---- Could not Analyze src/day8/plan8/spec/activity.cljs line:24 column:3 ----
Don't know how to create ISeq from: clojure.lang.Symbol
22 (s/def ::common
23 ;; defines the common parts of an activity
24 (s/keys :req-un common-attributes))
^--- Don't know how to create ISeq from: clojure.lang.Symbol
25
2016:09:14 22:45:26 seancorfield s/keys is a macro so it doesn’t evaluate its arguments (so this is by design)
2016:09:14 22:46:04 stuartmitchell strangely this works
(def common-attributes [::id ::type ::from-date ::to-date ::style])
(s/def ::common
;; defines the common parts of an activity
(s/keys :req-un (concat common-attributes)))
2016:09:14 22:46:15 stuartmitchell but will it fail later on?
2016:09:14 22:46:25 seancorfield It does not do what you think it does.
2016:09:14 22:47:45 seancorfield (I’m a bit surprised it conforms to spec’s specs — are you trying this on Alpha 12 or an earlier version?)
2016:09:14 22:49:58 stuartmitchell 1.8.0 I'll bump it up
2016:09:14 22:51:03 stuartmitchell but how should we compose specs i.e. I want spec ::b to be spec ::a with some extra keys
2016:09:14 22:51:40 stuartmitchell this is what I wanted
(def common-attributes [::id ::type ::from-date ::to-date ::style])
(s/def ::common
;; defines the common parts of an activity
(s/keys :req-un common-attributes))
(s/def ::tv
;; defines the attributes for a tv activity
(s/keys :req-un (concat common-attributes [::daymask ::tarp ::cpt ::rate
::market])))
2016:09:14 22:54:51 seancorfield Oh, so you’re using the backport of clojure.spec rather than the official version?
2016:09:14 22:55:35 seancorfield You want clojure.spec/merge for merging key specs.
2016:09:14 22:55:47 stuartmitchell ahh thank you
2016:09:14 22:55:58 seancorfield
boot.user=> (doc s/merge)
-------------------------
clojure.spec/merge
([& pred-forms])
Macro
Takes map-validating specs (e.g. 'keys' specs) and
returns a spec that returns a conformed map satisfying all of the
specs. Unlike 'and', merge can generate maps satisfying the
union of the predicates.
2016:09:14 22:56:11 seancorfield (note: also a macro)
2016:09:14 23:01:58 stuartmitchell Thx, I also tried the concat version on 1.9.0-alpha12 and it didn't give a compiler error however I am using clojurescript as well so someone should probably try it on vanilla java, if indeed it should fail spec's specs
2016:09:14 23:12:32 alexmiller there are no included spec specs
2016:09:14 23:13:44 alexmiller I’ve made some but they are a bit out of date and unclear whether we will release them. there are some recursion issues with checking specs in spec too. :)
2016:09:14 23:32:31 seancorfield @alexmiller Am I right that (s/keys :req-un (concat common-attributes)) is going to create a spec where the two required keys are the symbols concat and common-attributes? And that will error out when used…?
2016:09:15 00:21:04 kenny Can you set the :default method for a multi-spec? Just adding a multimethod with :default for the dispatch value results in no method when calling explain. Small example:
2016:09:15 00:38:01 hiredman I suppose the default stuff would be tricky to handle well for generative testing
2016:09:15 00:51:59 kenny Is there reasoning behind this?
2016:09:15 00:54:12 kenny Behind not handling :default in defmethod ^
2016:09:15 00:54:32 kenny Or has it just not been discussed?
2016:09:15 01:09:56 alexmiller @seancorfield that would be my guess too
2016:09:15 01:10:20 alexmiller @kenny if :default doesn’t work, it should be made to
2016:09:15 01:10:37 alexmiller at least as far as my thinking atm, so jira appreciated
2016:09:15 01:10:55 alexmiller I did put a patch in alpha12 to make hierarchies work again, but I don’t recall testing :default stuff
2016:09:15 01:12:23 kenny Oh shoot, I'm still on alpha11 because of the hash changes. I'll test it out in alpha12 and if it doesn't work I'll file a jira issue.
2016:09:15 01:15:03 kenny @alexmiller Yes it is fixed in alpha12. Thanks!
2016:09:15 06:11:45 curlyfry How "safe" am I in using spec in ClojureScript? Is it subject to any large changes in the future?
2016:09:15 09:43:22 jmglov Any tips for writing a spec for the string version of a positive int? Here's what I'm doing now:
2016:09:15 09:43:37 jmglov I assume that will need a custom generator though, right?
2016:09:15 09:43:51 jmglov Any more idiomatic way to do it?
2016:09:15 11:08:38 flipmokid Could spec be used to parse a vector such as '[1 + 3 * 4] and conform the input in such a way so that operator precedence is preserved?
2016:09:15 14:16:16 jmglov The first time you see a function spec actually catching invalid input data is absolutely wonderful!
2016:09:15 14:33:28 alexmiller @curlyfry spec is in alpha and subject to change (but probably unlikely to change dramatically, more additive)
2016:09:15 14:38:22 jmglov Is there a way to use conform to drop all map keys not explicitly mentioned in an s/keys spec?
2016:09:15 14:39:57 jmglov Let's say I have something like this:
2016:09:15 14:43:11 jmglov @alexmiller Wow! You guys thought of everything!
2016:09:15 14:43:59 jmglov BTW, it is possible to run spec.test/check on a private function; my failure to do so was operator error. 🙂
2016:09:15 14:46:23 alexmiller No
2016:09:15 14:46:35 alexmiller Right now at least
2016:09:15 14:46:53 alexmiller No one has filed an enhancement for that yeah afaik
2016:09:15 14:47:11 alexmiller I'm not sure whether it should be supported or not
2016:09:15 14:47:34 jmglov Well, it works. 🙂
2016:09:15 14:49:04 jmglov (stest/check 'foo.bar/baz) is just fine, even if baz is defn-'d.
2016:09:15 14:50:02 jmglov I personally think it should not be disallowed, since Clojure generally doesn't try too hard to save you from yourself with respect to "private" stuff in a namespace.
2016:09:15 14:50:31 jmglov e.g. you can happily call a private function as long as you grab its var.
2016:09:15 18:02:22 uwo how might I spec a map whose keys I do not know, but whose values all have the same shape?
2016:09:15 18:03:31 bfabry @uwo map-of
2016:09:15 18:03:37 uwo doh. thanks
2016:09:15 18:03:47 bfabry np
2016:09:15 18:52:21 richiardiandrea I have a little problem with this small test:
Am I spec-ing it right? (I don't see "here" in the logs)
2016:09:15 19:17:33 bfabry @richiardiandrea instrument does not check :ret or :fn
2016:09:15 19:17:43 richiardiandrea oh
2016:09:15 19:18:02 richiardiandrea only :args ?
2016:09:15 19:18:07 bfabry yup
2016:09:15 19:18:23 bfabry :ret and :fn are checked during clojure.spec.test/check checks though
2016:09:15 19:18:27 richiardiandrea and what do I need to use to check :ret at runtime while developing ?
2016:09:15 19:18:29 richiardiandrea ah ok
2016:09:15 19:18:57 richiardiandrea tnx @bfabry I did not know that
2016:09:15 19:19:16 bfabry np
2016:09:15 20:32:35 richiardiandrea so if I want to instrument all the instrumentable symbols in my project, should I run! on instrumentable-syms the function instrument?
2016:09:15 20:33:17 alexmiller No just call instrument with no args
2016:09:15 20:34:33 alexmiller Also be aware of enumerate-ns
2016:09:15 20:34:40 richiardiandrea @alexmiller cool, sorry it was right there in the docs
2016:09:15 20:35:07 richiardiandrea reading through it now
2016:09:15 22:17:59 smw oh my god… s/exercise is so cool.
2016:09:16 09:40:31 jmglov I have a couple of timestamp string specs that require custom generators. I'm using a function to make the generators with the following function spec:
I'd really like to spec the function, but that would require adding org.clojure/test.check to my non-dev dependencies, which feels a little yucky. Is there another way to ensure that the return value of the function is a generator? clojure.spec.gen doesn't have a generator? function.
2016:09:16 11:34:03 jmglov I have a data structure like this:
The schema version can either be "v1" or "v2", and the contents of the payload vary based on this. Is there any good way to write a custom generator that will match up version and payload? I can write the conformer reasonably easily, but I can't think of a good way to do the generator.
2016:09:16 11:57:36 jmglov OK, I came up with something that feels only mildly hackish:
2016:09:16 16:09:10 hiredman re: the v1 vs v2 thing, that sounds sort of like what multi-spec does
2016:09:16 16:21:22 hiredman re: :clojure.spec/unknown, my guess is it is something related to the Specize protocol, the fall through Object case for that protocol uses unknown as the name for the predicate
2016:09:16 16:38:57 jmglov @hiredman I remember reading about multi-spec now that you bring it up. I think you're right there!
2016:09:16 16:39:49 jmglov Any clue what I would need to do to solve the :clojure.spec/unknown problem?
2016:09:16 16:40:13 jmglov Or maybe multi-spec would sort that for me anyway. 🙂
2016:09:16 16:40:25 hiredman yeah, maybe
2016:09:16 16:42:05 alexmiller @jmglov that’s a known issue that Rich and I are working on
2016:09:16 16:42:14 alexmiller just hasn’t been finished yet
2016:09:16 16:42:21 jmglov Nice!
2016:09:16 17:17:26 uwo I’ve instrumented the following spec and it catches errors except for the optional parameter. Any advice?
2016:09:16 17:18:10 uwo I can provide any value for skip/limit and it won’t complain
2016:09:16 17:20:35 bfabry @uwo :opt-un not :un-opt
2016:09:16 17:20:54 uwo jeeze, I’m on a roll with stupid questions. thanks!
2016:09:16 17:21:39 bfabry haha it's all good. it'd be nice if spec caught those, but I imagine having spec spec'd would actually be a rather difficult problem
2016:09:16 18:57:48 alexmiller I have spec specs but using them presents some problems
2016:09:16 19:46:04 liamd does anyone know of any guides that go beyond explaining how spec works and show how to integrate it into a real project?
2016:09:16 19:53:54 hiredman you know, there isn't a non-alpha version of clojure released with spec yet
2016:09:16 19:54:28 liamd i know, but how is it intended to be used? what do i do with my specs in an actual codebase?
2016:09:16 19:55:40 hiredman like where to put specs in relation to what they are specing?
2016:09:16 19:57:47 liamd and i see the instrument function
2016:09:16 19:58:08 liamd is the idea that you have them as like accessories to your test suite or that they would run on deployed code?
2016:09:16 19:58:43 hiredman I think spec can be split in to 1. descriptions of data 2. ways to use those descriptions
2016:09:16 19:58:55 liamd yeah so 2 is where i'm foggy
2016:09:16 19:59:02 hiredman right
2016:09:16 19:59:24 hiredman #2 can be basically anything, but spec provides a few things out of the box
2016:09:16 20:00:04 hiredman a. generating data that matches the description b. checking if data matches the description
2016:09:16 20:00:22 hiredman and it actually does b. in a few different ways
2016:09:16 20:01:08 hiredman one way it can check that data matches is you can call instrument
2016:09:16 20:01:53 hiredman and instrument will instrument your functions with checks (and some of those checks rely on generating data, so you really shouldn't instrument out side of testing)
2016:09:16 20:02:22 hiredman another way is using valid? like in that person-name function
2016:09:16 20:07:22 hiredman instrument is very similar to sort of the classic idea of what you can do with assertions, assert all these expensive pre and post condition checks in development, then compile with assertions turned off for production
2016:09:16 20:07:56 hiredman (not that I have ever seen a clojure build that turns off assertions, but I am sure they exist)
2016:09:16 20:08:01 seancorfield @liamd I think it’s mostly too early for standard patterns of usage to have settled down (and I don’t think there’s One True Way to use specs anyway). Have you read the http://clojure.org Guide? And watched Stu Halloway’s webcast demos of it? Also the Cognicast with Rich talking about the motivations for it is good listening.
2016:09:16 20:08:22 liamd i've listened to the cognicast and the generative testing was what sounded most useful to me
2016:09:16 20:08:26 liamd and yeah i've read the guide
2016:09:16 20:08:29 liamd i'll check out the webcast
2016:09:16 20:08:37 bfabry as an aside, instrument doesn't do what would normally be considered :post checks. It only checks the :args section of an fdef
2016:09:16 20:08:56 liamd maybe it's because i'm thinking of spec as an answer to the lack of typing? are those related concerns?
2016:09:16 20:09:28 seancorfield At World Singles, we’re spec’ing data primarily and using conform as part of our validation and transformation across application layer boundaries. We’re using instrument to augment our testing. We’re starting to work with the generative side more at this point but we haven’t settled into a good flow for it yet.
2016:09:16 20:10:01 hiredman maybe more like a database schema than what you would typically think of as a type
2016:09:16 20:10:08 seancorfield @liamd Rich is pretty clear that spec != type system and should not be viewed the same way
2016:09:16 20:12:04 liamd so basically my thinking is:
- in a statically typed system you can make some assumptions about your passed in args since they wouldn't compile otherwise
- without static typing we can just make those assumptions anyway but we might have to debug and get ugly errors down the line
- or we could do some defensive programming and handle incorrect args gracefully (by some standard, a nice error or whatever)
2016:09:16 20:12:11 richiardiandrea A side question: how folks use exercise?
2016:09:16 20:12:44 bfabry it's not a type system, but it's clearly geared at solving some of the same problems in a different way, as well as solving some problems that type systems don't. documentation, correctness, better errors
2016:09:16 20:12:48 liamd so does spec just make that third point easier? do a conform and then handle malformed inputs in every function without have to rewrite ugly validation logic everywhere
2016:09:16 20:13:50 bfabry @liamd instrument and clojure.spec.test are attempts to make us more confident about point 2
2016:09:16 20:13:55 hiredman I would say no
2016:09:16 20:14:18 hiredman if you have a system where you are calling conform in every function on all your arguments, something has gone way wrong
2016:09:16 20:14:20 seancorfield @liamd We’re finding spec to be a better way to write our existing validation and transformation code since it abstracts out the specification part "as data". We don’t view it as "defensive programming".
2016:09:16 20:15:04 liamd @shaun-mahood i'll take a look
2016:09:16 20:15:06 liamd thanks!
2016:09:16 20:15:32 hiredman like, where are your functions getting passed all these malformed inputs from?
2016:09:16 20:15:46 liamd it's a wild world out there
2016:09:16 20:15:52 hiredman at an edge, like a rest api, sure
2016:09:16 20:16:44 hiredman if all your functions are exposed an all passed arbitrary input from the outside world, sure
2016:09:16 20:16:54 liamd i was just thinking "couldn't we try to analyze the specs at compile time" and then i clicked spectrum
2016:09:16 20:17:02 liamd looks neat
2016:09:16 20:17:26 liamd but i mean isn't that the strength of type systems
2016:09:16 20:17:44 hiredman type systems are going to need help at the edge too
2016:09:16 20:17:44 liamd we can make assumptions about the data we're handling
2016:09:16 20:18:05 liamd right, parsing json in any type system is sticky
2016:09:16 20:18:24 hiredman type systems can only analyze data they see, random data coming from the edge isn't going to be seen at compile time
2016:09:16 20:19:25 arohner @liamd no, spectrum can’t handle that. It can prove that you haven’t validated the data though
2016:09:16 20:19:31 seancorfield Spec can also enforce data "structure" in a way that type systems cannot. It can specify valid ranges of numbers, relationships between parts of a data structure, patterns in strings and so on.
2016:09:16 20:20:19 arohner @seancorfield there are workarounds for that in spectrum, in some cases
2016:09:16 20:20:21 hiredman @seancorfield: that is just opening the door and waiting for someone to step in and start talking about dependent types
2016:09:16 20:20:43 arohner because spectrum can assert that the spec gets validated at runtime
2016:09:16 20:20:51 bfabry @liamd spec isn't going to start down the path of compile time checking other than for macros. at the end of the day clojure is a dynamic language. spec is an attempt to give us stronger confidence about what we're passing around at test and dev time using instrument and check. It also includes tools to validating the shape of things at runtime in production but imo I think that's more meant for things that are at high risk of being invalid (boundaries) rather than things that are likely fine
2016:09:16 20:20:53 seancorfield @hiredman Hahaha… true… but we don’t have those in very many languages and almost none that are in common production usage!
2016:09:16 20:21:26 seancorfield Besides, if folks want a type system, then go use a statically typed language. Clojure is not that language.
2016:09:16 20:21:46 hiredman I think the big thing about spec is the language of terms (regular clojure expressions) and the language of descriptions (specs) are the same, so you can easily get access to specs at runtime, and write some interpreter for them to do whatever you want
2016:09:16 20:22:10 liamd i see. so it should be though about as "a set of tools that helps mitigate the downsides of a dynamic language during your dev work flow"
2016:09:16 20:22:31 seancorfield I don’t see those as "downsides" 🙂
2016:09:16 20:22:42 hiredman calling them downsides reflects a particular stance that is not universal
2016:09:16 20:23:01 liamd well surely spec was developed as a solution to some perceived problem?
2016:09:16 20:23:16 liamd maybe i'm wrong in thinking that that problem is rooted in lack of typing
2016:09:16 20:23:26 bfabry it also gives you more confidence than any static type systems I'm aware of because predicates can be arbitrary
2016:09:16 20:23:36 richiardiandrea A very nice talk about this was the last one by Felleisen at Clojure/West
2016:09:16 20:24:15 hiredman @liamd: static typing is a solution to a similar set of problems
2016:09:16 20:25:12 hiredman but it is sort of like saying you have a problem P, and solutions A and B, and saying solution B is there to fix a lack of solution A
2016:09:16 20:25:22 hiredman no, B is there to fix P
2016:09:16 20:25:54 arohner there’s significant overlap between spec and static types, but neither completely subsumes the other
2016:09:16 20:26:18 arohner there are bugs that can be caught by static types that are hard to catch w/ spec, and vice versa
2016:09:16 20:26:34 bfabry static type systems give a lot of confidence about correct calls (probably more than spec) but provide no tools for regular data validation. they're also limited in what they can describe regarding correctness
2016:09:16 20:28:19 hiredman in most typed languages, the language of terms and the language of types are distinct
2016:09:16 20:29:25 liamd i see
2016:09:16 20:29:26 hiredman types end up being kind of their own langauge that exists largely at compile time
2016:09:16 20:29:48 hiredman specs are more like a database schema, they exist in the system and you can poke and fiddle with them live
2016:09:16 20:29:49 liamd so due to them being just plain old clojure we can ingest the specs and do something with them
2016:09:16 20:30:07 liamd is there a way to get the spec of something at run time?
2016:09:16 20:30:31 hiredman things don't inherently have a spec
2016:09:16 20:30:39 liamd right
2016:09:16 20:30:48 hiredman (another different from types)
2016:09:16 20:30:50 arohner yes, you can get spec definitions at runtime
2016:09:16 20:30:50 liamd we just have things and specs and we can smash them into each otehr if we want
2016:09:16 20:31:49 liamd would it make sense to use specs as part of your logic? like if this conforms to my Person spec do one thing else do some other thing
2016:09:16 20:31:57 bfabry totally
2016:09:16 20:32:06 liamd or i could see in web dev using them to validate incoming json
2016:09:16 20:32:18 bfabry convenient dsl for expressing that kind of logic
2016:09:16 20:32:39 liamd it'd be nice if explain could map back to a json error
2016:09:16 20:33:48 liamd i don't know if api consumers want to deal with ;; val: 42 fails spec: ::suit predicate: #{:spade :heart :diamond :club}
2016:09:16 20:34:03 arohner s/explain-data
2016:09:16 20:34:21 liamd ah, there it is
2016:09:16 20:34:23 liamd very nice
2016:09:16 20:50:12 seancorfield @liamd As I noted above, at World Singles, we are using spec to replace our custom validation and transformation logic so we can have separate, comprehensive specifications we can show to and discuss with product owners / stakeholders, and it removes a lot of bespoke code.
2016:09:16 20:51:07 seancorfield We are actually using spec for three layers: input, domain, persistence and conforming between them. The validate-and-conform aspect of spec is very nice.
2016:09:16 20:53:11 liamd is that running only during dev or when you deploy?
2016:09:16 21:06:57 seancorfield It’s a core part of our production code, calling conform.
2016:09:16 21:08:21 seancorfield We also use instrument in testing, and we’re using some generative testing too (`check`). As I said above, we haven’t yet settled onto a final workflow for the latter since generative testing can be slow and we want "unit tests" to be fast (so we can run them easily while writing code).
2016:09:17 20:04:10 surreal.analysis Chicago Clojure also recently had a meeting on Spec, which was posted to Youtube a couple weeks ago - https://vimeo.com/181231654
2016:09:18 14:01:41 wilkes Can someone post the slides for Chelimsky’s intro?
2016:09:18 19:11:43 lvh Hm; are there any specs somewhere that someone is collecting for datomic/datascript datoms?
2016:09:18 22:48:52 olivergeorge Nothing particularly magic in there. JDBC makes it easy to inspect types for table columns.
2016:09:18 22:49:52 olivergeorge An interesting extension would be to interpret relations and generate optional keys
2016:09:18 22:50:21 olivergeorge (My use case is generating dummy data for a re-frame UI which is fed from a pull api (which in turn mirrors a db schema)
2016:09:19 06:08:43 jasonjckn is there an inverse of s/conform ?
2016:09:19 17:28:04 jasonjckn @jeroenvandijk thanks! I ran head smack right into that issue yesterday 🙂
2016:09:19 17:28:20 jasonjckn @jeroenvandijk but used s/conformer to get around it
2016:09:19 17:28:39 alexmiller btw, that issue is imo a problem with conform, not unform :)
2016:09:19 17:29:02 jasonjckn aside from that issue is making macro writing a joy
2016:09:19 17:29:14 alexmiller are you using conform to destructure input?
2016:09:19 17:29:20 jasonjckn yes
2016:09:19 17:29:40 alexmiller I haven’t seen too many people doing that yet but our presumption has been that it would remove a lot of fragile gross code
2016:09:19 17:30:33 jasonjckn the pattern is walking + match->
2016:09:19 17:30:40 jasonjckn I implemented match->
2016:09:19 17:32:10 alexmiller cool
2016:09:19 17:38:36 patrkris Is clojure.spec a suitable tool for validating JSON formatted HTTP request bodies? I am asking because the pervasive use of namespaced keywords specs seems to conflict with that
2016:09:19 17:39:49 patrkris I mean using namespaced keywords as object keys in JSON seems unnatural
2016:09:19 17:50:53 alexmiller you can use s/keys with :req-un and :opt-un for that
2016:09:19 17:51:10 alexmiller many people are and it’s been discussed a number of times here - you might find it in the back chat
2016:09:19 17:52:51 patrkris That was what I thought about doing myself. But then I started thinking about nested maps and whether that would end up posing a problem. Probably not 🙂
You mentioned some time ago that you were thinking about adding a way to "rename" keys in s/keys. Any development on that?
2016:09:19 17:54:42 alexmiller nope. up to Rich whether that idea comes back, so I’m not sure if it will.
2016:09:19 21:28:50 liamd i’m not able to import clojure.spec.gen into my clojurescript project? any idea why?
2016:09:19 21:29:07 liamd
No such namespace: clojure.spec.gen, could not locate clojure/spec/gen.cljs, clojure/spec/gen.cljc, or Closure namespace “clojure.spec.gen”
2016:09:19 21:32:23 ggaillard Hi @liamd, according to the current clojure.spec guide, it seems you need to add [org.clojure/test.check "0.9.0"] to your :dev profile in your project.clj
2016:09:20 00:29:29 danielcompton Is there a JIRA issue open for spec coercions (in the same way that schema does them)? I’m aware of conform, but that isn’t quite the same
2016:09:20 01:20:09 alexmiller there is no plan to add that
2016:09:20 01:20:35 alexmiller Rich sees them as separate concerns
2016:09:20 08:00:49 otfrom I'm presuming I'm doing something wrong and that there is a function in core that will do this, but this bit of code is working for me to integrate spec tests into my normal clojure.test flow
2016:09:20 08:00:54 otfrom
(defn passes? [sym]
(-> sym
stest/check
first
:clojure.spec.test.check/ret
:result))
(t/deftest conformer-testing
(t/testing "Checking the conformer"
(t/is (true? (passes? `sut/my-specced-func)))))
2016:09:20 08:01:14 otfrom I've elided the bit where I define the :args :ret and :fn for my-specced-func
2016:09:20 08:02:33 otfrom is there something that does the same as passes? and is true? really the right way of putting this into t/is ? Success and failure both return truthy values which is why I'm using it atm
2016:09:20 08:22:41 jmglov I have a record that implements a protocol. How do I spec this?
2016:09:20 08:53:14 jmglov When I try to gen/sample some deeply nested spec, I get a "Couldn't satisfy such-that predicate after 100 tries." exception with no mention of the spec or predicate in question. Is there a good way to figure out which spec caused this?
2016:09:20 08:56:13 jmglov So far, my only hope is a binary search by running gen/sample on the sub-specs until I find the offender. 😞
2016:09:20 09:04:44 bahulneel @jmglov I had this problem also. I had to eyeball my specs and try exercising individual ones. In the end, it's usually ones that use and and limit the valid input to some "shape". To fix these I use with gen, an example I stole is here: https://github.com/nwjsmith/datomic-spec/blob/master/src/datomic_spec.clj#L6
2016:09:20 09:05:52 jmglov @bahulneel Yes, that was exactly it. My offending one turned out to be (s/and map? empty?). I changed it to #{{}}. 🙂
2016:09:20 09:06:35 bahulneel nice
2016:09:20 09:09:09 bahulneel @jmglov it's worth noting that, in that case (s/and empty? map?) will also work fine
2016:09:20 09:09:56 jmglov Of course! That's actually nicer to read, I think.
2016:09:20 09:09:59 jmglov Thanks!
2016:09:20 09:10:39 bahulneel the only issue is that the explanation would fail empty? before map? so the message may not be as useful
2016:09:20 09:39:00 mpenet I am not really interested in the conform part right now
2016:09:20 09:39:34 mpenet I just need generating decent json-schemas from specs
2016:09:20 09:42:00 jmglov @bahulneel Nice!
2016:09:20 10:03:57 ikitommi ok, not there yet. It would be easy to cook up a working prototype of the conversion from existing spikes, but still gaps to make it robust and extendable. Good thing that, there are smarter people here :)
2016:09:20 10:05:11 ikitommi Spectrums parse-spec might be a good way for it.
2016:09:20 10:06:22 mpenet Isnt it possible just to gen from s/form?
2016:09:20 10:06:55 mpenet It s a one time thing so perf doesnt matter
2016:09:20 10:11:40 ikitommi Hmm... there was one version with s/form & core.match, can't recall what was the problem with it. Maybe just not finished.
2016:09:20 10:14:52 ikitommi actually, it could be extended too, with multimethod on the spec symbol. Ping @miikka.
2016:09:20 11:40:30 mpenet I have something along these lines already, even tho the multimethod is kinda useless ultimately, it can be closed
2016:09:20 12:09:16 bhagany @jmglov: regarding protocol implementations - I read (on the mailing list, I think), that directly spec’ing protocol fns wasn’t supported, and the recommendation was to make the protocol fns “private”, provide wrapper fns, and spec those. This worked fine for me, especially because I had some common logic that fit nicely in the wrapper fns.
2016:09:20 12:19:54 jmglov @bhagany I'm not trying the spec the protocol functions themselves, but rather an argument to a function that should implement a certain protocol. i.e.
Of course implements? is not a real function, but this illustrates what I'm trying to do, I hope.
2016:09:20 12:26:31 mpenet you can use satisfies?
2016:09:20 12:32:36 jmglov @mpenet Exactly what I needed! Thanks!
2016:09:20 12:37:03 mpenet it gets more tricky if you want to add gen support
2016:09:20 12:38:07 jmglov No need for now, thank goodness.
2016:09:20 15:29:49 bhagany I’m trying to run “free” generative tests in cljs, using doo, but it seems as though the registry is not being populated in my test build. I’m importing a namespace that defines some named specs, and I can run the functions in that namespace, but clojure.spec.test/check and checkable-syms are both empty. However, in my figwheel repl, the registry is populated as I would expect. Any ideas what I might be missing?
2016:09:20 15:33:16 bhagany I should also add that clojure.spec.test/enumerate-namespace does see the fns that are spec’d in my test build
2016:09:20 15:34:56 bhagany and also that I can (s/valid? ::some/data-spec data) in my test build. It seems like only fn specs are missing.
2016:09:20 15:43:40 husain.mohssen Possibly stupid question: why is s/fdef not called s/defn ?
2016:09:20 16:05:34 alexmiller because that’s not what Rich named it
2016:09:20 16:17:06 husain.mohssen good enough for me 🙂 all hail Master Rich 🙂
(I was asking in case there was some subtlety I’m missing)
2016:09:20 19:40:00 liamd having a hard time wrapping my head around writing generators, how would i write a generator for a set of distinct strings?
2016:09:20 19:40:52 hiredman there is a function, it might be called one-of
2016:09:20 19:42:27 liamd i tried (gen/fmap set (gen/vector gen/string))
2016:09:20 19:43:00 hiredman oh, I misunderstood
2016:09:20 19:43:24 hiredman there is a generator for sets
2016:09:20 19:43:41 hiredman so (gen/set gen/string)
2016:09:20 19:45:07 liamd excellent that did it
2016:09:20 19:45:14 hiredman you may need to do some voodoo connect that to spec, I haven't looked too much at how test.check and spec are tied together
2016:09:20 19:45:24 liamd i just had to do with-gen
2016:09:20 19:45:28 liamd really easy
2016:09:20 19:48:40 liamd here’s another one: any idea how to generate long strings?
2016:09:20 19:48:45 liamd say > 200 chars?
2016:09:20 19:53:25 hiredman if you run more generative test iterations, the size of the strings will get larger, but it would likely be easier to create a special purpose larger string generator
2016:09:20 19:54:50 hiredman which maybe some combination of resized and the string generator would help you do
2016:09:20 20:05:01 bfabry there's sooooo much stuff in the clojure.test.check.generators namespace, trying to internalise all that is the one thing I'm not looking forward to when we start using spec
2016:09:20 20:08:46 liamd the docs don’t mention everything so you gotta dig into the sources
2016:09:20 20:08:58 liamd that said, using spec with devcards is kicking ass right now for building om ui's
2016:09:20 20:22:11 alexmiller in many cases it’s easier to just gen from a spec
This throws an error `java.lang.AssertionError
Assert failed: Arg to vector must be a generator (generator?
generator)`
How can I generate a vector of characters from the alphabet?
2016:09:20 20:54:58 madstap Other question, how can I make a generator from a set?
2016:09:20 20:56:18 hiredman what namespace is gen?
2016:09:20 20:56:33 madstap clojure.spec.gen
2016:09:20 20:56:51 hiredman I bet you need to invoke gen/char-alpha
2016:09:20 20:57:27 hiredman the things in clojure.spec.gen aren't actually generators, they are 0 argument functions that you invoke to get a generator
2016:09:20 20:57:50 madstap Thanks, that worked
2016:09:20 21:01:39 madstap And how can I make a generator from a set?
2016:09:20 21:02:43 hiredman you have a set of elements and you want a generator to pick elements from it?
2016:09:20 21:03:00 madstap Yeah
2016:09:20 21:03:07 hiredman gen/elements
2016:09:20 21:03:47 alexmiller or of course just (s/gen #{:a :b :c})
2016:09:20 21:04:02 alexmiller people do not leverage gens from specs enough imo
2016:09:20 21:04:18 alexmiller that should be your first choice
2016:09:20 21:04:54 alexmiller then stuff in spec.gen, then stuff in test.check
2016:09:20 21:05:02 alexmiller or test.chuck etc
2016:09:20 21:06:34 hiredman so (s/gen (s/spec number?)) instead of gen/number or whatever?
2016:09:20 21:06:58 alexmiller just (s/gen number?)
2016:09:20 21:07:00 alexmiller but yes
2016:09:20 21:07:29 ggaillard @alexmiller about that case, is it normal for
2016:09:20 21:07:47 alexmiller well any set with falsey elements is going to be a bad time
2016:09:20 21:08:17 alexmiller that’s kind of a special case - it will return falsey elements which is the signal that the value doesn’t match the spec
2016:09:20 21:08:31 alexmiller in this case (s/gen boolean?) will be better
2016:09:20 21:08:58 ggaillard Oh, I get it !
2016:09:20 21:09:41 alexmiller and if you want to support nil,true,false then (s/gen (s/nilable boolean?))
2016:09:20 21:12:30 ggaillard Thank you !
I had some bad time with (s/gen boolean?) in ClojureScript, throwing an error. I figured it out with with-gen and (gen/elements #{true false}). I don't know if it's a known bug or not btw…
2016:09:20 21:13:37 alexmiller sounds like a bug
2016:09:20 21:14:11 alexmiller you might ask in #clojurescript to see what @dnolen thinks
2016:09:20 21:17:34 dnolen @ggaillard it should work, make sure you are on the latest 1.9.229, if it still doesn’t work, file an issue in JIRA
2016:09:20 21:19:49 ggaillard You are quick! I didn't had time to post in #clojurescript 🙂
Thank you, I'll check that.
2016:09:20 21:34:27 ggaillard I was using 1.9.216, 1.9.229 fixed it. Sorry for that 😕
2016:09:20 22:56:58 devn The clojure.spec/nilable docs say it takes a pred, but it also seems to work if it's passed a spec. Is that behavior I can rely on?
2016:09:21 02:22:22 gardnervickers Hey folks, if I had a key in a map that I want to constrain to be a function that conforms to a spec, how would I go about that? I thought I would be able to pass the results of s/fdef to (s/def ::my-key …) but that does not seem to be the case.
2016:09:21 02:25:16 hiredman fspec
2016:09:21 02:27:14 gardnervickers Oh jeeze, missed that one thanks
2016:09:21 02:27:20 hiredman I am sort of surprised that the result of fdef didn't work though
2016:09:21 02:49:09 alexmiller It returns the result of def not fspec
2016:09:21 02:49:24 alexmiller Fdef is def + fspec
2016:09:21 02:49:50 bfabry ah, so you could use (s/spec ::my-key) @alexmiller ?
2016:09:21 02:50:03 bfabry wait
2016:09:21 02:50:06 bfabry that makes no sense
2016:09:21 02:50:24 bfabry so what does fdef define?
2016:09:21 02:50:48 alexmiller Same as def but the key is a symbol not a keyword
2016:09:21 02:51:24 alexmiller It registers, not defines
2016:09:21 02:51:37 alexmiller Or at least that's how I think about it
2016:09:21 02:51:42 bfabry oh, so you could use the symbol 'my-function in place of the spec? or at least (s/spec 'my-function)
2016:09:21 02:51:57 alexmiller Yes - fully qualified though
2016:09:21 13:31:28 kestrel7 @alexmiller some feedback. I created a generator that did not conform to the spec (doh!). The generator contained the such-that predicate. When I tried creating a sample from the generator I got this error:
ExceptionInfo Couldn't satisfy such-that predicate after 100 tries. clojure.core/ex-info (core.clj:4725)
I assumed that it referred to my custom generator but that was a red herring because in fact spec must be using such-that to ensure that the generated value conforms to the spec, and it was this such-that that generated the failure, not the one in my custom generator.
Code (with the problem corrected but showing the such-that in my generator:
(defn mod11-checkdigit
"Calculate the checkdigit see "
[n]
(let [x (->> (map #(Integer/parseInt (str %)) (take 9 n))
(map * (range 10 1 -1))
(reduce +))
y (mod x 11)
c (- 11 y)]
(cond (== 10 c) nil
(== 11 c) 0
:else c)))
(def nhs-number-gen
"Generates a valid NHS number"
(gen/fmap #(str (+ (* 10 %) (mod11-checkdigit (str %))))
(gen/such-that #(mod11-checkdigit (str %))
(gen/choose 100000000 999999999))))
(defn nhs-number?
"Returns true if passed a valid nhs number else returns false"
[n]
(and (string? n) (= 10 (count n)) (= (str (mod11-checkdigit n)) (str (last n)))))
(s/def ::nhs-number (s/with-gen nhs-number?
(fn [] nhs-number-gen)))
2016:09:21 13:33:35 kestrel7 It would be nicer if the error thrown due to the generated value being non-conformant with the spec stated this.
2016:09:21 13:46:42 patrkris Say I have a namespace with two functions create-component-a and create-component-b. Each function takes a map with options. How would I go about spec'ing those option maps in the same namespace as the functions? Let's say that each map takes an :id (or something else) attribute, but they're semantically different. If I want to use the ::keyword shortcut to create a namespaced keyword, I can't call them both ::id. I could create a namespace for each type of option map, alias them and use something like ::a-options/id and ::b-options/id, but that seems wrong.
2016:09:21 14:10:13 alexmiller @kestrel7 you are correct in your assessment - spec doesn’t trust custom generators and re-confirms they match the spec. Agreed that it would be nice to know the cause here more explicitly (kind of a special case of the general desire to know more about which part of a generator failed to generate). Happy to see a jira for enhancement.
2016:09:21 14:11:09 alexmiller @patrkris your last suggestion is what I would do there (you don’t have to alias of course - you can just use fully-qualified too)
2016:09:21 14:11:14 alexmiller why does that seem wrong?
2016:09:21 14:13:07 patrkris @alexmiller: I don't know. Maybe it shouldn't 🙂. It's probably just from being used to define a schema (with herbert) in the same namespace as the function relying on it. But using the fully qualified version also feels a little wrong, because then you have hardcoded the namespace. It possibly just something I need to get used to
2016:09:21 14:15:36 patrkris I thought about whether allowing the use of "subnamespaces" in aliases would be useful. For instance ::a-options/id without having an ::a-options alias. But I can imagine that it would turn out to be problematic.
2016:09:21 14:16:47 patrkris Ehr... by "subnamespaces" I meant being able to say ::order.delivery/type without having a order.delivery alias created
2016:09:21 14:19:28 alexmiller well, I don’t think we’ll do that :)
2016:09:21 14:21:45 patrkris But it's not definite no? 😉 I'm kiddin'
2016:09:21 15:21:26 kkruit Is there a way to require that in {:a "a" 😛 "b" :c "c"} :a is required and 😛 or :c is required and still have the other be optional? I've tried this using s/keys but cant figure out how i'd do it and I can't see how i could use s/map-of and still require :a...
2016:09:21 15:27:10 kkruit oh man! I tried s/or... Thanks!
2016:09:21 15:28:05 minimal the or is syntax in the keys macro rather than clojure.core/or or spec/or
2016:09:21 15:28:59 kkruit oh, I was just thinking. how is that working as clojure.core/or ... guess it's not haha
2016:09:21 15:29:07 kkruit thanks again!
2016:09:21 15:29:17 minimal I didn’t notice it at first either
2016:09:21 18:48:35 sparkofreason Some functions, like exercise, allow you to override generators by name. Is there any mechanism to do the same with conformers?
2016:09:21 18:49:07 leongrapenthin @alexmiller I think a good solution to aliasing keyword and symbol namespaces not existing in the codebase like :human.programmer/language would be (defalias hp human.programmer). So :hp would still be :hp, but ::hp/language would resolve to :human.programmer/language . This would overwrite aliases created via require. Should I write a tickent?
2016:09:21 18:49:46 alexmiller @dave.dixon no
2016:09:21 18:50:32 alexmiller @leongrapenthin alias already exists
2016:09:21 18:50:50 alexmiller the problem with it is that it requires the ns you are aliasing to exist
2016:09:21 18:51:12 leongrapenthin Right
2016:09:21 18:51:18 alexmiller we are already considering loosening that constraint or auto-creating the ns
2016:09:21 18:51:31 alexmiller I don’t need a ticket for it
2016:09:21 18:51:59 leongrapenthin interesting. auto-creating seems like a good strategy
2016:09:21 18:52:20 alexmiller that is by far easier given the existing implementation of Namespace etc
2016:09:21 18:52:27 leongrapenthin yeah
2016:09:21 18:52:56 leongrapenthin I just realized that I could use create-ns in combination with alias
2016:09:21 18:53:09 leongrapenthin So there is a good workaround
2016:09:21 18:53:17 alexmiller yes that works fine now
2016:09:21 18:54:08 leongrapenthin DK about Cljs though
2016:09:21 19:17:07 alexmiller have you looked at multi-spec?
2016:09:21 19:17:55 alexmiller has that kind of a feel to me
2016:09:21 19:58:44 manderson I keep running into the same problem where I have two maps that have the same named key, but with different specs. For example: two maps that both have a key called name, where in the first it is a string?, but the second is int? because it is more of an ID. How do I specify this so that the specs can be different, but the field name the same in both maps? Only solution I see is to change name to something else (like person-name) and just living with the name change, but that doesn't work especially if modeling a domain I don't control. Thoughts?
2016:09:21 20:11:33 bfabry @manderson :opt-un and :req-un in s/keys
2016:09:21 20:13:29 manderson Not sure I'm following... I still have to define a spec called ::name in two different ways. Yes, I could use or, but that muddies the definition since they are actually separate specs.
2016:09:21 20:17:39 bfabry @manderson sorry I should've been clearer, define a spec called :my.app.person/name and a spec called :my.app.service/name or whatever makes semantic sense
2016:09:21 20:20:02 manderson Ok, so using the namespacing. That makes sense and should work for most of my cases (often translating from a format like JSON where there aren't namespaces). I suppose in the case where namespaces are expected, it would probably already be modeled correctly from the beginning (hopefully). Thanks @bfabry
2016:09:21 20:20:52 bfabry no worries
2016:09:21 21:40:37 seancorfield @manderson To (hopefully) clarify: if you’re spec’ing with :req-un and :opt-un then you are using simple (non-namespaced) keys in your maps, but you can use a namespaced spec name / key name there to distinguish the two specs.
Both person and service specs are maps with a required :name key, but the specs used to validate those keys’ values will be different for ::person and ::service — does that help?
2016:09:21 21:42:50 seancorfield (it was something that I did not get on the first two or three reads of the Guide and it was Alex that set me straight on that)
2016:09:21 23:19:49 stuartmitchell Hi I am beginning a large multi-developer project that will use spec extensively I was wondering if there is any advice for how to place specs in namespaces.
I was thinking of having a namespace structure like
project-name.spec
-common
-thing1
-thing2
...
and then requiring these namespaces in the actual code, probably when doing s/fdef below the defn
this will require lots of map access to be ::thing1/id or ::thing1/date etc
is this a good idea or should I be trying to keep all specs within the namespaces they deal with so ::id can be used when possible
2016:09:21 23:48:03 hiredman tying your data model (names of attributes) to the way your code is currently organized seems like a bad idea
2016:09:21 23:49:30 hiredman I would recommend picking good names for things in your maps, and sure namespace them, but don't think you have to use the same namespace names as your code
2016:09:21 23:50:25 hiredman :: is a real pain when moving code around
2016:09:22 00:03:46 stuartmitchell so you are saying :thing1/id (non aliased namespace) used everywhere we were thinking that, but it doesn't seem to look like the examples (which I assume are best practice)
2016:09:22 00:14:04 stuartmitchell I guess you loose the uniqueness of the keywords though.
2016:09:22 00:14:18 hiredman what uniqueness?
2016:09:22 00:16:20 hiredman in five years on a large clojure project we had several shakeups where namespaces were changed or renamed in mass, a long with regular day to day code churn
2016:09:22 00:16:46 stuartmitchell for instance if i define :spec/thing1 in library code it could conflict with someone else defining :spec/thing1
2016:09:22 00:17:00 hiredman use a better namespace then
2016:09:22 00:17:18 hiredman my.orgs.spec/thing1
2016:09:22 00:18:33 stuartmitchell yeah good point, I was just pondering what the differences would be
2016:09:22 00:27:53 hiredman data is something that can be exchanged with other systems, stored long term, etc, so using names that reflect the organization of code in one codebase isn't the best
2016:09:22 12:50:39 jmglov @skjutare Hi!
2016:09:22 12:50:55 jmglov Just noticed you here. 😉
2016:09:22 12:58:50 jmglov I think I'm running into an infinite loop with a custom generator. I need an atom holding a map with specific keys, which seems pretty straightforward:
When I try to generate a value, my REPL becomes unresponsive for awhile and then eventually crashes with a really odd stacktrace:
user> (gen/generate (s/gen ::db-atom))
...
2016:09:22 13:03:43 jmglov Can anyone spot my error?
2016:09:22 13:36:29 manderson Thanks @seancorfield, just saw your reply. That does make sense. I think I need to rethink the way I organize my specs. Right now, I have everything in a single namespace and now I'm wondering if I should break these into sub-namespaces. I know that I can still collect my specs in 1 namespace and just add different namespaces to the keys. Not sure which is the best approach. How do you group your specs? I'm still trying to get a feel for the proper balance of the right amount of spec without becoming cumbersome.
2016:09:22 13:41:54 manderson Ha, funny. I just actually read back through the rest of the messages and see @hiredman making some good suggestions to my very question... sounds like best practice is to de-couple top level namespace of a file from namespace of a keyword. So, all could be grouped in a file, but with explicit namespaces for keys instead of ::.
2016:09:22 13:50:00 patrkris @manderson: @hiredman talked about specs for data that might be exchanged between systems. If you want to spec data used only inside your system, creating namespaces for that may be feasible. For example, if you have a component in a com.example.payments-processor with a factory function that takes an options map, you could potentially spec it inside com.example.payments-processor.options, and alias that namespace wherever you need to create the options map. Hope I make sense.
2016:09:22 13:52:55 manderson Yep, that makes sense. I suppose there is no one right answer. Like I said, I'm still getting the feel for what works and what doesn't. We're building our first real app with this (as I imagine most are!) and running into pain points with name conflicts and a large spec file, so I've been thinking of how to best maintain it.
2016:09:22 13:55:29 patrkris Yes. I have been thinking a lot about this too. The problem is that you can't use aliased namespaces in specs if there isn't actually a file containing that namespace. In those cases you need to spell out the whole namespace of the spec, which can be verbose.
2016:09:22 13:56:37 manderson Hm. that's a good point.
2016:09:22 13:59:47 manderson It's kind of similar to the question of lots of fn's in a single namespace so only 1 to require, or fn's spread over ns's, which is more modular, but adds complexity. With spec, you would have to keep up with more ns's, but then you could alias them for simpler inline usage... It seems to encourage simplifying your model domain as much as possible in terms of defining it, which is probably a good thing.
2016:09:22 16:12:49 seancorfield There are workarounds for the alias issue that let you introduce aliases for non-existent namespaces.
2016:09:22 16:15:34 seancorfield @manderson: we group our data specs into files organized for what they spec, primarily, with almost no code in those files. Then we require them into namespaces that use those specs (i.e., where the code / functions are).
2016:09:22 16:16:41 seancorfield That's probably not a very helpful answer without you knowing a lot more about our codebase 😸
2016:09:22 17:03:59 manderson Thanks @seancorfield, that's similar to what we're doing now, but it's a single spec file for a new app with a changing domain so it's been a bit of a pain. I think it's just going to take some trial and error to work out the kinks.
2016:09:22 17:13:22 seancorfield Yeah, we’re still evolving our approach. One of the issues for us, particularly, is that our domain model spec is the most extensive part and we’re working to automate the validation and conformance from our API spec to our domain model spec, so we’re moving toward dynamic derivation of API specs from domain model specs, which I hadn’t considered at first… so that’s changing the dependencies between our spec namespaces as well as how we use the specs in the first place. And we haven’t even gotten very far into spec’ing functions at this point (spec’ing the domain model and doing data validation and transformation has proved much more valuable for our initial focus).
2016:09:22 17:21:36 kkruit I've noticed that, for specs that are specific to your application, clojure src is using them in a sub-namespace e.g. all the specs for clojure.core are in clojure.core.specs. This has seemed to work okay for me.
2016:09:22 17:25:30 kkruit @patrkris I've been using create-ns to make namespaces for that issue.
2016:09:22 17:28:25 seancorfield That’s nicer than the workaround I was using, from Alex. I think I’ll switch to that @kkruit — thanks!
2016:09:22 19:03:22 mpenet it's just a wrapper for create-ns+alias
2016:09:22 19:12:12 seancorfield Ah, from another project...
2016:09:22 19:13:04 mpenet it's a 3 line fn, as i said, create-ns + alias
2016:09:22 19:13:42 seancorfield Since your alias is so long, it took me a while to realize it even was an alias… and the args are the opposite way round to alias which made it look even stranger:
2016:09:22 19:14:24 seancorfield Given that alias will probably be modified to auto-create the ns, I’m preferring alias directly since I can probably just remove the create-ns call at some point.
2016:09:22 19:14:52 mpenet you have a point
2016:09:22 19:16:35 mpenet I just wanted to get rid of "qbits.alia" if I recall, since I used this single file for the whole project I still like to keep some ns'ing
2016:09:22 19:46:07 nicola We had discussion on clojure-russia hangout about clojure.spec & namespaced keys and there is an idea by @prepor: we have reader macro to do ::key => :my.ns/key , we have sugar to do (require ... :as a) ::a/key => :my.required.ns/key. But there are a lot of cases where we need something like ::.subns/key => my.ns.subns/key. For real world deeply nested data-structures create a lot of namespaces just for fully qualified keys for spec looks like overhead...
2016:09:22 21:12:33 patrkris @seancorfield: what was that workaround you mentioned earlier?
> That’s nicer than the workaround I was using, from Alex. I think I’ll switch to that @kkruit — thanks!
2016:09:22 23:55:36 noprompt has anyone been able to bend spec to parse and conform strings?
2016:09:23 00:03:38 devn @noprompt: what mad science are you up to now? :)
2016:09:23 00:09:18 noprompt @devn my problem is that i want to conform nested string data so i don't need to do it in two steps. it's possible to use conformer but when there's a problem the error message is really, really bad (and there's nothing available in the public api for custom explanations). i'd like to simply write a custom spec against the protocols but alexmiller, in a previous discussion, has cautioned against that.
2016:09:23 00:10:48 noprompt basically i'm at a crossroads where it's either write the data portion of the parser by hand, forgoing spec, or bend spec to do my bidding.
2016:09:23 00:56:02 kenny How do you spec a function with optional args that should be conformed to a map?
(defn foo
[& {::keys [bar]}]
;...
)
2016:09:23 00:59:16 kidpollo hmmm sounds interesting and actually useful @noprompt It might actually also help my use case
2016:09:23 01:00:17 kidpollo when an input param for example is a string but the string contains an elastic search query
2016:09:23 01:00:49 kidpollo an extra ste would need to be made to validate the syntax of the string
2016:09:23 01:05:03 kenny This is a more concrete example:
2016:09:23 01:09:49 bfabry I knew I'd seen a specific function for it but it took me ages to find that again
2016:09:23 01:42:33 alexmiller @noprompt I will stick with my caution in trying to use spec regex to parse strings - that's not what it's designed for and I a normal regex is going to be a better choice. Why cant you use a regex predicate?
2016:09:23 14:18:46 djpowell what is the recommended way to make a spec for a constant string?
2016:09:23 14:26:07 djpowell ah yeah, a set seems to work ok
2016:09:23 15:15:16 alexmiller yes, a set #{“foo”}
2016:09:23 17:10:13 misha good day!
is there a way to spec a function, which I expect to get as an argument?
like "I expect function of 2 arguments: int and string"
2016:09:23 17:12:09 misha so far I see only s/fdef, which expects a symbol identical to existing function name.
2016:09:23 17:13:58 misha s/fspec!
2016:09:23 17:24:57 bfabry ^ yes 🙂
2016:09:23 17:26:41 bfabry also, something which I think is a bit not clear from the docs, fdef adds an fspec to the registry under the fully qualified name of the function, so if you've written an fdef for foo, you can use the symbol `foo in place of a spec
2016:09:23 18:07:00 spieden anyone know how to spec a map by predicates over its keys and values? really expected this to work:
2016:09:23 18:13:26 spieden that’s what i’m going with. thanks
2016:09:23 21:11:12 noprompt @alexmiller i can't use regex in my case because the strings i'm parsing are context-free.
2016:09:23 21:12:08 alexmiller so use a parser like instaparse?
2016:09:23 21:13:36 noprompt @alexmiller i probably would but unfortunately the story for clojurescript isn't solidified yet on that account.
2016:09:23 21:15:04 alexmiller @bfabry I would not use s/+ for that example above but (s/every (s/tuple string? keyword?)) - that’s basically (minus things like :kind and :into) what map-of translates to
2016:09:23 21:15:09 noprompt even still, the story for plugging into explainability with conformer is just not good enough. yes, the result does tell what string is wrong but i'm not able to say where in the string something is wrong.
2016:09:23 21:15:51 noprompt to be sure, i'm not complaining about spec. it's already saved me a huge amount of work. if i could tap into explainability it'd be a closed case for me.
2016:09:23 21:16:41 alexmiller not sure what to tell ya - you’re asking for things spec is unlikely to provide ¯\(ツ)/¯
2016:09:23 21:17:02 noprompt why wouldn't spec want to expose hooks to explainability?
2016:09:23 21:17:29 alexmiller because one of the ideas in spec is to make explainability generic
2016:09:23 21:18:51 noprompt that presupposes that such an idea is possible.
2016:09:23 21:18:55 alexmiller I haven’t talked with Rich about this specifically so this is just my read on it
2016:09:23 21:19:17 alexmiller when you get to a predicate the explanation is: the predicate failed
2016:09:23 21:19:47 alexmiller if you want more detail, make finer-grained predicates and combine them
2016:09:23 21:20:26 alexmiller if you want that + some sort of string parsing, I think you’re into the territory of custom specs
2016:09:23 21:20:30 noprompt right, however, sometimes the context is a little more granular e.g. failure alone is not enough for reporting. sometimes there's additional context for why the predicate failed.
2016:09:23 21:20:57 noprompt can we get a promise to stabilize the protocols? 🙂
2016:09:23 21:21:03 alexmiller now? no
2016:09:23 21:21:04 noprompt hahah
2016:09:23 21:21:14 alexmiller certainly not in alpha
2016:09:23 21:21:15 noprompt i meant when 1.9 is stable.
2016:09:23 21:21:38 noprompt honestly, that would be phenomenal for my use case.
2016:09:23 21:21:44 alexmiller I don’t have an answer on that
2016:09:23 21:22:45 alexmiller we can provide guidance when things move past alpha
2016:09:23 21:22:49 noprompt chances are most folks won't be interested in implementing the protocols and will just use "stock" spec. a handful of people, like myself, would probably greatly appreciate and benefit from the protocols being stable.
2016:09:23 21:23:02 noprompt awesome.
2016:09:23 21:23:15 noprompt would this conversation be better on the dev mailing list?
2016:09:23 21:23:42 alexmiller doesn’t make any difference to me
2016:09:23 21:23:55 noprompt fair enough.
2016:09:23 21:24:05 alexmiller I read both :)
2016:09:24 00:35:23 seancorfield @alexmiller I was just looking at https://github.com/clojure/clojure/commit/7ff4c70b13d29cf0a1fc7f19ba53413a38bb03d5 and noticed nonconforming which I had not previously seen in the API… and now it’s deliberately undocumented… I have several places where I have to conform with or and then use (conformer second) to "undo" the conformed result to get back what I need. Is there any likelihood that nonconforming would be "restored" to the documented API as something usable by developers? Or was it always intended to be an internal implementation detail and it just accidentally leaked?
2016:09:24 00:36:13 alexmiller It was accidentally public and stands a good chance of being removed
2016:09:24 00:36:18 seancorfield (I note that there actually is a test added in that commit for the nonconforming function, even tho’ that commit also removes it from the documentation)
it would generate a map with multiple keys… but when used in prop/for-all always just a single key-value?
2016:09:24 00:37:39 seancorfield Do you think "flattening" out an or is a good use case for having something like nonconforming tho’?
2016:09:24 00:38:05 alexmiller But now that nilable doesn't use nonconforming Rich is considering removing it
2016:09:24 00:38:34 alexmiller You can still easily do that with a conformer on val
2016:09:24 00:38:57 seancorfield True, for a simple spec. It gets harder with more complex specs.
2016:09:24 00:39:11 alexmiller Well, rely on it at your peril :)
2016:09:24 00:40:13 seancorfield So far that’s the only case I’ve run into tho’, where I need s/or in order to combine specs but I really don’t want that branch in the result. (s/and (s/or :a ::some-pred :b ::other-pred) (conformer second)) seems a bit of a mouthful 😸
2016:09:24 00:41:35 alexmiller @ag test.check is a bit different from spec in that spec wraps generators in a no-arg function - maybe that's causing the disconnect?
2016:09:24 00:42:46 alexmiller @seancorfield: advice from us is: use conformers very sparingly
2016:09:24 00:47:27 ag hmm… I still have no clue how to deal with that.
2016:09:24 00:47:39 ag wrap into something like gen/fmap?
2016:09:24 00:50:31 ag using overrides? second param of s/gen ?
2016:09:24 00:51:54 alexmiller I don't totally understand what you're trying to do
2016:09:24 00:53:25 ag I have a spec that generates a map (with gen/map), when used in gen/generate - it generates maps with multiple keys. but when used with test-check’s prop-all and quick-check it always generates maps with single key/value pair
2016:09:24 00:57:23 alexmiller Hmm, not sure
2016:09:24 01:03:09 seancorfield @alexmiller My use case — and it’s cropped up in a few specs I’ve needed at work — is that we have situations where we have some data that is either an X or a string that converts to an X (partly because of legacy constraints) but we want to conform the data to X in both cases and don’t care whether it was initially an X or a string.
2016:09:24 01:05:40 seancorfield Now, you can certainly argue "Don’t do that!" but having it baked into the spec and being able to just (s/conform ::my-spec data) is nicer than having (if (string? data) …) everywhere that uses it.
2016:09:24 06:14:07 plexus is this a bug or a feature?
2016:09:24 08:24:06 seancorfield It's a feature. :clojure.spec/invalid is special -- it only ever satisfies s/invalid?.
2016:09:24 08:56:08 mpenet Odd, seems like an impl. quirk
2016:09:24 16:17:47 borkdude It would be nice if t/instrument also gave a warning when the :ret spec doesn’t conform. I know spec only tests this with t/check, but is there a reason the first one isn’t in spec (optionally) right now? Seems useful for dev purposes.
2016:09:24 16:25:39 borkdude If I could programmatically insert a post-condition on a function that would get me far
2016:09:24 20:03:40 alexmiller @borkdude: you can use s/assert to state postconditions if you like
2016:09:24 20:04:39 alexmiller They can then be turned off and even optionally compiled out
2016:09:24 20:22:08 borkdude @alexmiller ah yes, like in the snippet
2016:09:24 21:18:57 jfntn If I have defined a spec for a value like :datomic/tempid, how can I reuse that in a map spec that’d say that the :db/id key should have a value that conforms to :datomic/tempid?
2016:09:24 21:54:10 jfntn I think I got something usable! The point is that I want to define a map spec for an entity, and reuse that spec in describing the inputs to a tx and the outputs from a read. The trick is that the read and write specs should be identical with the exception that :db/id takes on different values
2016:09:24 21:55:57 jfntn Not sure that’s a good way to do this, would appreciate some feedback
2016:09:24 22:58:32 misha (I want to register a function spec to a keyword, to be able to "say" later:
this map contains :foo/bar key, and its value is a function of 2 args: number and keyword)
2016:09:24 22:59:22 misha or just to conform any other function to this function signature
2016:09:25 05:50:25 olivergeorge Perhaps my use case is special but it seems like something which might be useful to others.
2016:09:25 05:55:20 olivergeorge The other fiddly bit was working with the function sym. I needed to resolve the sym to the current local var value but also keep the sym to call (s/get-spec ...). In the end I used (var). Not sure if there's downsides to that I'm missing.
2016:09:25 10:43:26 olivergeorge I think that was my inspiration. I've added a bit more finessing about showing interesting random data (overrides) and controls (seed & size) which might make for a more useful UI dev tool.
2016:09:25 10:46:23 olivergeorge Needs some helper generators. I never leave home without my lorem ipsum generator.
2016:09:26 14:22:31 kestrel7 Is it possible to define a spec that depends on multiple keys within a map? e.g. ::date-1 must be greater that ::date-2
2016:09:26 14:45:53 bcbradley i'm having issues with setting up clojure.spec
2016:09:26 14:47:02 bcbradley i'm getting "could not locate clojure/test/check/generators__init.class or clojure/test/check/generators.clj on class path"
2016:09:26 14:47:04 bcbradley what does that mean?
2016:09:26 15:02:10 thegeez If you are using Leiningen, you can try to put the dependency at the top level. So put it next to [org.clojure/clojure ...] in :dependencies
2016:09:26 15:02:47 bcbradley you mean don't use a dev profile?
2016:09:26 15:03:24 misha @bcbradley clj or cljs ?
2016:09:26 15:03:30 bcbradley clojure
2016:09:26 15:03:55 bcbradley the function causing the issue is exercise-fn
2016:09:26 15:03:55 thegeez Yeah, just to check if that works. You might be using the test.check function in a way that does not fall under leiningen dev profile
2016:09:26 15:04:19 thegeez or in a place, rather than a way
2016:09:26 15:05:32 bcbradley i can use clojure.spec/fdef
2016:09:26 15:05:44 bcbradley but can't use clojure.spec/exercise-fn
2016:09:26 15:15:52 thegeez hmm curious, it works on my machine 😉
2016:09:26 15:17:16 bcbradley here is what i'm looking at
2016:09:26 15:22:43 seancorfield @bcbradley You’re AOT’ing that namespace so you’re forcing all those specs — including exercise-fn — to run at compile time.
2016:09:26 15:23:10 bcbradley i see
2016:09:26 15:23:17 bcbradley i'm sorry for being so new at this 😞
2016:09:26 15:23:30 bcbradley the AOT is an acronym i assume
2016:09:26 15:23:34 bcbradley what does it stand for?
2016:09:26 15:23:50 seancorfield Ahead Of Time (compilation) — relating to the :aot in your project.clj
2016:09:26 15:25:31 seancorfield Still, having the exercise-fn call at the top-level of your code means it will be executed whenever that ns is run — put it inside -main so it only runs when you run the -main...
2016:09:26 15:25:44 seancorfield Same with generate.
2016:09:26 15:25:54 bcbradley ok, i was wondering though
2016:09:26 15:26:10 bcbradley if i wanted to make a library how would i test the code? Would i need a main?
2016:09:26 15:26:43 seancorfield You’d normally put test code in a separate ns and have it run via one of the test runners.
2016:09:26 15:27:05 seancorfield In your project tree, you’ll see a test folder next to src.
2016:09:26 15:27:26 seancorfield Hmm, no, in your case you won’t — did you create that project manually?
2016:09:26 15:28:19 seancorfield (I’m not familiar with how NightCode creates projects… but normally lein new app myapp will create a test folder as well as a src folder)
2016:09:26 15:28:38 bcbradley i just chose "create new console project"
2016:09:26 15:28:41 bcbradley i've used lein before
2016:09:26 15:28:49 bcbradley but importing a lein project into nightcode gave me some problems
2016:09:26 15:29:08 seancorfield I’m surprised (and a bit disappointed) that NightCode doesn’t create a test folder tree for all projects 😞
2016:09:26 15:29:13 thegeez I think this is an issue with nightcode. A top level exercise-fn with :gen-class and aot would works
2016:09:26 15:29:24 bcbradley i've had trouble finding a good ide for clojure that i like
2016:09:26 15:30:31 seancorfield @thegeez Well, if test.check is part of the regular :dependencies yes, but you don’t want that code running at compile time or ns load time...
2016:09:26 15:30:38 bcbradley i don't wanna use emacs because its 2016, i don't like eclipse because its sluggish and smells of enterprise, i don't like intellij + cursive because i don't want to pay for it, I don't like netbeans because netbeans. Light table isn't working with clojure 1.9-alpha12
2016:09:26 15:30:50 bcbradley it looks like nightcode is my only choice unless i want to rock it in vim
2016:09:26 15:31:23 seancorfield The majority of Clojure devs use a form of Emacs according to the annual "State of Clojure" surveys 🙂
2016:09:26 15:31:44 bcbradley i'd prefer not to develop carpal tunnel before i turn 30
2016:09:26 15:32:32 bcbradley jabbing aside, even after removing the AOT and pulling it into main i get some issues
2016:09:26 15:34:05 thegeez @seancorfield yes that setup is not ideal, but more meant to isolate the problem
2016:09:26 15:35:56 thegeez @bcbradley now you have the name my-index-of and a spec for my-index-of2
2016:09:26 15:36:07 seancorfield @bcbradley Your spec has my-index-of2 but the function … yeah, what @thegeez said 🙂
2016:09:26 15:36:20 bcbradley you are right!
2016:09:26 15:36:23 bcbradley that fixed it xd
2016:09:26 15:37:02 bcbradley i know i'm probably being a pest, but can you explain the difference between clojure.spec and any normal library like clojure.set?
2016:09:26 15:37:13 bcbradley i mean in terms of pitfalls,
2016:09:26 15:37:31 bcbradley like for instance, i didn't know that i "shouldn't" invoke certain methods or macros at the top level
2016:09:26 15:37:48 bcbradley normally i wouldn't invoke anything at the top level in an application, but i figure heck
2016:09:26 15:37:51 bcbradley if it works on a repl
2016:09:26 15:37:56 bcbradley i'm just trying to learn
2016:09:26 15:38:00 bcbradley i need to know what NOT to do
2016:09:26 15:44:19 seancorfield In general you don’t want any executable code at the top-level of your namespace.
2016:09:26 15:46:04 seancorfield What’s your language background, prior to Clojure? (so we’ll have a frame of reference for what you’re familiar with)
2016:09:26 15:48:33 seancorfield And regarding the difference between clojure.spec and "normal" libraries, probably the big one is that there’s a bunch of "testing" stuff in clojure.spec that belongs in test code, where test.check gets brought in via a :dev profile (for REPL and running tests, as opposed to "production" line code). Does that help?
2016:09:26 15:49:42 seancorfield The specs themselves — and valid? and conform — are intended for production code, but the generative testing stuff (`generate`, exercise, check) is intended for either interactive use or for running as part of test code.
2016:09:26 16:02:50 bcbradley sorry i was away, my language background: C, C++, Java, Python, Javascript and then Clojure
2016:09:26 16:03:11 bcbradley i've got 8 years in C, 6 years in C++, 3 years in python, 2 years in javascript and 1 year in clojure
2016:09:26 16:04:25 bcbradley i'm in my senior year for software engineering and would like to get a job where the primary language is clojure, or at the very least some other functional oriented job with scala or erlang or whatever
2016:09:26 16:04:50 bcbradley i've seen enough of stateful programs to know they are a pitfall
2016:09:26 16:04:57 seancorfield 😄
2016:09:26 16:04:59 bcbradley and i'm not just espousing things i've heard
2016:09:26 16:05:06 bcbradley i've had time to think about it for myself
2016:09:26 16:06:06 bcbradley my experience with clojure is probaby summarized best as follows:
2016:09:26 16:06:30 bcbradley i know how to use map, reduce, apply partial blah blah blah, i've been doing it for a while
2016:09:26 16:06:41 bcbradley when i moved from javascript to clojure i did it for elegance
2016:09:26 16:06:41 seancorfield 'k so a Clojure namespace produces a class, and all the top-level forms become static initializers in that class… if you think of it in Java / C++ terms, those static initializers get run when a class is loaded, long before any class instance is created… so the same caveats apply.
2016:09:26 16:07:27 bcbradley ah i see
2016:09:26 16:07:55 bcbradley i don't think i'm actually familiar with static initializers
2016:09:26 16:08:32 bcbradley well a search on stack overflow was fast enough
2016:09:26 16:08:40 bcbradley good to know thats how clojure namespaces work
2016:09:26 16:10:58 bcbradley so basically what you are saying is that if I have a bunch of top level forms that depend on some other class already being loaded, but it hasn't been loaded, i'll get a "class nil" error or some such
2016:09:26 16:11:02 bcbradley that makes perfect sense
2016:09:26 16:50:43 bfabry @bcbradley cursive is free for non-commercial use, fyi
2016:09:26 16:52:12 bfabry and in a commercial setting, I can't imagine any company would care about the cost
2016:09:26 17:05:01 bfabry it's also rapidly catching up with the the perpetual yak emacs 😛
2016:09:26 17:20:55 seancorfield Some of us really like yaks...
2016:09:26 17:33:22 dominicm My vim shaving is unmatched in fun vim
2016:09:27 14:53:17 alexmiller this allows you to specify any number of kwarg options at the end in any order
2016:09:27 15:42:20 madstap Right, I had missed s/keys*, thanks for the clarification. Sometimes it seems like you guys think of everything, probably all that hammock time.
2016:09:27 15:45:12 kurt-yagram ah, ok... thanks!
2016:09:28 00:07:25 jfntn I can’t find a good solution to write different specs that all have a :db/id key but with different values
2016:09:28 00:08:04 danielcompton @jfntn what do you mean by ‘different values’?
2016:09:28 00:09:12 jfntn Well one would be say a ::temp-entity spec that’d check that :db/id is a datomic/tempid, and the other would be an ::entity spec that would check that :db/id is an entity id (long)
2016:09:28 00:10:11 jfntn So I need to attach multiple specs to the same namespaced key, but s/or won’t do since the point is to be able to assert whether an entity is temporary or not...
2016:09:28 00:11:33 jfntn I found a weird trick using multiple slashes in the spec name and using :req-un in the spec, but those don’t seem to really be supported since I’m getting intermitent analyzer errors and the keys won’t compile in cljs
2016:09:28 00:15:04 bhagany I think in this case, I would use s/or, and use the conformed value to say whether it’s a temp id or not
2016:09:28 00:16:00 bhagany it’s an extra step, but I don’t see another way to make it work
2016:09:28 00:18:38 bhagany if you’re spec-ing a function that needs to take a temp entity, or something like that, you could always make a predicate function, temp-entity? or some such, that conforms its argument and checks which branch s/or took
2016:09:28 00:32:17 jfntn @bhagany that’s a good idea!
2016:09:28 00:33:14 jfntn Not as direct, but I’m able to express what I wanted, which is a reusable keys spec and combine it with the temp and final specs
2016:09:28 06:42:42 przemek Hi, while trying to migrate from schema to spec I've come across an issue. Following the guide I was able to define specs for maps, but I think I have a case that is not covered. Basically the same key i.e. :id or :name may be an attribute of different entities. If I define ::name then the only way to use a different type definition for :name is to use namespaces which puts me in sort of OO style when I keep module per "noun".... Is there not a way to decouple key from value so that I can define i.e. ::plan-name type, but use :name in my map?
2016:09:28 06:50:34 misha @przemek what about (s/keys :req-un [...] :opt-un [...]?
2016:09:28 06:55:21 misha basically, you'd have multiple namespaced specs for name: :foo/name, :bar/name; and then use them in map specs, but inside :req-un (or :opt-un), with the only requirement of keeping (name kw) identical
2016:09:28 07:02:47 przemek hell yes 🙂 That looks like what I was looking for. I makes sense to document more how namespacing looks in action. I am gonna try this one in a moment. thanks
2016:09:28 07:11:58 przemek it works. I guess for Datomic that :xxx/yy pattern is heavily used (I have not used it though). thanks again!
2016:09:28 07:53:38 jmglov Or is this something better posted to the mailing list?
2016:09:28 08:22:11 przemek Another question. Given this definition (s/def ::email (s/and string? #(re-matches email-regex %))) I am not able to generate a sample getting "Couldn't satisfy such-that predicate after 100 tries".
2016:09:28 08:25:47 przemek Ok. Solved. Found a paragraph in the guide on custom generators.
I would like to declare an event that has fixed ::type:
(s/def ::command-event (s/merge ::event-common -- what do I put here? --))
2016:09:29 01:53:21 richiardiandrea (I was also wondering if I can use s/and with my ::event-common def)
2016:09:29 03:59:42 alexmiller @richiardiandrea not entirely clear what you want to do. maybe:
(s/def ::command-event (s/merge ::event-common (s/keys :req [::type]))) ?
2016:09:29 04:01:29 alexmiller or are you asking because they can’t both be satisfied that way? I suspect this is the kind of thing where you’re happier using s/multi-spec
2016:09:29 04:12:17 richiardiandrea @alexmiller tnx for answering, I have a multi-spec in place so that I can check if the event conforms to my expected :types. Re-reading my question I see that maybe I am looking at it from the wrong angle..I will clarify my own ideas first and ask again
2016:09:29 04:17:23 richiardiandrea I think I wanted to be sure that a function returns only a certain :type of events. So I need to combine the ::common-event spec with a predicate...probably an s/and will suffice.
2016:09:29 04:29:35 alexmiller Yeah s/and is fine
2016:09:29 06:32:27 seancorfield So I still keep running into situations where I have a spec for a domain level entity that is, say, a set of keywords and then the spec for the input level entity is a set of strings that are the names of the keywords... so my gut says to spec the input as (s/and string? (s/conformer (comp keyword str/lower-case)) ::domain-entity) with a generator of (fn [] (g/fmap name (s/gen ::domain-entity)))
2016:09:29 06:33:05 seancorfield But @alexmiller has said to avoid s/conformer so I suspect this is an anti-pattern, even tho' it's cropping up over and over again.
2016:09:29 06:35:07 seancorfield The other (anti-)pattern I hit is a domain spec for an int (int-in or whatever) and the input spec is string? and parse to long and the domain spec, again with a generator of fmap str over the domain spec generator...
2016:09:29 07:26:23 mpenet Thinking about using a set of specs just for json/http params coercion here, then pass the result to domain specs, I think I prefer to keep both clearly separate.
2016:09:29 07:28:25 mpenet makes it a bit easier to reuse also maybe
2016:09:29 07:29:05 mpenet anyway, still have a ton of Schema code in our codebases, and still undecided on if/how to port it so far.
2016:09:29 11:03:10 mpenet is there an equivalent of describe/form that will resolve specs recursively?
2016:09:29 11:04:41 mpenet I mean resolve specs from the registry when it's composed from these
2016:09:29 11:05:41 mpenet -> apparently not
2016:09:29 11:33:02 mpenet (probably another anti-)pattern I encountered: setting a default value if nil when conforming, it's useful at input level too
2016:09:29 11:53:30 mpenet another annoying quirk, the following won't compile on alpha13:
(defn foo [x] :clojure.spec/invalid)
ExceptionInfo Call to clojure.core/defn did not conform to spec:
:clojure.spec/args (foo [x] :clojure.spec/invalid)
clojure.core/ex-info (core.clj:4725)
2016:09:29 11:58:55 mpenet makes it impossible to write a declared conformer ...
2016:09:29 12:01:11 mpenet well you can create an alias first (def invalid :clojure.spec/invalid) and use it instead of :clojure.spec/invalid in the code, but that's kind of ugly
2016:09:29 16:49:48 jeroenvandijk Question, do people add docs to specs? I’m making specs for some configuration data and I want to add meta data/documentation to individual specs to give more background information about the specific options
2016:09:29 18:18:15 seancorfield I don’t think you can add docs to specs…?
2016:09:29 22:28:03 jrheard first time spec user here, trying to wrap my head around the best way of using namespaced keywords, never used ‘em before
what’s the more correct pattern: having my keywords have namespaces that match my project’s namespaces, like :myapp.system.collision/foo?
or having my keywords have namespaces that correspond to arbitrary concepts, unrelated to my app’s file/namespace structure, like :collision/foo?
2016:09:29 22:38:26 jrheard so far i’m going with the latter, and it feels pretty good, but i’d just like to confirm with the experts 🙂
2016:09:29 22:48:48 jrheard also - do you folks have any favorite example github repos that use spec, like, completely idiomatically? i’m switching my project over from schema, and some parts of the transition are straightforward, but other parts just feel like they could be done in a variety of different ways and i’d love to have a few canonical examples to look at and learn from
2016:09:29 22:52:46 bfabry it's probably too early for that to exist =/
2016:09:29 22:53:09 bfabry fwiw I'm leaning toward the latter of your two options as wel
2016:09:29 23:08:25 seancorfield @jrheard We’re creating new namespaces specifically for the data structure specs, and then requiring them into code namespaces as we need them. In a few places we have both data/specs and code. It’s whatever seems most appropriate.
2016:09:29 23:10:40 seancorfield (if you click that link, you’ll see two more follow-on comments)
2016:09:30 01:20:51 ag hey guys… anyone here using spec and prop. based testing in clojuresript? I have encountered strange problem. (s/gen) within a test that works perfectly on my machine, on CI machine (same java, lein, node and phantomjs versions) says ""Couldn't satisfy such-that predicate after 100 tries.”,
2016:09:30 01:21:23 ag is it possible that a certain spec would work on one machine and fail on other?
2016:09:30 01:22:01 ag it’s not like random. I run it many times. it passes on my machine and it ALWAYS fails on CI machine
2016:09:30 01:22:19 ag other tests work fine though
2016:09:30 01:22:24 ag ¯\(ツ)/¯
2016:09:30 01:56:29 danielcompton what kind of spec is it?
2016:09:30 02:15:23 ag oh… oh… it’s a bit nasty… I need to generate iso-date strings and I did this:
2016:09:30 02:38:47 bfabry @ag my first guess would be a different timezone setting between CI and your pc
2016:09:30 06:26:33 danielstockton I know this has probably been asked 1000 times but what is the prevailing preference for where to put specs? In their own parallel namespace or inline?
2016:09:30 06:27:19 stathissideris my impression is inline for function specs, different namespace for “domain” specs
2016:09:30 06:27:55 seancorfield It depends.
2016:09:30 06:29:07 seancorfield But yeah, I think @stathissideris comment reflects what is becoming the most likely best practice at this point. It's what we're doing at World Singles so far...
2016:09:30 06:29:41 danielstockton and "domain" spec means specs which are likely to impact more than one namespace and be reused a lot?
2016:09:30 06:30:04 seancorfield Data structure specs, as opposed to function specs.
2016:09:30 06:36:20 danielstockton Know of any open source projects that are already using it, that I could take a look at?
2016:09:30 06:42:49 danielstockton Thanks. Looks like they follow the some.namespace and some.namespace.spec convention instead
2016:09:30 06:43:40 danielstockton I was thinking an advantage of in-lining is that it provides some documentation, but am I right that spec adds something to the docstring anyway?
2016:09:30 06:45:03 danielstockton s/they/yourself/ i see
2016:09:30 06:45:10 bfabry fdef specs are added to the output of doc ya
2016:09:30 06:46:26 danielstockton Ok, I might be leaning towards a separate namespace in that case
2016:09:30 06:47:03 seancorfield Inlining fdef makes sense for documentation, but not for portability across versions. def for data structures in a separate namespace makes sense regardless.
2016:09:30 06:48:00 seancorfield We've generally found if we start to def a data structure spec in a namespace with functions, it's gets messy ... keeping the data structure specs separate is more flexible...
2016:09:30 06:50:27 danielstockton I'm worried things will be messy with function specs too, it seems the documentation advantage is somewhat negated by the docstring
2016:09:30 06:55:35 seancorfield Not sure what you mean...? Specs just add to the docstring...
2016:09:30 07:08:34 danielstockton I mean that if it's added to the docstring then it isn't much of an advantage having it inline as well.
2016:09:30 07:09:31 seancorfield Right, yes, but the docstring is only updated if you've loaded the ns with the spec so you might as well have the fdef inline (IMO)
2016:09:30 07:10:37 bfabry yeah.. different documentation requirements. documentation for people reading/maintaining the code, documentation for people using the function
2016:09:30 07:20:36 yenda Hi, I am trying to use spec to conform the arguments of a function: there is at most 4 arguments with 4 distinct types. I want to conform them into a map but the solution I came up with is not fully compliant because it allows 2 arguments of the same type. Is there a way to make sure there isn't 2 arguments of the same type ?
2016:09:30 07:20:59 seancorfield Not sure I follow... can you share some code?
2016:09:30 07:21:18 yenda @seancorfield yes sorry you are too quick 😄
2016:09:30 07:21:23 seancorfield 🙂
2016:09:30 07:27:29 seancorfield Your coll-of is a map so each :message item overwrites any prior item before :distinct can be checked.
2016:09:30 07:28:09 seancorfield And if you used :into [] you would still accept those two :message arguments because they are different values so they are distinct.
2016:09:30 07:28:47 seancorfield The :distinct flag says that all the conformed values must be distinct -- which they are.
2016:09:30 07:29:22 seancorfield You'd need your ::argument spec to conform to just the type, not the type and value.
2016:09:30 07:34:38 seancorfield Hmm, that doesn't quite work either:
2016:09:30 07:35:12 yenda @seancorfield if I have 2 identical arguments the predicate distinct? is triggered so the distinct check happens before the into {}
2016:09:30 07:36:10 yenda however this not really what I want I could remove it, what I would like is to have distinct types for the arguments not distinct arguments
2016:09:30 07:37:33 seancorfield Yeah, I think you're right about the :distinct check... it would check distinct values before the conforming.
2016:09:30 07:37:48 bfabry there's something from a very long time ago whispering in my ear that this is an irregular grammar, and so you're going to need a custom predicate to validate it. but I could be wrong
2016:09:30 07:41:43 seancorfield So each argument needs to conform to its type (throwing away the value) and then the arguments collection needs to flow the conformed arguments into a distinct collection
2016:09:30 07:43:13 yenda wow thank you it works but it also makes me realize I still have a lot of work to do on spec 🙂
2016:09:30 07:48:38 seancorfield Yeah, it's radically changing how we approach several types of problems.
2016:09:30 07:49:24 yenda oh ok I understand now because I didn't realize that I don't get the values anymore while conforming
2016:09:30 07:51:45 yenda the function I don't get is conformer
2016:09:30 07:55:27 seancorfield Took me a while too. conformer uses its function argument to transform the data rather than just being a true/false predicate.
2016:09:30 07:55:59 seancorfield But Alex says it's an anti-pattern so be cautious.
2016:09:30 08:20:35 yenda well I'll play a bit to see if I can keep my conformed map and still have the validation on distinct keywords
2016:09:30 08:21:34 yenda btw do I misunderstand unform ? I thought this call (s/unform ::x (s/conform ::x data-to-conform)) would return data-to-conform but it just returns the conformed data
2016:09:30 08:30:22 yenda @seancorfield is the explanation on why it is an anti-pattern available somewhere ?
2016:09:30 08:32:00 yenda below is the final spec I made thanks to your input that matches my needs, I could eventually just do the map transform outside of the spec iff conformer is an anti-pattern
2016:09:30 08:39:03 jmglov @ag Here's how I generate date strings:
(require '[clj-time.core :as t]
'[clj-time.format :as f])
(import '(org.joda.time.format DateTimeFormatter))
(defn parseable-timestamp
"Returns a spec for a timestamp string can be parsed with the specified
datetime formatter."
[formatter]
(s/and string?
(fn [timestamp-str]
(try
(f/parse formatter timestamp-str)
true
(catch Exception _
false)))))
(defn make-timestamp-gen
"Returns a generator for a timestamp string between the minimum year and
maximum year (exclusive) that can be parsed with the specified datetime
formatter."
[min-year max-year formatter]
(fn []
(let [year-gen (s/gen (s/int-in min-year max-year))
month-gen (s/gen (s/int-in 1 12))
day-gen (s/gen (s/int-in 1 28))
hour-gen (s/gen (s/int-in 0 23))
m-s-gen (s/gen (s/int-in 0 59))]
(->> [year-gen month-gen day-gen hour-gen m-s-gen m-s-gen m-s-gen]
(map #(gen/fmap vector %))
(apply gen/cat)
(gen/fmap #(->> (apply t/date-time %)
(f/unparse formatter)))))))
(def timestamp-formatter (f/formatters :date-time-no-ms))
(s/def ::timestamp
(s/with-gen
(parseable-timestamp timestamp-formatter)
(make-timestamp-gen 2000 2050 timestamp-formatter)))
2016:09:30 13:26:03 mpenet asking again: I'd like to generate documentation from specs, is there a way to resolve/expand specs from (s/form ...) (or other) ?
2016:09:30 13:28:47 mpenet I guess I can hack this walking the (s/registry) + (s/form ...), but that seems a bit "hairy"
2016:09:30 13:41:35 mpenet (it's for front-end team, so I cant just spit keywords, i need to resolve the leafs of specs to stuff they understand)
2016:09:30 13:53:43 mpenet might be a good fit for some specter juggling
2016:09:30 14:02:57 alexmiller are you talking about a “deep” version of s/form?
2016:09:30 14:03:32 mpenet yes
2016:09:30 14:03:52 alexmiller yeah, we’ve talked about providing that but it doesn’t exist right now
2016:09:30 14:04:19 mpenet that'd be nice, it's should be easy enough to write, but that's something one would expect out of the box imho
2016:09:30 14:04:21 alexmiller I have specs for spec forms (kind of, modulo a number of bugs in s/form) which could help a lot
2016:09:30 14:04:39 mpenet sounds good
2016:09:30 14:04:49 alexmiller because you could conform the spec and then pick the parts per form that you need
2016:09:30 14:05:01 alexmiller I will probably get around to this soon-ish as I need it for other things
2016:09:30 14:05:11 mpenet I didn't actually think about approaching it that way, neat
2016:09:30 14:05:35 alexmiller having specs for specs opens up all sorts of things
2016:09:30 14:06:19 alexmiller for example, automated testing of spec by generating specs from spec specs, then checking conformance on data generated for the generated spec
2016:09:30 14:06:47 mpenet that's spec inception
2016:09:30 14:06:56 mpenet oh since you're here, I have another question about the conformer + :clojure.spec/invalid issue I mentioned yesterday
2016:09:30 14:07:02 alexmiller yes, it feels like that :)
2016:09:30 14:09:57 mpenet it makes writing conformers a bit tricky, I basically have to (def invalid :clojure.spec/invalid) and use this in the body of the functions I am using
2016:09:30 14:12:02 alexmiller yeah, but wait for a spec on def to break that too :)
2016:09:30 14:12:11 mpenet 😞
2016:09:30 14:12:22 alexmiller it’s a broader problem
2016:09:30 14:12:58 alexmiller one possible solution would be to do something like is done in the reader where a reader is handed a token that it can return to indicate a special case
2016:09:30 14:13:45 alexmiller so rather than being expected to return ::s/invalid, you are given a token that you can return (and that can be a per-instance (Object.))
2016:09:30 14:13:48 alexmiller something like that
2016:09:30 14:14:34 mpenet ok
2016:09:30 14:15:06 alexmiller that still doesn’t solve the problems around core specs though
2016:09:30 14:15:23 alexmiller it needs a longer conversation with Rich and he hasn’t had the time to have it
2016:09:30 14:15:54 mpenet any hint of what might come in the next alpha?
2016:09:30 14:28:58 alexmiller whatever we fix or add next :)
2016:09:30 14:30:41 alexmiller CLJ-2024, CLJ-2027, and CLJ-2026 all have patches that are ready for Stu and Rich to look at so I expect those to be in the next alpha
2016:09:30 14:30:51 alexmiller (presuming they like them)
2016:09:30 14:51:52 mpenet Actually walking the specs isn't really possible in my case: if you do (s/def ::foo ::bar) (s/form ::foo) will have expanded ::bar to a predicate so I cannot check my stop point (since I don't want to expand down to the last bits, I have a set of specs that are the surface I want to expose to the front-end people)
2016:09:30 14:54:13 mpenet ((s/registry) ::bar) works in that case actually
2016:09:30 14:54:21 mpenet nevermind
2016:09:30 19:01:27 jrheard i’ve got a somewhat complex map spec that i’m trying to s/exercise, and i’m getting the dreaded "Couldn't satisfy such-that predicate after 100 tries.” error.
what are the primary things i should be looking for when tracking down the root cause?
2016:09:30 19:01:44 jrheard i’m familiar with s/and and how you should eg do int? first, followed by even?, etc
2016:09:30 19:02:00 jrheard is naive s/and usage the primary trigger for this error, or are there other categories of spec misuse that often cause it?
2016:09:30 19:02:59 jrheard i’ve only got one s/and in here, and it’s a (s/and number? pos?), so i feel like there must be some other category of thing i’m doing totally incorrectly
2016:09:30 19:14:39 jrheard hm, seems to be specific to :component/input
2016:09:30 19:17:52 jrheard the documentation on :kind says: "Note that if :kind is specified and :into is not, this pred must generate in order for every to generate.”; and adding :into #{} fixes the issue. guess i need to sit down and take some time to understand the semantics of :kind and :into. thanks all ❤️
2016:09:30 19:26:44 jrheard (if you click on those links after coming back from lunch, pretend that the (s/coll-of) calls with :kind specified do not have a corresponding :into; that’s the state the file was in when i linked it, and that’s what the problem was 🙂 )
2016:09:30 19:32:59 seancorfield @jrheard Late to your spec party but, yeah, naïve s/and has been my primary trigger for that error — I’ve taken to using s/with-gen quite a lot to "help" clojure.spec exercise stuff.
2016:09:30 20:04:42 danielstockton I s/fdefed a function but the doc string doesn't contain my spec. Did I miss something?
2016:09:30 21:28:22 kenny Is this the expected behavior?
2016:09:30 21:44:41 bfabry still, wouldn't flowing back through vector? cause it to be turned into a vector? shrug
2016:10:01 00:09:49 jrheard what’s the recommended way of actually running stest/check tests?
2016:10:01 00:09:58 jrheard i’m in a cljs project, using lein-doo and cljs.test
2016:10:01 00:10:29 jrheard should i be using (stest/summarize-results (stest/check `foo/bar)), and verifying that there’s no :check-failed key in the returned map? or is there a better way?
2016:10:01 13:34:38 puzzler So could I just "patch" arg-list?
2016:10:01 13:37:20 puzzler I guess if I s/def :clojure.core.specs/arg-list in my library, that will overwrite the spec for other consumers of my library as well, so maybe I shouldn't do that.
2016:10:01 13:55:59 alexmiller maybe you could make the patching a separate (temporary) step
2016:10:01 15:27:16 decoursin @alexmiller Thanks for having a look. It's working now. I'm not sure, if it comes up again, I'll let you know
2016:10:01 20:12:20 jrheard i’m having a lot of trouble getting spec’s generative testing to work
2016:10:01 20:12:39 jrheard when i call (stest/check `foo/bar), and the output is [], how should i interpret that empty list?
2016:10:01 20:14:57 jrheard i get [] from calling (stest/check) on each fdef’d function in this file, eg https://github.com/jrheard/voke/blob/spec/src/voke/input.cljs#L85 , and don’t really know where to go from here. has anyone seen this behavior? is there something obvious that i’m doing wrong?
2016:10:01 22:52:33 lvh Hm, going from -alpha11 to -alpha12/-alpha13 I get a pretty unhelpful NPE when calling explain on some data. clojure.spec portion of the stacktrace:
2016:10:01 22:52:45 lvh I dunno if anyone’s seen anything like that or if it’s worth producing a minimal sample
2016:10:01 22:53:45 lvh which suggests that pred in spec-impl is nil
2016:10:01 22:59:32 lvh Did anything change about how registry-ref works?
2016:10:01 23:04:12 lvh (The weirdest thing that code path does is (with-redefs [s/registry-ref (atom @@#'s/registry-ref)] …), which it needs to because it’s generating specs so I don’t want to have tests fail because of stale state)
2016:10:01 23:19:28 lvh That does not look like it’s part of the problem.
2016:10:01 23:36:38 ag I am dealing with strangest bug I cannot reproduce anywhere else: it works everywhere except Circle CI. Basically this spec:
ERROR in (test-datetime-string) (:)
Uncaught exception, not in assertion.
expected: nil
actual: #error {:message "Couldn't satisfy such-that predicate after 100 tries.", :data {}}
2016:10:01 23:38:35 ag first I thought it’s due to incompatible version of phantomjs. and then version of Leineingen, and then I disabled any parallelism on CI. and still can’t figure it out. I thought maybe I could replicate in a Docker container - still nothing
2016:10:01 23:39:26 ag Anyone, any ideas?
2016:10:01 23:39:34 lvh hm, that’s weird: it would only make sense if the generator portion is being ignored
2016:10:01 23:40:06 ag but why would that happen only in Circle CI and nowhere else?
2016:10:01 23:40:08 lvh maybe you should create a test PR that s/gen + gen/samples that spec
2016:10:01 23:40:37 lvh that could be a few reasons, including profile.clj files or whatever
2016:10:01 23:40:53 lvh s/gen + gen/samples will tell you what it thinks the generator is for that spec
2016:10:01 23:41:32 lvh how did you determine it wasn’t an incompatible version of phantomjs?
2016:10:01 23:42:41 ag I am replacing phantom with the right version
2016:10:01 23:43:44 gfredericks LEIN_NO_USER_PROFILES=1 can be helpful for debugging things that only happen in ci
2016:10:01 23:45:23 lvh I’ve seen a lot of problems with Phantom where phantom versions insisting they were the same, although that regex does look innocuous
2016:10:01 23:45:37 lvh 1.9.8 is like seventeen different webkits
2016:10:01 23:45:45 gfredericks Also have you rerun the ci? That "such-that" error could also just be highly improbable
2016:10:01 23:46:35 lvh Wouldn’t the such-that error never happen with with-gen? I thought that with-gen did not check that generated values match the spec?
2016:10:01 23:48:21 lvh (that may have changed in recent alphas, or not be true in JS, or I might be misremembering entirely of course)
2016:10:01 23:49:45 gfredericks I don't know where the such-that came from actually
2016:10:01 23:50:05 lvh well, I’m thinking of the spec by itself
2016:10:01 23:50:22 lvh IIRC s/and uses the first arg to determine type
2016:10:01 23:50:36 lvh so it won’t even be able to guess “string” let alone “string that incidentally matches this regex"
2016:10:01 23:51:43 lvh (Sorry, to be less obtuse: I think the such-that is coming from the s/and)
2016:10:01 23:52:46 gfredericks Oh I didn't see an and
2016:10:01 23:53:22 gfredericks It shouldn't matter though since it's wrapped in with-gen
2016:10:01 23:55:48 lvh exactly
2016:10:01 23:56:40 lvh @ag: Can you please dump the full test ns somewhere
2016:10:01 23:56:43 lvh ideally a reproducible sample
2016:10:02 00:02:22 jrheard @gfredericks - have you ever seen s/check output the empty list like i mentioned above? sorry to bug you directly, i’ve just seen your name around test.check-related things, figured you might have an answer off the top of your head 🙂
2016:10:02 00:02:51 jrheard i’m a complete newbie to generative testing, so for all i know this is normal well-understood behavior, but i’m having trouble making heads or tails of it
2016:10:02 00:03:35 gfredericks I expect it's a clojure.spec-related thing
2016:10:02 00:04:00 gfredericks I've seen that happen when I've used it but can't remember why
2016:10:02 00:04:28 jrheard cool, i’ll keep poking at it
2016:10:02 18:36:48 lvh I found my problem from yesterday: between -11 and -12, a nil spec changed meaning
2016:10:02 18:36:59 lvh ostensibly from meaning any? to silent failure
2016:10:02 18:37:26 lvh I appreciate that there might be a bootstrapping failure, but that seems like it could be found at spec definition time and not with an NPE thrown from a reify
2016:10:02 18:43:10 bfabry generated values are still checked against the spec, I'm guessing there's something in the environment causing .format to return a differently formatted date
2016:10:02 18:52:21 lvh oh, huh; I wonder if that’s now or I’m just horribly misremembering
2016:10:02 18:52:58 lvh I guess .format isn’t awfully specific
2016:10:02 18:53:45 bfabry there's probably some dumb version of the JS engine that returns "Monday the 2nd of Feburary 10:13PM Australian eastern daylight savings time"
2016:10:02 18:57:21 lvh right
2016:10:02 18:57:35 lvh Moment.js takes explicit format strings though, so hopefully easy to recover from
2016:10:02 18:57:37 lvh @ag ^
2016:10:02 23:11:14 alexmiller @lvh There was a bug introduced with conforming nilable in 12 that is fixed in 13
2016:10:02 23:13:28 lvh @alexmiller In -alpha11, a nil spec was effectively any? afaict
2016:10:02 23:13:42 lvh that’s gone in -12 and -13, which broke my tests; but really it was just exposing something that was a bug anyway
2016:10:02 23:15:20 lvh this is not a problem unless you’re generating specs, which I guess is the part of clojure.spec I’m exercising a little more that most folks 😄
2016:10:02 23:15:29 lvh (my generated specs included nils by accident; that was actually a bug)
2016:10:02 23:54:02 jrheard @alexmiller - have you seen situations where (stest/check `foo/bar) returns []? what type of mistake does that return value usually indicate?
i’m having a hard time figuring out what causes it - it seems to return [] sometimes and eg [{:spec #object[cljs.spec.t_cljs$spec8685], :clojure.test.check/ret {:result true, :num-tests 1000, :seed 1475452376922}, :sym voke.input/intended-directions->angle}] other times, the behavior varying from run to run when none of the code has changed
2016:10:03 00:10:36 gfredericks that sounds pretty weird
2016:10:03 01:27:48 jrheard i think tomorrow i’m just gonna have to learn how to use checkout dependencies and start adding printfs to spec.test code so i can see what’s going on in there
2016:10:03 02:21:08 bhagany @jrheard fwiw, I’m eager to hear what you learn. I tried getting doo to run my cljs spec tests a few weeks ago, and hit a wall.
2016:10:03 04:02:42 jrheard well, for starters i think i’ve found a small issue with the cljs.spec documentation
2016:10:03 04:03:20 jrheard but actually, they should look like {:clojure.test.check/opts {:num-tests 100}}
2016:10:03 04:36:24 jrheard i haven’t managed to reproduce this [] return value issue using my local checkout of clojurescript, so maybe this behavior has been fixed on master? will poke at it some more in the morning
2016:10:03 04:55:46 seancorfield Are you sure about the options map @jrheard ? When I was trying that on Clojure, it definitely needed to be :clojure.spec.test.check/opts (even tho’ no such namespace exists).
2016:10:03 04:56:20 seancorfield If it really is the latter, as you say, then that’s a bug in my opinion — ClojureScript should follow Clojure there I think?
2016:10:03 06:56:04 decoursin How to override keys in (s/merge ..)? My failing attempt:
2016:10:03 07:18:51 misha @decoursin: what do you expect merged spec to be?
2016:10:03 07:20:01 misha "Int or string"?
2016:10:03 07:23:22 decoursin Either one, make a choice. Always take the first or always take the second.
2016:10:03 07:25:01 decoursin probably the same way clojure.core/merge works which is the second (or last one.)
2016:10:03 07:25:33 decoursin
(merge {:a 3} {:a 4})
=> {:a 4}
2016:10:03 07:27:26 misha
Takes map-validating specs (e.g. 'keys' specs) and
returns a spec that returns a conformed map satisfying all of the
specs. Unlike 'and', merge can generate maps satisfying the
union of the predicates.
does not sound like it can satisfy int? and string? with a same value on a single key
2016:10:03 07:29:13 bfabry I think in terms of usefulness the way it's behaving atm is probably best, though I could see an argument for it behaving like core.merge
2016:10:03 07:30:08 decoursin Yeah I think it should behave like core.merge. I'd rather see that what @misha described to be called like union or something
2016:10:03 08:22:34 decoursin How could I do this then? Have one key override another
2016:10:03 08:42:51 misha what is your use case? or rather "problem"
2016:10:03 11:31:17 mpenet is there a way to validate against a (dynamic) subset of keys in a map? ex I define an exhaustive spec for a map (with mixed req/opt), and at runtime I want to be able to validate against a subset of it
2016:10:03 11:32:43 mpenet a bit in the flavor of s/merge, I'd like s/select-keys (kind of)
2016:10:03 11:33:38 misha you can give names to those subsets, and just merge those in uber-map-spec. no?
2016:10:03 11:34:05 mpenet I dont want to define as many subsets are they are key combos no
2016:10:03 11:34:59 mpenet ex with Schema you can just do (since map specs are just Maps) :
2016:10:03 11:35:22 mpenet seems like s/select-keys could be a nice addition actually, thoughts @alexmiller ?
2016:10:03 11:38:41 mpenet just putting every key as optional is not an option either
2016:10:03 11:42:14 misha given spec's "global" nature, I'd go and name all the combos.
also multi-spec might be a good fit too
2016:10:03 11:42:46 mpenet that's a lot of combos for a 20 key map for instance ... it'd be horrible
2016:10:03 11:43:10 mpenet I'll check multi-spec, but my gut feeling it doesn't fit that problem either
2016:10:03 11:43:44 misha depending on how you'd define combos out of those 20
2016:10:03 11:44:32 misha you might have only 2-3 "domain-valid" combos.
2016:10:03 11:44:34 mpenet doesn't matter
2016:10:03 11:44:44 mpenet in my case the user can update any of them and I cannot pass all of it
2016:10:03 11:45:09 mpenet (there are concurrent update issues for the "pass the whole thing" case)
2016:10:03 11:45:27 misha not enough information then
2016:10:03 11:46:48 mpenet it's a common enough problem imho, any REST api would hit it for partial update for instance
2016:10:03 11:47:11 mpenet the least horrible (yet it sucks) imho would be to call valid? against every key in the set but yuk. Or just have a s/keys spec with all as optional for updates, but that's a lot of duplication I'd like to avoid
2016:10:03 11:47:55 misha then probably I'd have 20 different specs defined, and constructed map spec on demand based on incoming map's keys
2016:10:03 11:48:55 misha 20 for possible keys, + maybe some for key combinations, like "if that one present, those 2 are required".
2016:10:03 11:51:12 misha a collection of specs for possible keys should not necessarily be a ready to use map spec.
2016:10:03 11:51:13 mpenet not going to do that, that's truly awful, not to mention for 20 fields (the example here but some of our records have more than this) that's a large number of specs (a lot more than 20)
2016:10:03 11:52:07 misha not everything can be solved with oneliners kappa
2016:10:03 11:52:38 mpenet it's not the point, you don't have a solution to this, no biggie
2016:10:03 11:53:02 mpenet At the moment there's no elegant way to solve this with spec.
2016:10:03 12:35:43 mpenet That's more or less what I mentioned earlier, but I dont like it. Having 1 separate spec with all opt keys is probably nicer
2016:10:03 13:46:26 danielstockton Is there a way to instrument everything at once, rather than individual functions?
2016:10:03 13:47:36 alexmiller Just call instrument with no args
2016:10:03 13:52:19 danielstockton Is there a good way to hook this into the test runner? I imagine it can be quite helpful to know not only what tests failed but which functions got unexpected data.
2016:10:03 13:55:25 mpenet I think you can just call instrument at top level of your test
2016:10:03 13:55:46 danielstockton I suppose, i'd have to do it in every namespace
2016:10:03 13:56:33 danielstockton Might also be useful to turn it on whenever starting lein repl or in a dev environment
2016:10:03 14:09:03 mpenet @alexmiller (s/valid? #{false} false) => false that's what you mention on github?
2016:10:03 14:09:35 mpenet it does look like a bug from the outside (without considering/knowing how it's implemented under the hood)
2016:10:03 14:14:12 mpenet I guess its (#{false true} false)
2016:10:03 14:14:16 alexmiller Yes that's what I meant. It's not a bug, just a consequence of using sets as specs
2016:10:03 14:14:21 mpenet ok
2016:10:03 14:14:38 alexmiller Sets with falsey values won't work
2016:10:03 14:15:19 alexmiller So don't do that
2016:10:03 14:15:52 alexmiller You've used a function that returns false for what you consider to be a valid value
2016:10:03 14:16:06 mpenet yep, got it
2016:10:03 14:26:31 mpenet actually about my s/select-keys proposal earlier, why not making s/keys spec an c.l.Associative instance and allow us to compose this stuff with normal core fn?
2016:10:03 14:27:18 mpenet maybe it's crazy talk
2016:10:03 15:35:27 alexmiller the problem with that is that we capture the key set for describe and form, so you would lose that ability
2016:10:03 15:36:19 alexmiller that’s the reason it’s a macro and not a function that takes data now
2016:10:03 15:42:06 alexmiller @jrheard no, haven’t seen that
2016:10:03 15:59:58 mlimotte hi. I'm just starting to play with clojure.spec. I'm ok with some basic specs and validation that I've tried. But having trouble with even a trivial example of specing a higher-order fn. Here's what I'm seeing:
(def my-spec (s/fspec :ret string?))
=> #'user/my-spec
(s/conform my-spec (fn [j] (str j)))
IllegalArgumentException No implementation of method: :specize* of protocol: #'clojure.spec/Specize found for class: nil clojure.core/-cache-protocol-fn (core_deftype.clj:568)
2016:10:03 16:04:31 alexmiller (s/fdef my-spec :ret string?) will be better for you right now
2016:10:03 16:05:23 alexmiller sorry, I misread that first def as s/def, let me read again
2016:10:03 16:06:44 alexmiller so the issue here is that to verify that the function you’ve passed is valid, it will generate args based on the :args spec for the function and invoke it
2016:10:03 16:06:50 alexmiller but you’ve passed no args spec
2016:10:03 16:08:03 mlimotte true. I was going with a simple example -- just validating that it is a fn that returns a string. I did try with args before, and was getting different errors, so I was trying to simplify the example.
2016:10:03 16:08:08 mlimotte i'll try w/ args again
2016:10:03 16:08:17 alexmiller although I am seeing some weird stuff on this path
2016:10:03 16:09:11 mlimotte (s/def my-spec (s/fspec :args (s/tuple integer?) :ret string?))
=> user/my-spec
(s/conform my-spec (fn [j] (str j)))
IllegalArgumentException No implementation of method: :specize* of protocol: #'clojure.spec/Specize found for class: nil clojure.core/-cache-protocol-fn (core_deftype.clj:568)
2016:10:03 16:09:41 mlimotte 😞 same thing, even with args.
2016:10:03 16:14:00 alexmiller ah, so conform takes a qualified keyword or symbol
2016:10:03 16:14:20 alexmiller
(s/conform `my-spec (fn [j] (str j)))
2016:10:03 16:14:39 alexmiller will fully-qualify my-spec
2016:10:03 16:15:30 alexmiller @jrheard the idea in both of those is to pass a map of options through to test.check - I think the underlying option keys differ in clj vs cljs test.check
2016:10:03 16:16:06 alexmiller so there may be a disconnect between clj vs cljs and docs here
2016:10:03 16:16:38 jrheard i was using :clojure.spec.test.check/opts as the docs recommend, but nothing was happening
2016:10:03 16:16:41 alexmiller I suspect so - prob should file a jira (and if you like a patch!)
2016:10:03 16:16:52 mlimotte ahh! it does say that in the spec guide! Maybe it would be helpful for the conform and valid? docstring(s) could also highlight that requirement.
2016:10:03 16:16:53 jrheard sounds good, it’ll be my first of both of those 🙂 time to go read the contributing guidelines again
2016:10:03 16:17:26 alexmiller @mlimotte seems like there should be a better error message in this case
2016:10:03 16:17:39 alexmiller I will log that and add a patc hfor it
2016:10:03 16:19:47 mlimotte yep. that would help. here's another question. What's a recommended way to spec the requirement that the fn should be a one-arg fn? I tried:
2016:10:03 16:21:40 jrheard (i’ve made a jira account, username jrheard - alex, are you the right person to bug to add whatever necessary permissions to that account, or should i go bother someone else?)
2016:10:03 16:22:05 alexmiller you can add tickets with the starting permissions
2016:10:03 16:22:08 jrheard awesome
2016:10:03 16:24:29 mlimotte what is any?, AFAICT that is not in clojure.core nor clojure.spec
2016:10:03 16:24:35 alexmiller it’s in core
2016:10:03 16:24:36 lvh It’s new in Clojure
2016:10:03 16:24:42 alexmiller it’s just (constantly true)
2016:10:03 16:25:08 lvh hm; I figured it might take only 1 arg
2016:10:03 16:25:26 alexmiller actually it does
2016:10:03 16:25:46 alexmiller I was more sketching than saying the actual def :)
2016:10:03 16:25:54 alexmiller
(defn any?
"Returns true given any argument."
{:tag Boolean
:added "1.9"}
[x] true)
2016:10:03 16:26:26 lvh ah, gotcha 😄
2016:10:03 16:27:29 mlimotte i see. I'm using clojure-future-spec w/ clojure 1.8, so this works:
(s/def my-spec (s/fspec :args (s/cat :j clojure.future/any?) :ret string?))
2016:10:03 16:28:17 alexmiller ah
2016:10:03 16:28:18 mlimotte as does:
(s/def my-spec (s/fspec :args (s/tuple clojure.future/any?) :ret string?))
Was your choice of s/cat over s/tuple just stylistic, or is there something more significant?
2016:10:03 16:28:59 alexmiller @jrheard I added edit permissions for you as well in jira
2016:10:03 16:29:02 jrheard thanks!
2016:10:03 16:31:49 jrheard opened CLJS-1808 , will submit a patch later today
2016:10:03 16:31:55 alexmiller thanks
2016:10:03 16:32:56 alexmiller @mlimotte tuple is fine too. we usually use regex ops to describe args. Args are effectively syntax and that’s what regex ops in spec are the best match for.
2016:10:03 16:33:45 alexmiller one benefit of cat is that you will get named parts for conforming the parts whereas you will get just indexes for tuple
2016:10:03 16:34:19 alexmiller so that affects both the conform result (which you’ll see in the :fn spec) and the explain errors
2016:10:03 16:34:43 alexmiller generally I find being able to tie the spec to the arg name helpful
2016:10:03 16:35:52 mlimotte got it. thanks for the help.
2016:10:03 17:43:52 jrheard @alexmiller - i just added a couple patches to the ticket. is there anything else i need to do, or should i just wait and expect to get a response at some point in the future?
i’m asking you this not because i’m in an incredible hurry to have someone immediately look at this jira issue this very second; i just want to make sure i’m not missing some “notify the maintainers” step before i close all these tabs 🙂 thanks!
2016:10:03 17:47:33 alexmiller @dnolen: ^^
2016:10:03 17:48:33 alexmiller For cljs-1808
2016:10:03 17:48:39 mlimotte Could there be a problem with s/tuple? I have the code below, which bombs w/ the error below, but if I change the '(s/tuple integer?)` to (s/cat :a integer?) than it works.
=> #'user/myfn
=> user/myfn
=> [user/myfn]
ExceptionInfo Call to #'user/myfn did not conform to spec:
val: (1) fails at: [:args] predicate: vector?
:clojure.spec/args (1)
:clojure.spec/failure :instrument
:clojure.spec.test/caller {:file "form-init1342783443404146057.clj", :line 4, :var-scope user/eval30918}
clojure.core/ex-info (core.clj:4617)
2016:10:03 17:50:58 bfabry @mlimotte s/tuple specifically expects its input to be a vector, the :args list is not a vector
2016:10:03 17:51:28 bfabry s/cat just validates a sequence, which the :args list is
2016:10:03 17:53:42 mlimotte i see. in retrospect, kind of obvious from the error message. i should have gotten that.
2016:10:03 17:55:04 dnolen @jrheard just assign it to me I will look at it when I have time
2016:10:03 17:55:11 jrheard great, will do, thanks!
2016:10:03 21:15:13 tokenshift Can s/conform handle coercion? For example, could I have an s/def that defines a ::uuid to be either a (valid) string or a java.util.UUID, and always conforms it to a java.util.UUID?
2016:10:03 21:28:22 lvh tokenshift Check out “conformer"
2016:10:03 21:29:50 tokenshift I’d pass a conformer (the result of s/conformer) to s/def, e.g. (s/def ::foo (s/conformer #(do-stuff %)))?
2016:10:03 21:31:04 tokenshift Answered my own question; yep, that works
2016:10:03 21:31:07 tokenshift Thanks!
2016:10:03 21:33:08 tokenshift Now I have to decide if that’s a good pattern, or if it’d be better to keep input coercion completely separate from specs
2016:10:03 21:38:27 lvh I think it’s fine to conform to a specific object
2016:10:03 21:38:53 lvh conforming is not something that you want in your hot loop though IIUC; it’s a measurable performance impac
2016:10:03 21:38:54 lvh t
2016:10:03 21:39:04 lvh but at the edges? sure; you’re gonna do that anyway, might as well have it be declarative
2016:10:03 21:39:22 tokenshift Hmm, that’s a good point
2016:10:03 21:39:57 tokenshift If it’s only at the edges, then I can assume the input will be a string (for a web service, at least)
2016:10:03 21:40:17 tokenshift Though if the backend requires a UUID, I’d probably still want to coerce it as early as possible.
2016:10:03 21:40:56 tokenshift I guess if the coercion is lossless, it’s safe; but if it’s lossy, I’d want to have an explicit coercion step.
2016:10:03 21:48:35 tokenshift Heading out
2016:10:03 21:48:39 tokenshift Thanks for the help!
2016:10:04 00:10:48 jrheard heh, enabling (s/instrument) on all instrumentable vars does not do wonders for my game’s performance
2016:10:04 00:10:58 jrheard invaluable for finding the 7 random spots where i’d goofed when converting to namespaced keywords, though 🙂
2016:10:04 00:11:42 jrheard i’m not complaining, i absolutely expected this degree of perf degradation when instrumenting like every function in this tight loop, it was just funny to go from 60fps to 0.5 😄
2016:10:04 00:15:59 jrheard instrumenting only at the edges brings me back up to good perf, nice
2016:10:04 00:22:25 ag so it turned out that my regex is bad
2016:10:04 00:22:38 ag it does not take into account timezones
2016:10:04 00:22:54 ag that minus sign sometimes is a plus sign
2016:10:04 00:23:42 ag although I think it would be helpful if spec told me why exactly it couldn’t satisfy the predicate
2016:10:04 00:25:15 jrheard i worked with a guy once whose most famous quote around the office was:
<jlatt> dates suck
<jlatt> f&@# time
2016:10:04 00:25:51 jrheard i’ve had similar feelings when running into the “Couldn’t satisfy such-that predicate after 100 tries” error - i wish there was some way it could tell me how i’ve messed up
2016:10:04 00:25:54 ag also I think test.chuck generator does not generate values with random timezones. maybe it should
2016:10:04 00:25:54 jrheard i’m not sure how it could do that though
2016:10:04 13:20:54 danielstockton Argh, I think I want s/cat instead of s/and actually
2016:10:04 13:21:03 danielstockton now i see that s/and is adding two specs
2016:10:04 13:21:48 danielstockton yep, thanks @minimal, somehow i was blind to that
2016:10:04 22:23:26 seancorfield We officially have clojure.spec-based code in production now — Clojure 1.9.0 Alpha 13 and all of our REST API search code relies on spec for validation and conformation!
2016:10:04 23:08:19 jcsims @seancorfield any plan to write about it?
2016:10:04 23:35:08 seancorfield @jcsims I submitted a proposal to Clojure/conj about it… so if that gets accepted, I’ll have to write about it. If it doesn’t get accepted, I’ll probably still write it up for my blog, but in a very different format I expect.
2016:10:04 23:36:06 seancorfield Part of my concern is that core to a lot of what we’re doing with spec is something that @alexmiller said is essentially an anti-pattern (heavy use of s/conformer).
2016:10:04 23:36:54 seancorfield That, and we are still at times struggling mightily with how to apply spec to some situations...
2016:10:04 23:36:55 jcsims well, looking forward to it either way :+1:
2016:10:05 07:04:45 mpenet it makes s/merge a bit useless tbh, not sure why it's not considered a bug
2016:10:05 07:05:59 mpenet I guess a workaround would be to do the composition at a higher level, and not rely on spec/merge for merging specs like this, which is... odd
2016:10:05 07:14:29 mpenet and the fact that it's macros all the way down makes this workaround also a bit hairy
2016:10:05 08:43:27 jmglov Is it possible to spec protocols in any way?
2016:10:05 08:43:42 jmglov Or objects that satisfy them?
2016:10:05 11:08:17 mpenet I think it's encouraged to wrap protocol fn invocation, so you can instrument/spec the wrapper fn. otherwise if you mean spec predicate it's just #(satisfies? %), possibly with (s/spec pred :gen ... ) if you want to have the gen part
2016:10:05 11:43:13 odinodin Anyone used spec with DataScript? Need tips on how to deal with the fact that entities are not maps, and specifically how to get the same functionality as clojure.spec/keys but for Datascript entities
2016:10:05 11:53:45 misha @odinodin what do you mean by "not maps"? pull pretty much returns maps.
2016:10:05 11:54:21 misha what exactly do you want to cover with spec?
2016:10:05 11:54:46 odinodin I have a function that takes DataScript entities
2016:10:05 11:55:16 odinodin I'd rather avoid creating maps from entities, and just use entities directly
2016:10:05 11:56:58 misha oh, you mean thing returned by d/entity? no idea : )
2016:10:05 11:57:44 odinodin 🙂
2016:10:05 12:29:38 Yehonathan Sharvit Is it possible to define a spec that references itself in the definition - like in Context-Free Grammars?
2016:10:05 12:30:23 alexmiller Sure, via it's registered keyword
2016:10:05 12:30:49 alexmiller Recursive and mutually recursive specs are fine
2016:10:05 12:31:10 Yehonathan Sharvit Could you share an example?
2016:10:05 12:32:44 alexmiller There are several in clojure.core.specs - destructuring is recursive
2016:10:05 12:33:11 alexmiller Some of the ns stuff with prefix lists , etc
2016:10:05 12:36:21 alexmiller ::prefix-list there is self-recursive
2016:10:05 12:37:19 alexmiller ::binding-form can be ::seq-binding-form or ::map-binding-form, which can both include ::binding-form
2016:10:05 12:38:07 Yehonathan Sharvit thx
2016:10:05 12:38:14 Yehonathan Sharvit I’m looking at it
2016:10:05 12:40:46 alexmiller A really good example to try is just a simple tree of leaf and branch
2016:10:05 12:41:18 alexmiller Where branches can contain either leaf or branch
2016:10:05 13:49:18 mlimotte What's the best way to trigger clojure.spec tests with lein test? I.e. I have a namesapce with (stest/check (stest/enumerate-namespace 'wwai.common.util.instant)), I want it run those checks when I do lein test.
I'm really using expectations, but if I see how to make it work with standard test, I should be able to adapt from their.
2016:10:05 16:48:49 ag @mlimotte clojure.test.check’s defscpec worked for me
2016:10:05 16:52:46 jrheard @ag i’d love to see an example of that on github if you have one, np if not, just figured i’d check 🙂
2016:10:05 18:08:40 mlimotte I came up with this hack, which kind of works, although the output in the error case, isn't very friendly:
2016:10:05 19:11:34 bfabry instrument does not check return values
2016:10:05 19:13:22 luxbock I see
2016:10:05 19:15:33 bfabry you're probably looking for clojure.spec.test/check
2016:10:05 19:16:16 bfabry it runs the function a bunch of times with generated args (in this case always nothing) and checks the result conforms to :ret and that any supplied :fn relationship between :args and :ret is true
2016:10:05 19:20:46 luxbock thanks, yeah the function which I simplified here probably won't need a spec at all, but it's good to know this for the future
2016:10:05 20:02:29 jasonjckn how do I specify that ::start_date must start before ::end_date in a map
2016:10:05 20:02:48 jasonjckn so far I have (s/keys :req-un [::start-date ::end-date]) but I need to add in a predicate
2016:10:05 20:02:55 jasonjckn the relationship between these two
2016:10:05 20:07:54 jrheard perhaps you could write a constructor function rather than making instances of that map by hand, and it’s got an (s/fdef) that encodes that information?
2016:10:05 20:08:53 jrheard doesn’t 100% solve the problem though
2016:10:05 20:09:08 mlimotte maybe you could use s/and and a predicate, something like:
(s/and (s/keys :req-un [::start-date ::end-date]) #( date-before? (:start-date %) (:end-date %)))
2016:10:05 20:09:26 jrheard oh interesting
2016:10:05 20:09:30 jrheard i like that better
2016:10:05 20:10:51 jasonjckn cool thanks, testing now
2016:10:05 20:38:34 luxbock what is the use case of the optional argument unf to conformer?
2016:10:05 20:40:43 bfabry @luxbock I'm guessing it's for a custom unformer
2016:10:05 20:42:41 luxbock ah right, yeah I get it now
2016:10:05 21:47:28 jasonjckn is there a way to include doc string with the spec?
2016:10:05 21:47:56 jasonjckn e.g. (s/and (s/keys :req-un [::start-date ::end-date]) "start date must come before end date" #( date-before? (:start-date %) (:end-date %)))
2016:10:05 21:48:46 jasonjckn (and retrieve the doc string in explain-data)
2016:10:05 22:05:27 jrheard has anyone else had trouble doing eg
2016:10:05 22:06:01 jrheard i see a stacktrace with error messages like java.lang.RuntimeException: No such namespace: stest, compiling:(/private/var/folders/zl/bh7pbyz95rg7pmcvcc_f_kdm0000gn/T/form-init2230077429322923281.clj:6:19); gonna dig into it a bit, just curious if this is known / expected / if anyone else has dealt with this
2016:10:05 22:22:34 jrheard @dnolen am i doing something wrong / is this known/expected?
2016:10:05 22:23:06 dnolen @jrheard yes won’t work, not intended to work
2016:10:05 22:23:12 jrheard cool, good to know
2016:10:05 22:23:27 jrheard should i just instrument vars by hand, naming one at a time, or is there some better way?
2016:10:05 22:23:36 dnolen instrument is a macro - must take symbols that reference vars directly
2016:10:05 22:23:44 dnolen just skip the locals stuff
2016:10:05 22:23:50 jrheard i don’t follow
2016:10:05 22:24:00 dnolen remove the let binding and it will work
2016:10:05 22:24:12 dnolen i.e. do this inline
2016:10:05 22:24:19 jrheard when i do
(stest/instrument (stest/enumerate-namespace 'my.ns)), i get an error message complaining that stest doesn’t exist
2016:10:05 22:24:34 jrheard is that what you mean by doing this inline, or am i misunderstanding?
2016:10:05 22:24:42 dnolen so that’s a bug, file an issue in JIRA
2016:10:05 22:24:46 jrheard gotcha, will do! thanks!
2016:10:05 22:25:06 dnolen thanks, to be clear, do this in the CLJS JIRA
2016:10:05 22:25:08 jrheard you got it
2016:10:05 22:30:05 jrheard ( CLJS-1811 )
2016:10:05 23:58:45 olivergeorge Today's random spec experiment: Can you define an SQL schema from a clojure.spec definition?
2016:10:06 00:00:24 olivergeorge @bfabry no promises that it's the most elegant approach. I'm looking forward to seeing how others work with spec forms.
2016:10:06 00:16:29 mattly heh, I'm actually working on something to define graphql forms from specs definitions
2016:10:06 00:17:16 jrheard unrelated: you’re using graphql? d’you like it?
2016:10:06 00:17:33 jrheard i haven’t used it myself but the talks i’ve seen on it are pretty compelling
2016:10:06 00:20:51 olivergeorge @mattly that's on my radar too. We are replacing our simple "datomic style" pull api with graphql soon. Here's what I came up with generating recursive pull specs from clojure.spec forms: https://gist.github.com/olivergeorge/9d822447e4838092d07138ea51af5782
2016:10:06 00:21:15 mattly oh neat
2016:10:06 00:21:27 olivergeorge (again, no promises that it's a good approach... really want to see other peoples code and learn some effective techniques)
2016:10:06 00:22:05 mattly I'm going to keep my pull-style api, but aim to hookup a graphql parser to it
2016:10:06 00:23:45 olivergeorge That makes sense.
2016:10:06 00:24:27 mattly mostly because some of my query endpoints, I need expressive arguments
2016:10:06 00:26:32 olivergeorge We bolted a "pull" arg to our REST endpoints which was a practical step forward at the time. Same old REST params etc but more flexibility around what data we return. Now that GraphQL is looking fairly stable we're intrested in standardising on it's interface. That decouples our frontend from the backend which should allow us some flexibility.
2016:10:06 00:27:22 mattly I'm fairly lucky in that mine is a greenfield project
2016:10:06 00:28:05 olivergeorge Nice. We're in the enterprise space so typically the backend tech stack is dictated to us.
2016:10:06 00:29:52 mattly I may try to abstract out my resolver into a library
2016:10:06 00:30:23 mattly it has a separate planning stage to help with the n+1 problem
2016:10:06 00:30:40 olivergeorge @mattly I'd love to see what you come up with.
2016:10:06 00:33:54 olivergeorge I'm not sure it's related but I did some thinking around translating graph queries into flat sql queries.
2016:10:06 00:37:18 mattly eh, removed the link, there's a bug in that
2016:10:06 00:37:39 mattly I need to properly extract it
2016:10:06 00:38:45 olivergeorge 🙂
2016:10:06 07:35:45 mikeb @olivergeorge any thoughts about reversing direction and generating spec definitions from a sql schema?
2016:10:06 09:30:15 olivergeorge @mikeb I've had some fun with that actually. It's easy to produce something for each column in a database but the relations are trickier.
2016:10:06 09:32:37 olivergeorge The slightly fiddly bit is that you can't really do it dynamically. At least I can't work out how to have my script generate and run macros. In the end I generated some forms and committed them as code.
2016:10:06 15:04:55 Yehonathan Sharvit I was looking into clojurescript code and I saw that clojure/core/specs.clj from clojure is not there.
Why clojure/core/specs.clj hasn’t been ported into clojurescript?
2016:10:06 15:10:12 Yehonathan Sharvit @dnolen is it on purpose?
2016:10:06 15:11:42 dnolen Just haven't gotten to it yet
2016:10:06 15:14:51 Yehonathan Sharvit would you be interested in a patch?
2016:10:06 15:22:48 mlimotte I see this in the spec guide:
"check also takes a number of options that can be passed to test.check to influence the test run, as well as the option to override generators for parts of the spec, by either name or path."
Any examples of the "path" option for generator overrides?
2016:10:06 15:25:44 jrheard i’d love to see an example of generator override usage, path or no
2016:10:06 15:28:59 mlimotte @jrheard here's a basic example with override by name:
2016:10:06 15:43:50 dpiatek @mlimotte I think you need to do (s/fdef foo :args (s/cat :x (s/spec clojure.future/int?)))
2016:10:06 15:46:32 mlimotte The foo spec above seems to work even w/out (s/spec ...). I'm still not sure how to do a generator override for this case where the spec is not named, though.
2016:10:06 15:48:53 dpiatek Oh, I see - sorry, misread the conversation!
2016:10:06 15:49:05 mlimotte np
2016:10:06 18:34:18 naomarik i'm sorry about this naive question: if i were to spec a map that has strings as keys or even another type, i would have to convert them to keywords first before passing them into spec?
2016:10:06 18:42:05 mlimotte i don't think so, do you have an example of what you're trying to do?
2016:10:06 18:43:17 bfabry @naomarik no. you need keywords for keys to use s/keys but there's still lots of other predicates that you can use to validate maps, including writing your own
2016:10:06 18:46:44 naomarik i've been struggling to find an example for this, let's say just validating something simple like {"type" "tweet" "user" {"type" "registered"}}. I would know how to go about doing this from reading the official guide if these were keywords, but not like this
2016:10:06 18:48:46 mlimotte i think you're saying that those are required keys. So, like bfabry said, you can't use s/keys. you have to write your own predicates to do that.
2016:10:06 18:49:57 bfabry @naomarik it depends on what you want to validate. if you know what keys you're going to get in advance and what their semantics are and which ones are required then I'd recommend converting them to keywords. it just makes sense. if the keys are dynamic then I would leave them as strings and validating other properties about the map
2016:10:06 18:55:16 mlimotte @naomarik if it's just required keys, a simple predicate isn't too bad:
If it's more complicated and you need all the features of s/keys, than writing your could be a hassle.
2016:10:06 18:57:43 naomarik ah i see
2016:10:06 18:57:58 naomarik s/keys is a lot more convenient at that point
2016:10:06 18:58:54 bfabry yeah like I said, if you know what keys your map contains I would convert them to keywords. for lots and lots of good reasons including that you get nice spec tools for working with them
2016:10:06 19:00:10 naomarik i always write all my maps as keywords, but is there a discussion online why namespaced keywords are enforced so i can be enlightened?
2016:10:06 19:01:10 bfabry there's been a few, I think maybe the cognicast podcast with rich hickey talking about spec might be the best bet. they're not enforced though
2016:10:06 19:02:01 bfabry s/keys has corresponding :req-un and :opt-un options that you can use for un-namespaced keywords
2016:10:06 19:05:38 mpenet Yeah but some stuff is brittle with un- keys ex broken conforming with s/merge
2016:10:06 19:06:29 bfabry eh?
2016:10:06 19:06:48 mpenet See recent discussion about it in history
2016:10:06 19:07:29 mpenet Clj-1981
2016:10:06 19:08:35 bfabry alex's comments seem to indicate that's confusion over how conforming works with merge, and isn't related to namespaced keys
2016:10:06 19:09:34 mpenet Wrong ticket, my google fu failed me
2016:10:06 19:13:36 bfabry so even though s/keys doesn't specify ::value as req or opt, because ::value is namespaced and has a spec associated it automatically gets conformed and validated
2016:10:06 19:15:09 bfabry so it's not that merge behaves differently with namespaced keys, it's that s/keys behaves differently with namespaced keys (it gives you free stuff if they're namespaced)
2016:10:06 19:19:36 Yehonathan Sharvit Something weird about s/and:
2016:10:06 19:40:13 seancorfield So you want a sequence of integers that is even in length and when you partition that sequence into pairs, each pair is ordered…?
2016:10:06 19:40:28 Yehonathan Sharvit exactly!
2016:10:06 19:42:11 seancorfield So (s/and seq-of-ints length-is-even every-partitioned-pair-is-ordered)
2016:10:06 19:43:11 mpenet @bfabry Yep, thats what I meant. The fact that ns keys are so deeply rooted in spec makes this stuff (merge for instance) counter intuitive.
2016:10:06 19:46:01 Yehonathan Sharvit @seancorfield But I want to do it in an idiomatic clojure.spec way
2016:10:06 19:46:20 bfabry again, I'd say it's (s/keys) that's counter-intuitive, not s/merge, but yeah possibly the automatic conform/validate nature of namespaced keywords could be called out more
2016:10:06 19:46:26 Yehonathan Sharvit Actually, my real need is to build a spec for sequence of integers that are palindromes...
Here is a context-free grammar for syntactically correct infix algebraic expressions in the variables x, y and z:
S → x
S → y
S → z
S → S + S
S → S - S
S → S * S
S → S / S
S → ( S )
2016:10:06 20:08:30 Yehonathan Sharvit Do I have a chance to succeed @seancorfield and @alexmiller ?
2016:10:06 20:10:39 alexmiller I would have written something like this for palindromes (& is much better than and here as it stays in regex)
2016:10:06 21:04:28 Yehonathan Sharvit @alexmiller please help (again) 😰
2016:10:06 21:17:03 alexmiller Sorry I'm off duty for the night :)
2016:10:06 21:22:34 Yehonathan Sharvit No problem: Actually I’m in Israel and it’s 12:30 AM - I’m too tired to continue with this tough topic. We’ll catch up later. If you find something please mention me on slack.
2016:10:06 21:45:13 joshg Is there an idiomatic way to define a spec for a constant value? Something like (s/def ::some-constant #(identical? % "foo"))?
2016:10:06 21:45:55 Yehonathan Sharvit You can use set like this: #{“foo”}
2016:10:06 21:46:12 Yehonathan Sharvit because sets behave like functions
2016:10:06 21:46:12 joshg That works, thanks
2016:10:06 21:46:47 bfabry yes, sets are the idiomatic way to do that, sets also work as generators (while #(identical? % "foo") does not)
2016:10:07 02:26:39 richiardiandrea Hello folks! I know how to spec a map with keys which I don't care the values of: (s/map-of #{:mid :seq} any?) thanks to Alex, but is there a way standard pred to say "any non-nil value" or I must create my own predicate?
2016:10:07 02:28:23 richiardiandrea I tried (complement nil?) but then the generator fails
2016:10:07 03:41:53 seancorfield That probably needs … enhancing ...
2016:10:07 04:10:28 richiardiandrea For generating maps only I was thinking of using an s/and but for a first approximation of this spec some? is perfect
2016:10:07 04:10:39 richiardiandrea That is good investigation though
I expected this not to work because NOT is also a symbol and that none other alt matches the rest
2016:10:07 11:07:02 borkdude Is there a conform variant that throws like s/assert but returns the conformed value?
2016:10:07 12:23:08 alexmiller No but it's easy to make one
2016:10:07 12:23:27 alexmiller @prepor should work fine with aot
2016:10:07 12:24:53 alexmiller @seancorfield: if you filed a ticket on that any? some? generator business I would take a look at it. I keep forgetting to do so
2016:10:07 12:57:38 prepor @alexmiller ok, thank you
2016:10:07 14:27:31 lvh still gotta fix the #{’NOT} if you want generation to work though
2016:10:07 14:27:47 borkdude ah
2016:10:07 14:27:51 lvh also: arrow
2016:10:07 14:27:57 borkdude oh wow, I didn’t even think of generation
2016:10:07 14:28:11 borkdude must. try.
2016:10:07 14:28:59 lvh borkdude It’s pretty cool. I’m currently generating specs for swagger from the swagger json-schema definition, which I then use the generate swagger instances, from which I then generate instances of that swagger schema, which I then validate against the original swagger
2016:10:07 14:29:15 borkdude 😄
2016:10:07 14:29:57 borkdude what reads better, '#{LSHIFT RSHIFT AND OR} or quotes before all symbols?
2016:10:07 14:30:25 lvh errr
2016:10:07 14:30:36 lvh I dunno; I’d go with quotes before syms
2016:10:07 14:35:50 borkdude It works. How do I get s/cat to generate vectors instead of whatever it generates now?
2016:10:07 14:44:53 borkdude How do I get a generator that only generates lower case alphabetic strings?
2016:10:07 14:55:45 nwjsmith
(gen/not-empty
(gen/fmap
(fn [s] (str/replace s #"[A-Z0-9]" ""))
gen/string-alphanumeric))
2016:10:07 14:56:21 nwjsmith that's a really naive generator, but it'll get you there
2016:10:07 15:07:03 borkdude Thanks!
I needed symbols of one or two character, so I got this now:
2016:10:08 15:23:00 jrheard this is fun, don’t know why i didn’t do it when it came out! looking forward to the 2016 edition
2016:10:08 15:23:08 jrheard [assuming i can solve the next 18…]
2016:10:08 15:23:37 borkdude great!
2016:10:08 15:24:23 jrheard i like your use of s/conform, it felt like that was super relevant re: parsing but i guess i just didn’t reach for it; gonna take another look and see how you accomplished it
2016:10:08 15:26:34 jrheard delay’s neat too, haven’t used that before - thanks for sharing your solution, v educational! 😄
2016:10:09 07:01:49 bensu I defined an fdef spec for a macro and I’m testing it with (is (thrown? Exception (macroexpand ‘(my-macro bad-args))))
2016:10:09 07:02:11 bensu when I run it in the repl lein with-profile +test repl the test passes
2016:10:09 07:02:45 bensu but when I run it with lein test the arguments are not checked and the test fails
2016:10:09 07:03:01 bensu Has anybody hit a similar problem?
2016:10:09 07:17:50 hiredman is your macro code actually being loaded? macroexpand doesn't throw an error if my-macro doesn't exist and isn't defined as a macro
2016:10:09 07:29:01 bensu it was getting loaded
2016:10:09 07:29:48 bensu the problem turned out to be related: it was loaded but not recognized by macroexpand because I was not using a syntax-quote
2016:10:09 07:29:59 bensu @hiredman thank you for your help
2016:10:09 09:17:00 Yehonathan Sharvit Sometimes s/conform and s/unform are not inlined. For instance:
2016:10:09 09:18:29 Yehonathan Sharvit Is it a bug in clojure.spec @alexmiller ?
2016:10:09 12:17:42 sander is there a way to get a spec that (s/conform ::my-spec v) will always conform to? I want to fspec a function saying that it takes a spec-destructured value as an argument, and returns another spec-destructured value
2016:10:09 12:22:27 Yehonathan Sharvit did you try s/any??
2016:10:09 12:22:36 Yehonathan Sharvit @sander
2016:10:09 12:23:28 sander I should rephrase, I want a specific spec that only accepts values in the shape that (s/conform ::my-spec v) returns
2016:10:09 12:44:29 alexmiller @viebel yes, generally anytime conform unform returns a different result, that's a bug. This particular one is already logged.
2016:10:09 12:44:55 Yehonathan Sharvit jira?
2016:10:09 12:47:11 alexmiller @sander no, other than saying what you just said as a custom predicate - call s/valid? on the result with another spec
2016:10:09 12:47:36 sander ok, should be easy enough, thanks!
2016:10:09 12:54:58 alexmiller Rich and I talked about this one on Friday in particular - it's tricky
2016:10:09 12:55:14 Yehonathan Sharvit Why is it tricky?
2016:10:09 12:56:06 alexmiller Conceptually tricky about what an optional regex should return in different contexts, how it nests, etc
2016:10:09 12:58:03 Yehonathan Sharvit In my specific case, I was able to workaround this bug using s/conformer. But I still don’t understand fully how s/conformer works. Could you explain in simple words how does s/conformer work?
2016:10:09 12:58:15 Yehonathan Sharvit The docstring is a bit obscure...
2016:10:09 12:59:09 alexmiller It acts as a predicate but applies an arbitrary transformation in the conformed value
2016:10:09 13:00:07 Yehonathan Sharvit What is the simplest use case for conformer?
2016:10:09 13:00:46 alexmiller Applying an arbitrary transformation to the conformed value
2016:10:09 13:01:53 alexmiller It is primarily a tool for building more complex spec variants
2016:10:09 13:03:01 alexmiller It's generally recommended to use sparingly as embedding it in a spec means that you are overriding how spec conforms for all future consumers of your spec
2016:10:09 13:03:37 Yehonathan Sharvit with s/and?
2016:10:09 13:03:49 alexmiller Yes
2016:10:09 13:04:21 alexmiller Like if you used s/or but didn't want a tagged result
2016:10:09 13:05:31 alexmiller When you do that you throw away info about how something was conformed
2016:10:09 13:05:51 alexmiller One use might not need it but other possible future uses may
2016:10:09 13:06:09 alexmiller So baking it into your public spec takes away that option
2016:10:09 13:11:08 Yehonathan Sharvit Thx
2016:10:09 13:11:14 Yehonathan Sharvit I’ll play with that...
2016:10:09 13:45:36 Yehonathan Sharvit @alexmiller another use case would be when one wants to match regex with vectors
2016:10:09 13:45:58 Yehonathan Sharvit The naive code would be:
2016:10:09 15:31:12 jrheard noob question, i’m sure, re: conformer - i reached for conformer when i was operating on data that looked like “123”, but i wanted (s/conform ::my-spec) to parse the integer such that it ended up being 123. what should i use instead of s/conformer in situations like that, for coercion of this sort?
2016:10:09 15:31:26 jrheard i felt like i read about something that would solve this problem idiomatically in the rationale or guide, but couldn’t find it when i went back to look for it
2016:10:09 16:30:51 seancorfield @jrheard: yes I have a lot of cases where I have strings as input but I want them conformed to integers (for API input specs). But @alexmiller has told me a couple of times that is "not idiomatic" and to avoid conformers like that :(
2016:10:09 16:45:51 jrheard hm, i don’t understand - the strings have to be turned into integers at some point, right?
2016:10:09 16:46:18 jrheard is it more idiomatic to have a secondary, non-spec transformation layer that happens after s/conform?
2016:10:09 16:47:04 jrheard i’m sure @alexmiller has good reasons for saying that, but i don’t understand what spec users are actually supposed to do here
2016:10:09 17:44:41 alexmiller The downside is that once you enshrine a conversion into your spec, you have made a conformance decision for all future consumers that has removed information. If you understand this ramification and are willing to live with the consequences, then that's up to you.
2016:10:09 17:46:34 alexmiller There are many ways to apply conversions - you should be aware of when you are conflating conformance and conversion though
2016:10:09 18:01:54 jrheard i think i follow those statements, but a “in situations like that, try doing this instead” example would be supremely helpful 🙂
2016:10:09 19:40:49 alexmiller There's not one answer
2016:10:09 19:41:00 alexmiller You can apply post transformations
2016:10:09 19:41:14 alexmiller You can apply a second spec
2016:10:10 17:19:21 juliobarros I'd like to implement a multi step data pipeline where each step's "spec" is slightly modified and based on the previous step's spec.
For example at step 1 an address has some number of fields and at step 2 it has the same fields plus geo coordinates. Or in step 1 it can have optional string geo coordinates and after step 2 it has to have numeric ones.
It seems like spec requires/encourages you to duplicate base address fields in two separate specs (rather than build on a base spec) and possibly have different names for the string geo coords and double geo coords (since they are registered under the same name). This appears to be more straight forward in schema where I can modify the schema map as I go through the pipeline.
Am I reading this right? Am I missing something? What's the best spec based approach to this? How would you approach this? Thanks in advance.
2016:10:10 17:31:01 hiredman if your changes are purely additive between pipeline steps, you can compose specs using and
2016:10:10 17:35:40 hiredman my perspective on specs is you should treat them like defining columns in a sql database. if I had a column named date in a sql database, would I want the scheme to allow strings or dates? most likely not, I would have different columns for strings and dates.
2016:10:10 17:41:09 juliobarros Thanks @hiredman the sql table analogy is a good way to think about it.
2016:10:10 17:57:27 mattly I think specs that coerce in a conformer are useful, but I've taken to naming them with a ! suffix
2016:10:10 17:59:11 mattly or well, naming the function that the conformer calls with a ! suffix and typically only using them where absolutely necessary
2016:10:10 18:19:11 jrheard seems like a reasonable pattern, thanks for sharing
2016:10:10 18:23:11 Yehonathan Sharvit @mattly don’t forget to pass the 2nd param to s/conformer - otherwise s/unform will fail
2016:10:10 18:23:36 mattly heh
2016:10:10 18:23:47 mattly in this case I'm basically using it to validate environment variables
2016:10:10 23:23:14 bbloom @hiredman indeed the table/column analogy is interesting, but it’s not the whole story. i’ve been thinking about “refinement” (for lack of a better word) a bunch lately
2016:10:10 23:24:00 bbloom specs (and for that matter: types) have an inherit problem when modeling time
2016:10:10 23:24:38 bbloom for a simple example, consider before/after validation of some property
2016:10:10 23:25:27 bbloom you can have :unvalidated/foo and :validated/foo, but it seems like a weird way to go about that if they are going to be identical? to each other in all cases
2016:10:10 23:25:57 bbloom of course, you can always model the validation as coercion, such that you get the validated value or nil, so having two keys makes sense - and indeed that’s what i’d probably do with spec
2016:10:10 23:26:29 bbloom the real problem comes in when you talk about nested structure
2016:10:10 23:27:24 bbloom you don’t want to do O(N) allocations to validate a structure
2016:10:10 23:27:31 bbloom renaming all the keys, losing generality of functions
2016:10:10 23:31:18 hiredman I don't think I follow
2016:10:10 23:32:36 hiredman when I said sql column above, that is because I am more familiar with using sql, but obviously the best analogy is a datomic attribute
2016:10:10 23:33:03 hiredman so stick all your maps in datomic so you can scrub forward and backwards in time on them 🙂
2016:10:10 23:33:16 bbloom heh, sorry, let me try to explain again
2016:10:10 23:35:04 bbloom the ast typing problem is double bad b/c they aren’t modeling things with extensible records
2016:10:10 23:35:32 bbloom but the notion of having two recursive structures that are subtle distinct at different stages in a pipeline is a common theme
2016:10:10 23:36:12 bbloom it sucks to have to do O(N) work (eg copy a whole AST) in order to make some small changes to a subset of nodes
2016:10:10 23:36:48 hiredman isn't that the same argument against immutable datastructures?
2016:10:10 23:37:29 bbloom not quite, it’s more related to the argument in favor of covariant arrays in java/c# 🙂
2016:10:10 23:38:21 bbloom eg you have an array of integers and need an array of objects, so why should you have to copy every integer in order to satisfy the type checker? obviously you don’t have to do that with spec, but it’s still a problem if you have a recursive structure and use different keys for different specs
2016:10:10 23:38:51 hiredman I mean, your options are to write novelty in place, or accrete it
2016:10:10 23:40:00 bbloom sure, you’re totally right - this problem is usually easily avoidable in spec thanks to the dynamic instrumentation and extensible records
2016:10:10 23:42:01 bbloom i just think its interesting to think about refinement of specifications without having to introduce new names
2016:10:10 23:42:38 bbloom there’s definitely use cases to have “changed” structures vs “extended” structures, but yeah - go with the latter whenever possible
2016:10:11 08:02:21 danielstockton I'm calling explain on a spec and it's returning nil, why would that be?
2016:10:11 08:06:57 danielstockton answer: i need to use explain-str, explain just prints to out
2016:10:11 09:05:08 dm3 how would you create a spec for the following:
1) ns store - defines ::state as (s/keys :req [::data ::metadata]) where ::data is a map?
2) ns x produces a ::state with concrete ::data
How would the ::x-state spec be defined? (s/and ::store/state #(s/assert ::x-data (::store/data %))) doesn't look right...
2016:10:11 09:48:53 danielstockton On it's own, [1 1] conforms to the ::pattern spec
#object[Error Error: Unable to construct gen at: [] for: [object Object]]
Error: Unable to construct gen at: [] for: [object Object]
2016:10:11 14:53:42 samedhi So I am trying to reduce the size of generated ::paths to at most be of length 5. It isn’t part of the actual spec for a ::path, but it is something I would like when generating, any idea what I am doing wrong?
2016:10:11 15:00:33 jrheard @samedhi there is a 20% chance that what i’m about to say is relevant, but i’ll say it just in case
2016:10:11 15:00:51 jrheard i’ve run into trouble with s/coll-of specs that define a :kind but not an :into
2016:10:11 15:00:57 jrheard try adding :into [] and see if it does anything
2016:10:11 15:01:02 jrheard it’s probably irrelevant, but who knows
2016:10:11 15:01:30 jrheard this is definitely a thing if your :kind is set?, you need to define :into #{} for generators to work afaict
2016:10:11 15:10:42 samedhi @jrheard is this what you mean? In this case I am trying to return a vector of keywords or strings, so I set :kind to vector? and :into to []
2016:10:11 15:14:44 jrheard yeah, that’s what i mean; looks like it wasn’t relevant, sorry 🙂
2016:10:11 15:22:42 samedhi Hey, no problem, I appreciate it.
2016:10:11 15:40:28 thegeez @samedhi should that be gen/generate instead of s/gen around the gen/such-that?
2016:10:11 15:42:47 samedhi @thegeez, actually, that does work… Let me play a bit with it.
2016:10:11 15:43:32 samedhi On my first post, 2 paste up, I had.
2016:10:11 15:46:11 samedhi > Note that with-gen (and other places that take a custom generator) take a no-arg function that returns the generator, allowing it to be lazily realized.
2016:10:11 15:47:03 samedhi So it sounds like I want a zero-args function that returns a generator.
2016:10:11 15:47:29 samedhi Kind of seems like that is what I have above… right?
2016:10:11 15:48:37 samedhi Maybe gen/such-that is already a generator or maybe it is a spec?
2016:10:11 15:48:56 thegeez remove the s/gen around the gen/such-that
2016:10:11 15:49:42 thegeez gen/such-that is already a generator indeed
2016:10:11 15:55:06 samedhi Thanks, that helped me out a lot, data seemed to just take forever on generating without this. (though that may actually be my formater and not the actual generation). Fixed loading issues.
2016:10:11 17:10:06 samedhi Is actually slightly better, as it never fails to generate under the 100 try limit.
2016:10:11 17:18:40 lvh @alexmiller Suggestion: a special s/cat, primarily intended for specifying args in fdef, that automatically calls s/spec on each of its arguments
2016:10:11 17:18:58 lvh @alexmiller I feel like a lot of people have bumped their toe on the nested regex gotcha
2016:10:11 17:20:07 alexmiller I think that’s unlikely something we would add
2016:10:11 17:21:00 alexmiller and in general, I don’t find that that is (usually) what you want
2016:10:11 17:24:52 alexmiller it seems more common from specs I’ve written to have s/cat of s/coll-of for functions. For macros, you sometimes have this but I find it’s really important in that case to develop the awareness of when you are really needing to drop nested levels - that’s critical to build reusable parts, and often for making workable recursive specs
2016:10:11 18:56:57 danielstockton I think just making this clearer in the guides, in the specing functions section, would help
2016:10:11 18:57:15 danielstockton I found where it's mentioned now, but not when I was trying to work out what was wrong
2016:10:11 18:58:16 danielstockton You could argue I should have read the guide more thoroughly
2016:10:11 21:15:19 bhagany for me, it was just forgetting that cat is a regex operator. I didn’t feel like the guide could have been much clearer about it.
2016:10:12 08:04:05 jmglov Is it possible to spec functions defined in a protocol?
2016:10:12 08:04:26 jmglov Or the implementations thereof in a defrecord or reify?
2016:10:12 08:16:40 hiredman you can fdef a protocol function, and you'll be able to validate it, but instrument may behave oddly
2016:10:12 08:18:36 danielstockton I found instrument doesn't work, I make my protocol functions a simple proxy to another function that I can spec
2016:10:12 10:31:38 danstone Has anyone had any success spec'ing high level components, such as ring handlers? I would like to spec boundaries rather than implementation details in an app I'm working on and it seems like this will be a ton of work.
2016:10:12 11:38:30 jmglov @danielstockton That's what I'm doing for now. It would be really nice if instrument worked, though. 🙂
2016:10:12 11:38:55 jmglov @alexmiller Is that something worth submitting as a feature request? Or is unlikely to be considered?
2016:10:12 11:39:39 jmglov @hiredman So I can do something like this?
2016:10:12 13:11:59 alexmiller I'm not sure it's possible to do
2016:10:12 13:12:18 alexmiller I'm certainly not opposed to it
2016:10:12 14:52:02 jfntn Is there a working equivalent of (s/map-of ::k ::v :kind sorted?) for sorted maps?
2016:10:12 15:13:18 alexmiller There is sorted-map? predicate
2016:10:12 15:13:39 alexmiller I think? On phone so going from memory
2016:10:12 15:22:59 jfntn Hmm I don’t think so, at least in alpha12?
2016:10:12 16:55:59 clojuregeek Trying to use spec, and have a problem with setting up my spec ... basically i want style? to be one of those two values (for now :one :two) ... but it is not working, i've been studying the docs and i can't figure out where i am doing something wrong .
2016:10:12 16:57:27 jrheard @clojuregeek - i think you want to either do (def style? #{:one :two}) and (s/def ::style style?), or just (s/def ::style #{:one :two})
2016:10:12 16:57:58 jrheard i don’t think (s/def) expects you to give it a symbol like style? as the first argument, i think it expects a namespaced keyword like ::style
2016:10:12 16:58:24 jrheard also you have ::style?? in there at one point, which is probably confusing things as well
2016:10:12 16:58:36 bfabry yes, you've registered the spec under the symbol style? instead of under the keyword ::style?
2016:10:12 16:59:03 clojuregeek ok let me try some more ..
2016:10:12 17:00:27 clojuregeek ok i think maybe it was extra ? on style in job not a keyword when i defined style? spec
2016:10:12 17:00:31 clojuregeek thanks guys :+1:
2016:10:12 17:03:27 clojuregeek now if i can only stop typing confirm instead of conform 🙂
2016:10:13 06:52:28 tianshu It seems clojure.spec/assert always return original data in clojurescript.
2016:10:13 22:25:24 danielcompton Is there a way to spec a sequential collection? s/coll-of throws a massive error if you pass a map to it, as it tries to check each mapentry against a spec and displays a failure for each mapentry
2016:10:13 22:26:26 jrheard s/map-of might be useful for you
2016:10:13 22:26:51 danielcompton I want my spec to be a sequence of values, each conforming to a spec
2016:10:13 22:27:06 danielcompton If I pass a map, then each mapentry gets evaluated against that spec instead
2016:10:13 22:27:11 jrheard huh!
2016:10:13 22:27:40 jrheard would you mind pasting an example?
2016:10:13 22:29:04 jrheard s/cat is a good way of speccing sequences, but you’re probably already familiar with it so i hesitate to mention it
2016:10:13 22:29:20 jrheard i haven’t seen the behavior you describe re: coll-of and maps
2016:10:13 22:29:24 jrheard am very curious 🙂
2016:10:13 22:30:12 danielcompton I thought about s/cat but didn't think it was for this use case? I could be wrong though
2016:10:13 22:30:44 jrheard if i’m reading this right, you’re passing in {a-map}, and not [{a-map}] - could that be the issue?
2016:10:13 22:31:54 danielcompton I know what the issue is, I'm saying that the spec error message telling me that wasn't very useful when I used s/coll-of, so I was wondering if there was something else?
2016:10:13 22:43:56 alexmiller coll-of seems like what you want although I'm not entirely positive I understand what you're doing
2016:10:13 22:53:27 danielcompton I want my spec to be a sequential? collection of ::request-maps. I define ::request-map, then create (s/def ::request-maps (s/coll-of ::request-map)). If I pass "abc" and check it against my spec, I get
as it fails on the coll? predicate. This is a pretty easy to understand problem.
However if I check a single map against this spec, I get a very large error, because the map is treated as a sequence, and each MapEntry in the map is checked against (and fails) my spec ::request-map.
2016:10:13 22:55:02 danielcompton I can change my spec to (s/and sequential? (s/coll-of ::request-map)) but this seems like a fairly common use case, and wondered if there was another way to spec this more precisely?
2016:10:13 23:19:02 hiredman use the regex ops, + or *
2016:10:13 23:19:19 hiredman (s/* ::request-map)
2016:10:13 23:20:49 hiredman (I mean, I don't know, that is just a suggestion, seems nicer than coll-of)
2016:10:13 23:27:53 danielcompton That's a bit better, thanks. It short circuits earlier, but the error message is still not very illuminating:
2016:10:13 23:28:24 danielcompton It still treats a map as a sequence of map entries
2016:10:13 23:38:54 alexmiller I think coll-of is the best match for what you are trying to say
2016:10:13 23:40:36 alexmiller Using s/* by itself without other regex ops is less good
2016:10:13 23:41:01 alexmiller As it conveys a sequential with more interesting internal structure
2016:10:13 23:41:07 danielcompton Does it make sense to include maps as valid coll-of's?
2016:10:13 23:41:17 alexmiller Yes
2016:10:13 23:41:26 alexmiller They are a collection of tuples
2016:10:13 23:41:38 alexmiller And that's very useful
2016:10:13 23:42:27 danielcompton Is there scope to add a sequential-of? or something similar?
2016:10:13 23:42:49 alexmiller No, you could use :kind though
2016:10:13 23:43:09 alexmiller :kind sequential?
2016:10:13 23:43:16 danielcompton Nice!
2016:10:13 23:43:20 danielcompton That's wha I was after
2016:10:13 23:43:33 alexmiller You may also want :into []
2016:10:13 23:44:08 alexmiller Can't remember how it will conform without that
2016:10:13 23:44:25 alexmiller That might already be the default
2016:10:13 23:44:41 danielcompton I think so
2016:10:13 23:45:29 alexmiller That's used by gen too if you care about that so might double check that too
2016:10:13 23:46:55 danielcompton docs seem to suggest an :into is needed as well here, as I don't know if sequential? can generate?
2016:10:14 01:39:24 alexmiller I don't know - try it? You can specify a generator too
2016:10:14 02:02:17 danielcompton yep, generating works fine with :kind sequential?
2016:10:14 15:04:39 roberto fyi, snippets slow down slack a lot. Maybe we should default to creating posts instead of snippets.
2016:10:14 15:29:03 samedhi I keep having to with-gen or overspecify my generator in clojurescript as generation just takes too long and eventually times out.
2016:10:14 15:29:35 samedhi Specifically coll? as a :args always causes me to eventually time out
2016:10:14 15:29:41 samedhi Anything clever to fix this?
2016:10:14 15:30:52 samedhi Oh, I mean when using clojure.spec.test/check.
2016:10:14 15:37:49 samedhi
> (defn fx [coll] coll)
> (s/fdef fx :args (s/cat :coll coll?))
> (stest/summarize-results (stest/check `fx))
2016:10:14 15:37:54 samedhi As an example of timing out in cljs.
2016:10:14 15:39:07 alexmiller Have you tried using the :gen-max option?
2016:10:14 15:39:17 samedhi nope
2016:10:14 15:39:27 alexmiller For spec'ing collections that is
2016:10:14 15:40:01 samedhi yeah, I saw one example of that in the spec getting started guide, forgot about it.
2016:10:14 15:40:02 alexmiller You could use (s/coll-of any? :gen-max 3)
2016:10:14 17:00:29 spieden i’m looking at doing form validation using spec. any thoughts on mapping from an explain to a slightly more human readable messages to put under fields? can’t find anything to put metadata on
2016:10:14 17:01:38 spieden maybe just a map from spec keys to messages would do
2016:10:14 17:06:37 spieden hmm, looks like i’d need to key off the pred symbol actually
2016:10:14 17:25:57 samedhi @alexmiller Thank you for your help. I played around with it since then. Actually, the real issue was that I was including a spec in the :ret part of my s/fdef. This causes the value in :ret in the :fn part of the same s/fdef to be the spec-conformed-return version of the :ret, not the function-value-return version of the return. Kept causing failures that I just couldn’t understand. 🙂
2016:10:14 17:27:44 samedhi tldr; If your spec does not conform to the identity of the value passed to it, you should think about what you are doing when using it in s/fdef’s :ret
2016:10:14 17:31:49 spieden hrm, wrapping it in s/and allows you to retrieve the “ground” spec:
2016:10:14 17:32:45 spieden (difference is in :via values)
2016:10:14 19:21:43 samedhi @spieden Thanks you for letting me know, I did reach that solution as well. It all makes pretty good sense, it just caught me by surprise that :ret also effected the :ret in :fn.
2016:10:15 20:59:47 samedhi Oh, you mean within it
2016:10:15 21:00:48 samedhi Yes, it does state it takes a spec, but it should “contain predicates"
2016:10:15 21:00:54 samedhi innnteresting
2016:10:15 21:01:00 gfredericks I would wager "A spec of the relationship..." is a typo, unless things have changed recently
2016:10:15 21:01:08 gfredericks I'll check on that actually
2016:10:15 21:01:14 gfredericks what's the latest clojure version
2016:10:15 21:01:24 samedhi 1.9.0-alpha13 I think?
2016:10:15 21:02:07 gfredericks looks like there's 13
2016:10:15 21:02:18 samedhi Yeah, I just checked and saw that. 🙂
2016:10:15 21:04:31 samedhi > Note that :fn specs require the presence of :args and :ret specs to conform values, and so :fn specs will be ignored if :args or :ret are missing.
2016:10:15 21:05:18 samedhi Ugh, that line alone would have saved me hooours. I don’t know what you can do for people who don’t read the api docs (sorry), but figuring this out on my own just hurt.
2016:10:15 21:06:08 samedhi Maybe a warning if you include a :fn but not :args and :ret
2016:10:15 21:13:01 gfredericks looks like the s/and thing does work, and I don't know why
2016:10:15 21:13:04 gfredericks so TIL I guess
2016:10:15 21:15:36 samedhi @gfredericks Sounds good, yeah, no idea. Thanks for you help. TIL I should always read the docs right after going through the getting started section 😬
2016:10:16 05:22:38 samedhi I get the behavior, but I wanted to fdef check the metadata after a function call in the :fn. However, the :fn only has the conformed return values, so I don’t have the metadata. Any ideas how to do this. Or would you just say I shouldn’t include it in metadata?
2016:10:16 13:53:18 ggaillard Hello everyone! I am going crazy on a cljs.spec strange behavior.
I use (s/fdef) and (cljs.spec.test/instrument) in my dev mode to catch non-conform arguments. For a strange reason my spec-ed functions keep beeing called as if a generator was forever running. I have cljs.spec.test in my dependencies but I don't call it anywhere at the moment… any idea why ?
2016:10:16 15:37:56 jetzajac Hi! How do I spec named arguments of a function/marco?
2016:10:16 16:10:24 alexmiller cat
2016:10:16 16:35:11 gfredericks jetzajac: s/keys?
2016:10:16 16:35:21 gfredericks depends on what you meant by "named arguments" I guess
2016:10:16 17:16:00 jrheard @ggaillard i have the same problem using cljs.spec.test/instrument! i’m glad it’s not just me
2016:10:16 17:16:03 jrheard i can’t reproduce it reliably though
2016:10:16 17:16:27 jrheard i really hope you’re able to figure out what’s going on and open a jira issue
2016:10:16 17:16:32 jrheard it’s been driving me nuts
2016:10:16 17:20:07 ggaillard I'm not crazy! Thank you! ^^
I can't manage to reproduce it too. I'm building a re-frame app and this situation always appear after a page refresh, when React tries to render some spec-ed Reagent components. I searched deeply into tens of stacktraces but found nothing particular 😞
2016:10:16 17:22:07 jrheard and does not include re-frame
2016:10:16 17:23:07 jrheard but again i can’t figure out what’s going on, mainly because the behavior is inconsistent so coming up with a repro is maddening/difficult
2016:10:16 17:39:12 alexmiller @jetzajac: oh if you mean kwargs then cat and a nested keys* for the options
2016:10:16 17:50:22 jetzajac @alexmiller: thanx! this is what i was looking for
2016:10:16 19:15:44 jrheard (the first one's just a documentation issue)
2016:10:16 19:18:30 ggaillard I struggled with the first one (ended up looking the source). Thanks a lot for the second one !
2016:10:17 10:53:13 akiel Hi. Is there something like s/assert which is executed every time? I like to reserve s/assert for checks I want potentially disable in production. But I have some checks I like to execute in all cases.
2016:10:17 11:04:55 Niclas How would one go about specing a function that takes a callback that may take any kind of argument? For ex:
(defn myfunc [k cb]
(do-something-async k (fn [res err]
(check-err err)
(cb res err))))
2016:10:17 11:05:59 akiel Why takes the callback any kind of argument?
2016:10:17 11:07:35 Niclas If do-something-async in this example is an un-speced library function where I only care about specing functions in my project and not external ones
2016:10:17 11:09:25 akiel But if you know what cb should look like, it would be beneficial to fail fast at your function.
2016:10:17 12:31:00 alexmiller You can use fspec to spec a function arg and any? to spec an arg that takes any value
2016:10:17 15:34:50 Niclas @alexmiller Thanks, that’s what I was looking for!
2016:10:17 15:45:37 vikeri Should I be able to spec an infinite sequence as an argument to a function? I tried s/every but it got stuck in an infinite loop.
2016:10:17 16:02:15 alexmiller s/every samples so that should be able to work. Would be interested in seeing more.
2016:10:18 19:55:47 decoursin Is there a way to stub clojure.spec.test/check?
2016:10:18 19:56:50 decoursin
(clojure.spec.test/check `foo {:stub #{`bar}})
doesn't work
2016:10:18 20:06:03 donaldball Has anyone used clojure.spec at runtime, e.g. to validate the arguments of fns at system boundaries?
2016:10:18 20:20:34 seancorfield Yes, that’s how we use it @donaldball
2016:10:18 20:21:28 donaldball What’s your technique? Do you instrument the namespaces or fns about which you care, or manually check the args using a precondition or e.g. a custom defn macro?
2016:10:18 20:22:09 alexmiller @decoursin do that with instrument prior to the call to check
2016:10:18 20:24:48 seancorfield @donaldball We explicitly call conform and see whether we get invalid? data back, since we need to map to a set of known error codes that our API returns.
2016:10:18 20:25:30 seancorfield In a couple of places we use explain-data to help figure out which error code to return (but most are simply "parameter X was not valid").
2016:10:18 20:26:51 seancorfield Most of our specs are data structure specs. We have some function specs. We call instrument when we’re testing, and we have a few places where we run check explicitly as well. Mostly we’ve been doing the latter manually (in the REPL) so far.
2016:10:19 09:16:20 Niclas How does one set the default number of tests to run with cljs.spec.test/check?
2016:10:19 12:31:00 bhagany hopefully I didn’t misunderstand you there - if you’re looking for how to specify the number of tests to run when you invoke check, you’d pass an options map that looks like {:clojure.spec.test.check/opts {:num-tests 500}}
2016:10:19 12:44:40 decoursin @alexmiller Thanks Alex! I'll try next opportunity
2016:10:19 12:51:40 Niclas Thanks @bhagany! Yes, that was my understanding too from the docs, but the call doesn’t seem to take the option into account when I test it locally:
2016:10:19 15:52:32 jrheard @dnolen - this is what patch 1 on that issue fixes
2016:10:19 15:52:45 jrheard deleting patch 2, pretend you never saw it
2016:10:19 15:57:31 alexmiller would be nice to get that fixed
2016:10:19 16:08:40 dnolen @jrheard applied to master
2016:10:19 16:08:48 jrheard 😮😮😮
2016:10:19 16:08:53 jrheard thanks! 🙂 my first cljs commit!
2016:10:19 16:16:34 dnolen @jrheard nice 🙂
2016:10:19 20:34:57 Tim anyone using clojure-future-spec ?
2016:10:19 21:23:20 darwin @tmtwd yep, me
2016:10:19 21:26:08 liamd to use a spec in another namespace do you have to require anything?
2016:10:19 21:28:39 borkdude yes, we are using it too at work
2016:10:19 21:49:56 jrheard @liamd - my understanding of the world is that you need to either require the namespace that the spec was defined in, or require another namespace that requires that namespace
2016:10:19 21:50:47 jrheard like, the (s/def ::foo) line is a side effect, and so in order for you to use ::foo somewhere else, you need to ensure that that side effect was run
2016:10:19 21:50:56 jrheard this could be a poor mental model to use, but it’s the one i’m currently using 🙂
2016:10:20 02:16:59 alexmiller That's right
2016:10:20 02:17:41 alexmiller s/def updates the registry so that needs to occur before you can use the spec
2016:10:20 09:12:29 mpenet how do you wrap a function that returns values to be used as gen that don't depend and randomisation at test.check level (it's an external lib that already does this (sentence generation))
2016:10:20 09:13:17 mpenet I know I can (gen/fmap (fn [_] (some-lib/foo)) (gen/any)) but that seems a bit too convoluted
2016:10:20 11:34:47 alexmiller gen/return ?
2016:10:20 11:35:12 mpenet I don't think so gen/return will always return the same value
2016:10:20 11:36:45 alexmiller The fact that foo doesn't rely on test.checks notions of randomness is actually a problem (as it can't play in growing, shrinking, or creating reproducible results)
2016:10:20 11:37:14 mpenet I know it's a bit "dirty"
2016:10:20 11:37:17 alexmiller But I think you could also use gen/bind to do this
2016:10:20 11:37:32 mpenet bind is from generator to generator, I am not sure this works too
2016:10:20 11:38:05 alexmiller Yeah, I'm thinking of it as a gen
2016:10:20 11:38:45 mpenet I am not sure I understand, you have an example?
2016:10:20 11:39:27 alexmiller I guess its not any better than what you had with fmap
2016:10:20 11:40:27 alexmiller I mean you could directly satisfy the test.check requirements to be a generator
2016:10:20 11:40:44 alexmiller I think it's just a record with a function field
2016:10:20 11:41:02 alexmiller Something like that
2016:10:20 11:41:31 mpenet yep I saw that too, but it's almost the same as the fmap solution in the end
2016:10:20 11:41:36 mpenet it's a 2 arg func
2016:10:20 11:41:59 mpenet doesn't feel right to use it either, the "make-gen" function is private for instance
2016:10:20 11:42:32 alexmiller Well you're starting from the position of doing something you shouldn't :)
2016:10:20 11:42:42 mpenet true
2016:10:20 11:42:52 alexmiller So it's going to feel wrong whatever you do
2016:10:20 11:42:55 mpenet it's probably not an uncommon need tho
2016:10:20 11:43:08 alexmiller First time I've seen it asked
2016:10:20 11:43:19 mpenet but I can live with the "hack" for now
2016:10:20 11:43:57 alexmiller Since you're not going to use the value, I'd pick a simpler generator than any?
2016:10:20 11:44:08 alexmiller If you're doing the fmap one
2016:10:20 11:44:40 mpenet right
2016:10:20 11:44:44 alexmiller boolean?
2016:10:20 11:45:08 mpenet good idea
2016:10:20 11:45:31 mpenet or even (gen/return nil)
2016:10:20 11:45:41 mpenet but it doesn't make much diff
2016:10:20 11:45:41 alexmiller Yeah
2016:10:20 11:45:44 mpenet 😄
2016:10:20 12:23:32 mpenet is there a way to increase "randomness" of num generators in test.gen?
2016:10:20 12:23:46 mpenet it's often just +-1
2016:10:20 12:24:03 mpenet well, I guess it's just a matter of gen size actually
2016:10:20 12:24:13 mpenet nevermind
2016:10:20 12:25:56 odinodin Does anyone have any tips on how to use clojure.spec/keys on Datomic entities? Looking for a workaround for the following http://dev.clojure.org/jira/browse/CLJ-2041
2016:10:20 12:37:33 alexmiller I don't know that there is a solution to 2041 btw
2016:10:20 12:38:11 odinodin currently hurting us bad, since we work with entities all over, treating them as maps
2016:10:20 12:38:44 alexmiller The lighter weight lookup interfaces do not have a way to iterate over entries which keys needs to do
2016:10:20 12:39:31 alexmiller I know this has been discussed re Datomic but I'm not sure where things will end up
2016:10:20 12:39:31 odinodin right, also guess conform would call assoc as well
2016:10:20 21:10:04 liamd is there a spec equivalent of annotating types using the s/defn, etc. macros in schema with :- and all that?
2016:10:20 21:23:54 liamd or do i just use fdef
2016:10:20 21:31:31 liamd also, so the name of a spec is a key that corresponds to what it must be called in a map, but also is just the name of a thing if you’re using it not in a map?
2016:10:20 21:45:43 liamd so how do i spec a map where the keys are strings?
2016:10:20 21:59:35 jasonjckn use conformer
2016:10:20 22:55:17 bfabry @liamd s/keys requires keyword keys for validating maps. if you don't have keyword keys, you'll need to use map-of, coll-of, custom predicates etc
2016:10:20 22:55:29 bfabry and yes, use fdef
2016:10:20 23:32:10 jasonjckn one of my dependencies is broken ""call to clojure.core/defn did not conform to spec""
2016:10:20 23:32:17 jasonjckn is there a way to disable spec checks?
2016:10:20 23:33:36 seancorfield Most of the libraries "broken" by recent spec versions have already released new versions. What is breaking?
2016:10:20 23:34:09 jasonjckn a few different deps, i'm trying to upgrade them now, i'll let you know if it's still broken
2016:10:20 23:35:27 jasonjckn also fyi the error mesasge makes it very hard to know who the culprit is
2016:10:21 09:01:54 mpenet I did it wrong again: (gen/large-integer* {:min 1e6 ...}) -> Doubles (my fault I know), but maybe there could be an assert (or casting) there
2016:10:21 09:37:09 pyr @mpenet, thanks!
2016:10:21 09:40:27 mpenet Is it possible to "share" a generator value for multiple keys in a map (or values in a spec) -> ex: a user profile with N fields?
2016:10:21 09:43:31 mpenet time to look up gen overrides I guess
2016:10:21 11:17:05 jetzajac Hi! How do I override generators for test/check? doc says we can pass :gen parameter in there, but can’t find any example. passing something like (stest/check split-operation {:gen {:onair.ot/count (s/gen :onair.ot/bounded-count)}}) shows that it is not generator which is expected as a value, but what then?
2016:10:21 11:18:28 jetzajac getting error like clojure.test.check.generators.Generator cannot be cast to
clojure.lang.IFn
2016:10:21 11:20:30 jetzajac the problem I’m trying to address is actually testing for interger overflow, I don’t care about it, but generator for int passes huge numbers to a function, which causes it to fail with overflow exception. bounding value with predicate in spec causes a fail in conforming results =(
2016:10:21 18:01:55 hiredman clojure.spec.gen wraps clojure.test.check.generators, and the way it wraps it results in generators becoming no argument functions that return generators (this is to make the dependency on test.check optional). It can make it kind of confusing about when to use a test.check generator and when to use a function returning a generator, most notably when using the generator combinators in clojure.spec.gen
2016:10:21 18:33:09 alexmiller the answer is to always use a function returning a generator
2016:10:21 18:33:43 alexmiller I only find it to be confusing when using both spec.gen and test.check.generators at the same time
2016:10:21 18:33:50 alexmiller so I mostly try not to do that :)
2016:10:21 19:37:25 liamd i’m trying to s/exercise a function i spec’d and i’m getting:
ExceptionInfo Unable to construct gen at: [] for:
what does it mean “`at: []`"
IllegalArgumentException No implementation of method: :specize* of protocol: #'clojure.spec/Specize found for class: nil clojure.core/-cache-protocol-fn (core_deftype.clj:583)
=/
2016:10:21 23:19:50 jrheard @kenny i haven’t seen official guidance on this. have you seen the summarize-results function, though?
2016:10:21 23:20:13 kenny Yes
2016:10:21 23:20:21 jrheard i’ve seen people assert on its return value in regular deftests
2016:10:21 23:21:26 kenny I wrote a version of defspec that uses abbrev-result. So you could do something like
(defspec-test qc-myfn? `myfn)
2016:10:21 23:21:31 jrheard nice
2016:10:21 23:21:47 jrheard haven’t seen abbrev-result
2016:10:21 23:21:50 kenny It works for now. I'm curious what the official solution will be
2016:10:21 23:22:38 kenny Also combined it with clojure.spec/explain-out
2016:10:21 23:23:34 kenny clojure.test + generative tests don't fit perfectly -- it's not clear what the :expected and :actual values in the clojure.test report should be.
2016:10:21 23:24:20 kenny In my case I used the :ret for :expected and :clojure.spec.test/val for :actual
2016:10:22 16:12:02 cddr Has anyone thought much about using spec definitions to generate avro/protocol buffer/thrift schemas (and corresponding coercion of data in those formats into clojure and back) or is that a fools errand?
2016:10:22 17:04:28 cddr I think what I really want to do is associate specs (or perhaps conformers) with "hints" that would document the low-level type implied by some conformer.
2016:10:22 19:32:53 donaldball I have had modest success using them as parts of fixed-width file parsers
2016:10:23 00:52:01 cddr Does your approach require the generation of some sort of schema artifact representing the on-disk representation of your data?
2016:10:23 22:38:14 bfabry @cddr I fully intend to be using something that does spec->avro schema within the next 6 months. either someone else will write it and I'll use it or I'll write it
2016:10:23 22:50:28 cddr Would you want it to infer the avro type from the spec? Or would you be happy to annotate the avro part.
2016:10:24 07:37:19 mpenet @bfabry: In my small experience with this stuff specs is too "open" to allow conversions like this without hitting tons of problems down the road. Prolly better to limit yourself to a subset of specs, and maybe front all this will a small DSL that would avoid people to go wild with theirs spec
2016:10:24 22:35:41 bfabry @mpenet yeah, I definitely wasn't planning on trying to create a "complete" spec->avroschema thing. just register arbitrary predicates against their avro conversion
2016:10:25 10:48:55 pvinis what is the idiomatic way to write specs? in a spec directory? how do you run them? is there a lein spec like lein test? how do you guys do that? i guess specs are not just to be run in a repl while developing..
2016:10:25 12:39:59 alexmiller Specs are code. You can either put them inline with your other code or in a separate namespace.
2016:10:25 12:40:16 alexmiller You can use them in a variety of ways
2016:10:25 12:41:13 alexmiller If you want to use them in your production code you can call s/conform or s/valid?
2016:10:25 12:41:50 alexmiller If you want to be able to toggle them on or off you can use s/assert
2016:10:25 12:43:11 alexmiller If you want to use them during testing you can use stest/instrument (to check calls to a function) and stest/check to generatively test a spec'ed function
2016:10:25 18:21:02 donaldball I think I may have run into a bug in conform:
2016:10:25 18:22:56 donaldball Easy enough to work around using s/tuple instead of s/cat, but figured I’d mention it here. Should I file a bug report?
2016:10:25 18:48:50 alexmiller no, that’s expected behavior
2016:10:25 18:49:09 alexmiller you can pass :conform-keys true as additional map-of options to get both conformed
2016:10:25 18:49:52 alexmiller default is false (as often you don’t need it, and there’s a perf cost, and there’s the possibility of conformed keys overlapping and losing data in the conformed map)
2016:10:25 18:50:23 alexmiller this is in the docstring for map-of too
2016:10:25 18:51:03 alexmiller I’m assuming that you were objecting to the vals being conformed but not the keys
2016:10:25 18:51:45 alexmiller if you actually want neither conformed, you can do that with a conformer on the val spec
2016:10:25 18:52:29 alexmiller or change it to use tuple as you suggested
2016:10:25 22:10:50 olivergeorge Hey @alexmiller, I think there's a bug in (s/form (s/every pred ...)). (s/form (s/coll-of (s/keys :req-un [:PROJ_CONTACT/NAME]))) returns a pred of s/keys instead of clojure.spec/keys.
2016:10:25 22:11:48 alexmiller yes, there’s a ticket and a patch waiting for Rich already on that
2016:10:25 22:19:20 alexmiller that patch was super annoying with all the macrology - it’s hard to tell but I put a lot of work into making that as small a change as it is
2016:10:25 22:19:58 olivergeorge That code is quite tricky.
2016:10:25 22:20:39 olivergeorge While you're about. I'm interested in finding a good approach for walking through specs to transform. In my case I'd like to use specs to produce a simple version of a datomic pull query.
2016:10:25 22:21:05 olivergeorge I suspect there's a slightly more artful approach. Any chance you can point me at an example.
2016:10:25 22:24:12 alexmiller so one thing that is coming (and is behind all my spec form fixes) is specs for s/form’s return, basically specs for spec forms, which is the path you’re on there
2016:10:25 22:24:52 ericfode Has anyone tried using test.check cljs.spec and core.async together?
2016:10:25 22:25:02 alexmiller @olivergeorge so I think using spec to conform spec forms is imo a good approach
2016:10:25 22:27:13 olivergeorge @alexmiller Thanks. The spec form specs (!) sound useful. Perhaps those could be the basis for post-walk wrapper or zipper. I'm thinking that might make walking/transforming the spec tree less quirky.
2016:10:25 22:28:28 olivergeorge Nice to know i'm on the right track. Much appreciated.
2016:10:25 22:40:39 donaldball Thanks, alexmiller, I should have checked the docstring first.
2016:10:25 22:46:49 gfredericks is (s/keys) the best way to express s/map-of-style varargs?
2016:10:25 22:47:16 gfredericks assuming my key-spec is keyword? of course
2016:10:25 22:50:38 bfabry @gfredericks s/keys* is for varargs
2016:10:25 22:51:14 bfabry assuming you meant what I thought you did by that
2016:10:25 23:02:44 gfredericks yeah I did
2016:10:25 23:02:50 gfredericks in particular I mean (s/keys*) with no args
2016:10:25 23:14:12 alexmiller Why no args?
2016:10:25 23:14:41 alexmiller I mean, that's fine but opt-un is useful on the gen side
2016:10:26 00:21:37 gfredericks @alexmiller it would be better as a s/map-of but afaik I can't use that in a regex
2016:10:26 00:21:50 gfredericks I don't have any explicit keys
2016:10:26 00:23:12 alexmiller Well then sounds good
2016:10:26 00:23:53 alexmiller Any registered keys will get checked
2016:10:26 01:20:39 gfredericks that would be bad actually
2016:10:26 01:21:01 gfredericks I'm trying to spec gen/hash-map
2016:10:26 01:21:32 gfredericks come to think of it I can just do (s/* (s/cat :k keyword? :gen gen?))
2016:10:26 01:21:50 gfredericks and that's perfect
2016:10:26 13:34:13 gfredericks So I'm specing some macros
2016:10:26 13:35:11 gfredericks And it seems common for there to be macro args where there's a straightforward spec for the runtime value, but the macrotime form could be almost anything
2016:10:26 13:36:51 gfredericks So my first thought is to use any?, but that's unsatisfying since it doesn't tell users anything and doesn't even try to catch problems evident at macroexpansion time
2016:10:26 13:37:16 gfredericks But anything fancier would be somewhat complex
2016:10:26 13:37:26 gfredericks So I'm wondering if it's worth it
2016:10:26 13:37:47 gfredericks Or if we should just expect macro specs to have a lot of any?s in them
2016:10:26 13:41:57 alexmiller yes
2016:10:26 13:42:10 moxaj @gfredericks I think your macro specs shouldn't be (can't be?) concerned about runtime semantics
2016:10:26 13:42:44 alexmiller I’ve spent a fair amount of time looking at this and in general, it’s quite common for :ret to not be worth spec'ing
2016:10:26 13:42:57 alexmiller and in that you should just omit it
2016:10:26 13:43:31 alexmiller currently macros are skipped during check and instrument only checks :args, so there is not much value in spec’ing :ret anyways beyond docs
2016:10:26 13:44:39 alexmiller for :args where some can be scoped but others can’t, I think use of any? is fine (you can see some of this in the specs I’ve done in clojure.core.specs)
2016:10:26 13:45:07 alexmiller and then consider what you are expanding to - sometimes it’s better to spec that instead (particularly if it bottoms out at a function)
2016:10:26 14:23:52 gfredericks That makes sense; thanks
2016:10:26 15:11:35 quoll Is there a suggested approach to building specs in Clojure-1.8 code in anticipation of the 1.9 release? The “alpha” label puts managers off systems that are rapidly going into production, but I would rather be writing specs as I write code, rather than trying to go back to older code and updating it after 1.9 comes out.
2016:10:26 16:56:03 seancorfield @quoll Any chance that you can persuade them that Clojure 1.9 Alpha = Clojure 1.8 + spec?
2016:10:26 16:58:09 alexmiller not to shoot down a direction towards more spec usage, but that is not exactly true and will become less so
2016:10:26 16:59:35 alexmiller 1.9 is still alpha for good reasons and will not go beta until there are a few more things that have been stabilized. It is still reasonably likely that there will be significant changes in the spec internals.
2016:10:26 16:59:58 alexmiller most of that would probably not affect the api but it may affect the idea of forms as the information model for specs
2016:10:26 17:00:25 alexmiller there is a lot of concern that we get certain parts of spec “right” as they will be very difficult to change later
2016:10:26 17:01:45 alexmiller so if you’re worried about investing in something that might change, then your managers are right and you should hold off :)
2016:10:26 17:02:38 alexmiller if you’re worried about using alpha software because it may have known and significant bugs, you should also hold off, because there are some :)
2016:10:26 17:04:06 alexmiller 1.9 beta 1 to us signals that the first part (deciding what features to include and provide) is over
2016:10:26 17:04:20 alexmiller 1.9 rc1 signals that we believe the major bugs have been fixed
2016:10:26 17:08:37 alexmiller That said, I think the two approaches are to use clojure-future-spec with 1.8 or to separate your specs from your core so that you can build and deploy with 1.8 but test with 1.9 if you like
2016:10:26 17:30:10 seancorfield Understood. My point was more that clojure.spec is (almost) the only new thing in 1.9 so the risk of using the current alphas is actually very low, as far as existing 1.8 behavior is concerned.
2016:10:26 17:30:34 seancorfield I like the suggestion to have a 1.8-buildable system and an option to test against 1.9 with spec on top of that.
2016:10:26 17:32:07 seancorfield For us, we’ve used Clojure Alpha builds in production dating back to 1.3 Alpha 7 or 8 in 2011 and we’ve found it amazingly stable. We find the benefits of being able to leverage new features far outweighs the risks and I think we’ve only had to roll back one or maybe two prerelease builds in those five years.
2016:10:26 17:32:17 alexmiller @seancorfield we’ve had several regressions related to the changes in destructuring and namespaced map syntax
2016:10:26 17:33:01 seancorfield Yup, and there are often small regressions or breaking changes in each release but those have either not affected us or been very minor impact issues.
2016:10:26 17:35:35 seancorfield I can certainly understand some companies being alpha-averse tho’. With some of the projects I maintain, I’ve encountered companies who won’t even consider an RC build. I find the most risk-averse companies tend to be the ones with the worst test coverage and therefore more likely to be unable to detect a regression in their own code… 😐
2016:10:26 17:35:56 seancorfield I think the Clojure/core team has done an incredible job of stability with Clojure releases over the years.
2016:10:26 17:37:49 seancorfield We always run our test suite against both our regular selected Clojure release and against Clojure master snapshots to catch any new breaking changes early.
2016:10:27 02:46:29 cddr I sense that there is some profound wisdom in the "informational vs implementation" section of the spec overview but I fear it goes over my head. Is it a hint to users to not do something with spec that might seem tempting? Or is it a note to implementors to stay away from problems that cannot be solved in the general case.
Could anyone provide an example of an implementation decision that should not go into a spec.
2016:10:27 02:51:39 cddr Also it seems like most people talking about it here are using it to spec inputs/outputs of functions and macros. Is anyone using (or planning on using it) to define system boundaries? The note in the same doc about starting a dialog about semantic change seems particularly relevant to this type of usage.
2016:10:27 03:07:33 seancorfield @cddr We're using it to spec system boundaries more than function inputs/outputs.
2016:10:27 03:08:00 cddr How do you share the specs between the two systems?
2016:10:27 03:08:11 seancorfield We're spec'ing our REST API at the user boundary, the domain model boundary, and the persistence boundary.
2016:10:27 03:08:33 seancorfield Right now, those are still in a single process but that is changing.
2016:10:27 03:09:54 seancorfield We keep the specs mostly separate from the code. We haven't completely decided how to package that up in libraries for sharing across systems, but we expect to have the spec libraries as dependencies to both systems (everything's Clojure).
2016:10:27 03:10:33 cddr Those categories are interesting too. Is there any overlap between domain model and persistence for example?
2016:10:27 03:10:35 seancorfield We're mostly focusing on data -- and generators -- rather than functions.
2016:10:27 03:11:48 seancorfield @cddr We're still looking at whether we can (mostly) derive one from the other... we've had some success with that but there are... special cases... shall we say.
2016:10:27 03:13:06 seancorfield Some parts of our API boundary match our domain model better than others 🙂
2016:10:27 03:14:26 cddr Yeah, it seems like it would be easy to generate a spec from some sort of schema like avro, but a bit harder to go the other way.
2016:10:27 03:15:02 cddr Thanks for your thoughts. Are you going to be at the conj? I'd love to chat some more about this.
2016:10:27 03:15:18 seancorfield We are started with a legacy API and evolving a new one, piecemeal, so we don't have the luxury of anything like Swagger at the moment etc.
2016:10:27 03:16:01 seancorfield Yeah, I'll be there. I submitted a talk about this topic -- and it got accepted -- but partly for personal reasons I've had to withdraw it.
2016:10:27 03:16:21 cddr Ah well look forward to seeing you then. 🙂
2016:10:27 03:16:42 seancorfield I love Austin 🙂
2016:10:27 12:19:08 ikitommi @seancorfield @ccdr we have been writing the spec->swagger/json-schema conversion, should help in the docs part. Also, the JSON/String conforming of the specs. Trying to make those transparent.
2016:10:27 12:22:27 mpenet @ikitommi anything public ?
2016:10:27 12:25:28 mpenet personally I am holding my horses until this stabilizes (I am very much interested in having something solid for swagger, json-schema & all)
2016:10:27 13:24:51 ikitommi @mpenet nothing really usable as public. There is stuff under metosin/spec-tools (second take on the ->json-schema & original spike on the dynamic conformation). But just planned to put effort on the next two weeks with these. I'm working on the spec->json-spec & spec->string-spec transformation - e.g. generating differently conforming copies of the specs for different formats/use cases. If that works, will throw the dynamic conformations into trash bin. Coercion rules in both from Schema/Ring-swagger. Were you doing the json schema -> spec side? or the same?
2016:10:27 13:25:30 ikitommi and thanks for pointing out Alex’s message. Would really like to hear how thing will evolve.
2016:10:27 13:29:17 mpenet Something similar (or not), not sure. We have a thin facade that generate both specs with conformers and json-schemas, this way this shields us from changes in the underlying libs for now. Moderately happy about the current situation, hopefully the changes alex mentioned will make this stuff easier.
2016:10:27 13:29:56 mpenet going from spec to json-schema directly is something we tried but this was brittle, tons of possible breakage unless you set some strict rules of what you can / cannot use in spec
2016:10:27 13:31:49 ikitommi sounds cool. So, you are describing something in your own way and generate specs out of those? of do you start from plain specs too?
2016:10:27 13:32:14 mpenet yes, at the edge we have our own "spec" language
2016:10:27 13:32:22 mpenet I think also onyx does something similar
2016:10:27 13:32:40 ikitommi do you have something of that in public?
2016:10:27 13:32:46 mpenet but hopefully this is temporary
2016:10:27 13:33:11 mpenet no not yet, we might release this but no promises.
2016:10:27 13:34:54 ikitommi the old (plumatic) schema -> json schema was partly brittle too, and as the json schema (and expecially the swagger version of it) is less powerfull, only parts of the Schemas could be presented in it.
2016:10:27 13:35:19 ikitommi what did you find brittle in the direct spec->json-schema conversion?
2016:10:27 13:35:21 mpenet indeed. same problem
2016:10:27 13:36:55 mpenet the fact that a spec is a predicate makes it too open and hard to "understand", unless you are 100% aware of the limitations you can cause the generation of json-schema to go "wtf is that" to some validators/converters
2016:10:27 13:38:17 mpenet dunno what's best between limiting ourselves to a subset our the facade solution. the facade has the advantage of shielding us from potential dependency changes (Schema -> Spec), and allows the devs not to be afraid to break stuff in a dependency they never inspected
2016:10:27 13:39:47 mpenet seems like that could be a good solution for compojure-api & related libs imho
2016:10:27 13:43:00 ikitommi yes, at least the current spec maps are quite hard to use, was thinking of adding schema-like map -support ~ {::name string?, ::id integer?}
2016:10:27 13:44:12 ikitommi in web apis, currently we have lot’s of anonymous maps to describe the different paramerer sets for endpoints.
2016:10:27 13:48:12 mpenet I think the validators should be more static (for compojure-api & co), using string? integer? is not so good in that context, just keywords (:string :integer etc) that dispatch to a multimethod that knows what to make out of it and act as some sort of registry that's limited to that dsl. Allowing the user to create schemas and register/use them too in that context since they're known to be understandable by the coercion/validation chain later.
2016:10:27 13:49:52 mpenet anyway, I am curious to see what will come next
2016:10:27 15:27:40 zane Are conformers intended to be limited in scope? As in, should I use them exclusively for minor transformations, or would it still be idiomatic to use them for larger-scale manipulation?
2016:10:27 15:27:46 zane Hope that question make sense. Sorry for being vague.
2016:10:27 15:42:26 alexmiller they should be used cautiously, particularly when using them with registered specs as you are making decisions for all consumers of your specs (for now and the future, which is long :)
2016:10:27 15:42:54 alexmiller in general, I think generic data transformation is better done explicitly with functions in normal Clojure ways
2016:10:27 16:24:36 seancorfield The problem we ran into is that we had a lot of specs for our API that needed to accept a "string that could be converted to something that conforms to another spec" — and so your choice is either to make the spec produce the converted value (using conformer) or run that conversion twice: once to check you have a compliant string, and then again as Clojure code once you have "validated" your string input.
2016:10:27 16:26:18 seancorfield Your other choice is to run the conversion as a "pre-spec" layer and deal with conversion failures outside spec… which is a lot of boilerplate.
2016:10:27 16:28:24 seancorfield Given Alex’s dark warnings about using conformer "extensively" (which we were), we have sort of called a halt to our spec usage in order to see what falls out as the recommended approach for API specs (i.e., specs that must accept strings that essentially conform to other specs after "conversion").
2016:10:27 16:29:43 seancorfield If that recommended approach is "run the conversion twice", well, fair enough, but it seems a bit of a waste of effort.
2016:10:27 16:56:23 alexmiller I think perhaps you take my words more apocalyptically than they were intended :)
2016:10:27 16:57:46 zane Thanks for the exposition, @seancorfield. That's precisely the situation I'm in.
2016:10:27 16:59:46 seancorfield @alexmiller Well, you have repeatedly cautioned against (overuse of) conformer whenever this sort of pattern comes up — and I’m not seeing a recommendation for how to handle it otherwise.
2016:10:27 17:00:41 seancorfield For us, having the API spec also handle the conversion (via conformer) is the easiest way to do things. But easy != simple and I understand that it complects validation with transformation...
2016:10:27 17:04:31 alexmiller My point is and has always been: you are making a choice for all future consumers of your registered specs and you should think about the ramifications of this before you use conformers on everything
2016:10:27 17:05:15 alexmiller that’s it
2016:10:28 17:46:07 drewr I'm having a time with clojure.test.spec/check
2016:10:28 17:48:16 hiredman you are using a test.check generator directly somewhere, maybe for ::es-opts
2016:10:28 17:48:34 hiredman you need to wrap them in a thunk
2016:10:28 17:48:36 drewr yeah, I suspected that, but I don't :require test.check anywhere
2016:10:28 17:49:04 drewr ah wait, I see it now
2016:10:28 17:49:12 drewr yeah ::es-opts was it
2016:10:28 17:50:04 drewr I was getting that exception for a with-gen in a separate place and kept thinking it was the same place
2016:10:28 17:50:12 drewr @hiredman thanks kev!
2016:10:28 17:50:17 hiredman 🙂
2016:10:28 18:04:33 cap10morgan What are folks' thoughts on generative testing w/ spec on stateful fns (in this case modifying an atom)? How do you setup enough variations on the state for meaningful tests? Or is the answer that I should go to test.check directly instead of relying on what clojure.spec generates?
2016:10:28 18:07:49 bhagany @cap10morgan: I haven’t done anything stateful yet, but if I had special generative needs, I’d reach for passing a custom generator in with-gen first. If I understand you correctly, this is what you’re reaching for with the idea to use test.check directly.
2016:10:28 18:08:27 cap10morgan @bhagany OK, thanks. You answered my next question too. 🙂
2016:10:28 18:11:46 seancorfield @cap10morgan I’d try to separate the logic from the state and test the logic separately if possible.
2016:10:28 18:12:26 cap10morgan @seancorfield Hmm, that's a good idea. I need to get into the habit more often. Thanks!
2016:10:28 18:13:27 seancorfield I was just having a similar conversation in #component 🙂
2016:10:28 18:15:44 drewr how do you figure out which spec or gen is problematic when you get Couldn't satisfy such-that predicate after 100 tries.
2016:10:28 18:16:08 drewr the stacktrace doesn't provide any clues that I can see
2016:10:28 18:19:18 hiredman I think alexmiller has mentioned that as a pain point, that error comes out of test.check and plumbing the spec information through from spec to test.check and back again hasn't been done
2016:10:28 18:19:34 hiredman I would start looking at your s/ands I guess
2016:10:28 18:21:12 hiredman if I recall, the way generators for s/and specs are constructed is you take the generator for the first spec in the s/and, and then such-that the rest of the specs (as predicates)
2016:10:28 18:21:58 hiredman so if you have a really generic first thing, and more specific predicates afterwards, you can easily end up with a such-that that doesn't work
2016:10:28 18:24:11 drewr yep, that's exactly what it was
2016:10:28 18:24:37 drewr
(s/and string?
#(.startsWith % "http"))
2016:10:28 18:25:05 drewr I had a string, but it was "foo"
2016:10:28 18:41:51 alexmiller there is an exactly almost the same as this in the spec guide http://clojure.org/guides/spec
2016:10:28 18:42:10 alexmiller that shows how to make a custom generator for it
2016:10:28 18:45:44 drewr @alexmiller it wasn't that I misunderstood the spec, I just couldn't find the problematic one
2016:10:28 18:46:01 drewr thanks for that doc!
2016:10:29 02:03:49 steveb8n I have zipper data structure using the techniques described in https://puredanger.github.io/tech.puredanger.com/2010/10/22/zippers-with-records-in-clojure but I’m finding it hard to write a spec that validates child nodes using the protocol. I can’t use a map key because different impls of TreeNode (Zippable in my case) may use different keys. I’ll paste an e.g. next....
2016:10:29 02:04:49 steveb8n Any suggestions would be much appreciated
2016:10:29 04:04:29 grant Is there an easy way to figure out which spec in a large nested spec that is causing "ExceptionInfo Couldn't satisfy such-that predicate after 100 tries." when using (s/exercise ...)?
2016:10:29 04:25:43 jrheard not afaik but i’m not an expert
2016:10:29 04:27:41 jrheard usually it’s down to a string or something that has lots of constraints put on it and is difficult to generate reliably, but you probably already know about that; one gotcha that i don’t think is as well-known is that if you have a coll-of that’s got a :kind set? but doesn’t have an into #{}, that can cause the same couldn’t-satisfy-after-100-tries behavior
2016:10:29 04:28:02 jrheard might not be your issue, but i always feel compelled to mention it when people have this problem because it happened to me and took me a while to figure out
2016:10:29 04:30:46 grant Ah, good tip on the set thing, I haven't run into that yet, but I could see making that mistake.
2016:10:29 04:37:14 seancorfield It doesn't really help you but I highly recommend trying to s/exercise each spec as you write it and build up complex specs in terms of other specs (not just predicates).
2016:10:29 04:38:37 seancorfield Predicates tend to be opaque when you combine them into specs. If you define specs for those predicates, when you combine the specs, you get better error reporting.
2016:10:30 21:34:30 drewr I'm having trouble tracking down a :clojure.spec/unknown
2016:10:30 21:35:50 drewr the :clojure.spec.test/caller, :sym, and :via all point to different things, it doesn't seem to make any sense
2016:10:30 21:59:21 cddr Folks interested in clojure-spec might be interested to know about some prior art in the pharma industry. They have an organization called "CDISC" that defines a collection of XML standards for defining clinical databases for pharma trials. Their "operational data model" has an info model quite similar to spec. It used to be private only to members but looks like they've opened it up recently: https://www.cdisc.org/standards/foundational/odm
2016:10:30 22:25:59 mac @cddr How is this different from any other XML Schema ?
2016:10:30 22:27:30 cddr The bit that spec borrows is the way it defines "ItemDef"s independently of their containers
2016:10:30 22:30:18 mac @cddr Ok...
2016:10:31 12:03:58 tcoupland I feel like there's a missing flag on s/keys, something like 'complete' or {:allow-unspecified false} as an option. A couple of times while using s/check i've spelled something wrong, in the spec, or set a key badly in the transform code, and spec isn't yelling at me about it, despite everything being spec'd up, it comes down to this for me:
and have spec shout if it see's any key it's not ready for, then I'd be getting more out of my spec's! Just something that's bothered me this morning as i realised I'd mistakenly pluralised a key word 🙂
2016:10:31 12:04:26 tcoupland hopefully someones going to tell me it already exists somewhere now!
2016:10:31 14:00:25 donaldball The usual workaround is to combine the s/keys spec with an s/map-of spec
2016:10:31 14:02:06 donaldball (I agree that a :complete flag would be a welcome addition to the s/keys macro, but Rich seems committed to open maps as the preferred general data structure)
2016:10:31 14:06:02 alexmiller specs always make positive statements about data, not negative ones and we do not plan to add anything to s/keys along these lines
2016:10:31 14:06:27 alexmiller however there is something we’re considering that would allow making these additional assertions easier
2016:10:31 14:27:54 tcoupland @donaldball that does work, but is pretty ugly isn't it. Sound like a 'wait and see' situation
2016:10:31 14:46:06 donaldball It’s not a hard macro to write, a keys variant that allows a complete? flag
2016:10:31 14:48:10 tcoupland yeah i just started bashing it out 🙂
2016:10:31 14:49:30 drewr can anyone point me in the direction of tracking down why that's unknown?
2016:10:31 14:53:28 drewr I get a caller, but I go to that line, and it's downstream from the function that :sym references 🤔
2016:10:31 14:53:48 drewr so I must be misunderstanding something fundamental
2016:10:31 15:23:19 alexmiller the fact that it’s printing unknown there is a known issue with describing preds
2016:10:31 15:23:29 alexmiller so that part is a known bug
2016:10:31 15:25:31 alexmiller but you have the path so it should be pretty narrowly scoped to where the pred is failing?
2016:10:31 15:25:33 drewr ok, that helps, so go through and perhaps exercise my specs to see if anything's funny?
2016:10:31 15:25:59 drewr I'm probably doing this the worst way, by trying to migrate from schema instead of using on a fresh codebase
2016:10:31 15:26:48 alexmiller as with most programming things, it’s best to start small and build up
2016:10:31 17:22:02 drewr as a general comment, the jump to mandated generative testing is a bit of a leap
2016:10:31 17:22:22 drewr maybe I'm doing something wrong, but I don't always need this
2016:10:31 17:23:26 drewr with a lot of side-effects, traditional unit-testing can work just fine, and I just want to make sure functions deep in the call stack are getting the right arguments and returning something valid
2016:10:31 17:24:00 drewr my impression after a couple weeks is that spec makes this some of this more difficult over, say, schema
2016:10:31 17:24:19 drewr and maybe that's the intent: lower-level, more correct, more powerful, etc., it just also means there aren't yet companion libraries to help with the porcelain
2016:10:31 17:25:06 drewr maybe I'm just spoiled by clojure's quality... alpha is usually good enough 😄
2016:10:31 17:30:40 donaldball You don’t need to buy into generative testing to use clojure.spec. instrument with traditional example-based tests is pretty useful.
2016:10:31 17:31:57 drewr @donaldball that was my early reaction as well, but I lost the ability to check return values, which I get, that's for check, but then that forces me into the generative style
2016:10:31 17:32:25 drewr happen to have a piece of code you transitioned that you're happy with?
Why does that inner ::select clause not tagged the way the outer one is?
2016:10:31 21:29:22 seancorfield You need conform-keys
2016:10:31 21:31:18 seancorfield Hmm, :conform-keys is only listed as an option for map-of but I would expect it to work here too...
2016:10:31 21:33:41 zane Why would I need conform keys? "2.1" is being conformed by the s/or.
2016:10:31 21:34:17 zane It's like conforming stops at the recursive step.
2016:10:31 21:34:20 seancorfield Nope, only works with map-of — but if you change :subselect to be (s/map-of string? ::select :count 1 :conform-keys true) then you get this
2016:10:31 21:35:22 zane I still don't understand why :conform-keys has anything to do with this.
2016:10:31 21:35:27 zane It's the values that need conforming.
2016:10:31 21:35:47 seancorfield Yeah, I think I misunderstood what you were saying didn’t work..
2016:10:31 21:36:08 seancorfield Is the output I got with map-of what you want?
2016:10:31 21:36:25 zane I would just expect that in my original example "2.1" would be [:attribute "2.1"].
2016:10:31 21:37:01 zane Why is the recursive s/or not being conformed?
2016:10:31 21:37:04 seancorfield (You’re right, you don’t need :conform-keys but you do need map-of, not every-kv)
2016:10:31 21:37:07 zane It seems like it happens, but only sometimes.
2016:10:31 21:37:41 seancorfield Per the docs, every-kv does not conform all values — map-of does.
2016:10:31 21:37:45 seancorfield "Unlike 'every-kv', map-of will exhaustively conform every value."
2016:10:31 21:38:07 zane Ahhh.
2016:10:31 21:38:16 zane I missed that in the docs.
2016:10:31 21:38:30 seancorfield "Note that 'every' does not do exhaustive checking, rather it samples
coll-check-limit elements. Nor (as a result) does it do any
conforming of elements.” — and every-kv is like every
2016:10:31 21:39:58 zane Got it.
2016:10:31 21:44:46 zane Phew. Thanks, Sean.
2016:10:31 21:58:17 seancorfield (I must admit, I find it counter-intuitive that every / every-kv do not in fact check every element but coll-of / map-of do)
2016:10:31 21:58:43 seancorfield After all, they have every in their name so you’d sort of expect them to check every item...
2016:10:31 21:59:05 jrheard heh yeah
2016:10:31 22:19:49 bhagany it is the least-intuitive thing. I wish I understood the rationale.
2016:10:31 22:27:41 zane It's heartening that I'm not the only one. 😅
2016:10:31 22:59:21 alexmiller "every" is an assertion of belief
2016:10:31 22:59:50 alexmiller Not a statement of what they do
2016:11:01 03:33:02 settinghead I'd like to conform this vector [["John" 20] ["Jane" 30] ["Bob" 40]] into [{:name "John" :age 20} {:name "Jane" :age 30} {: name "Bob" :age 40}]. how should I write my spec?
2016:11:01 15:40:27 zane My experience has been that needing to unload a spec is a rare occurrence.
2016:11:01 16:46:00 hiredman is there something like s/keys, but instead of looking up specs via global names, I can pass it a spec for the value of each key?
2016:11:01 16:51:59 hiredman I have these tests that result in a data structure that is sort of a log of activity, and the result of the test is determined by checking that log
2016:11:01 16:53:31 hiredman the log is a sequence of maps, and it has a :messages key, and at each entry in the log, I know that key should have a specific value
2016:11:01 16:54:08 hiredman if I have to define the value globally (use s/def and s/keys) then all I can say is that it could be any of a number of different values
2016:11:01 16:55:09 hiredman ah, maybe s/map-of and s/merge
2016:11:01 17:03:31 hiredman nope
2016:11:01 17:06:05 bfabry @hiredman what do you mean "at each entry in the log", like you dynamically know what it should be or you have a list of things it should be in certain circumstances? because if the latter you could always use :req-un
2016:11:01 17:07:03 hiredman :req-un still looks up specs via a global name in the spec registry
2016:11:01 17:07:45 hiredman I have more refined information locally than globally
2016:11:01 17:08:24 hiredman globally all I can say is the value for the key should be either x y or z, but locally I know exactly what the value for the key should be
2016:11:01 17:11:20 settinghead @curlyfry thanks
2016:11:01 17:11:29 bfabry right, but I meant you can define multiple versions of :messages using :req-un
2016:11:01 17:11:40 hiredman yeah
2016:11:01 17:12:17 bfabry (s/def ::messages-1 (s/keys :req-un [:messages-1/message])) and then switch in the spec you need later on
2016:11:01 17:14:30 hiredman I was hoping to use spec as a sort of a data regex to validate these test results, but having to globally define map validators is a pain
2016:11:01 17:15:07 hiredman like if you had to globally define your capture groups before you could use them in a regex engine
2016:11:01 17:18:40 bfabry I see your point
2016:11:01 17:21:35 hiredman and if I had followed "best practices" and namespaced the :messages key, that work around wouldn't work at all
2016:11:01 17:28:54 hiredman I guess I can build something using s/alt
2016:11:01 18:26:24 madstap
(s/fdef update*
:args (s/cat :m map?
:k keyword?
;; I want to say:
;; The function needs to take at least one arg,
;; the current value in the map,
;; and can take zero or more extra arguments.
:f (s/fspec :args (s/cat :value-in-map any?
:args (s/* any?)))
:args (s/* any?)))
(defn update* [m k f & args]
(apply update m k f args))
(test/instrument)
(update* {:a 10} :a inc)
2016:11:01 18:26:59 madstap How do I do this in spec?
2016:11:01 18:28:35 madstap
;;=> Call to #'foo.core/update* did not conform to spec: In: [2]
val: (nil) fails at: [:args :f] predicate: (apply fn)
:clojure.spec/args ({:a 10} :a #function[clojure.core/inc])
:clojure.spec/failure :instrument :clojure.spec.test/caller
{:file "form-init8332152340913865653.clj", :line 342, :var-scope
foo.core/eval12509}
2016:11:01 18:31:26 zane @madstap You're probably better off just using a function.
2016:11:01 18:46:10 madstap From that example I'm having truble seeing how I could apply multi-spec to my problem
2016:11:01 18:47:12 madstap @hiredman could you explain?
2016:11:01 18:50:14 hiredman maybe not, multi-spec would only be useful if you knew the set of keys you would be pass to update*
2016:11:01 18:52:03 hiredman I guess that is not entirely true, if you knew the set of keys entirely before hand, you could use an s/alt or s/or, the advantage using multi-spec would be that choice would be open ended
2016:11:01 18:52:15 hiredman so easier to extend
2016:11:01 18:54:40 hiredman
(defmulti update-multi-spec (fn [m k fun & args] k))
(s/fdef update*
:args (s/multi-spec update-multi-spec)
:ret any?)
(defmethod update-multi-spec ::a [m k fun & args]
(s/cat
:map (s/keys :req [::a])
:key #{k}
:fun (s/fspec :args ...)
:args ...
))
2016:11:01 19:09:39 madstap I think I explained myself badly, what I want to do is say that the function passed to update needs to take at least one argument, and may take more.
2016:11:01 19:17:50 madstap Your multi-spec example helped me grok why multi-spec is useful, though, thanks
(s/fdef update*
:args (s/cat :m map?
:k keyword?
:f (s/fspec :args (s/+ any?))
:args (s/* any?)))
(defn update* [m k f & args]
(apply update m k f args))
(test/instrument)
(update* {:a 10} :a inc)
2016:11:01 19:42:26 madstap
Call to #'foo.core/update* did not conform to spec: In: [2]
val: (nil) fails at: [:args :f] predicate: (apply fn)
:clojure.spec/args ({:a 10} :a #function[clojure.core/inc])
:clojure.spec/failure :instrument
:clojure.spec.test/caller {:file
"form-init8954135211496312533.clj", :line 478, :var-scope
foo.core/eval13250}
2016:11:01 19:43:01 hiredman yeah, because any? is generating bad inputs for the function
2016:11:01 19:43:36 hiredman the way test check validates the function is it generates inputs for it
2016:11:01 19:44:20 hiredman your function is inc, in the spec you say it can take an any?, so spec is checking that by feeding it all kinds of values
2016:11:01 19:51:13 madstap Right, what I wan't to say that this function needs to take at least one arg, but doesn't necessarily take any more args.
2016:11:01 19:52:22 madstap In this case that passing (fn [] (rand-int 10)) to update will fail, but inc will work, as will (constantly true).
2016:11:01 19:57:17 hiredman the problem is when you turn instrumentation on, spec is validating inc against your fspec
2016:11:01 20:00:17 hiredman a spec that is general enough isn't going to be valid against a specific function, you will generate very general inputs that the specific function won't handle
2016:11:01 20:01:13 hiredman I think the best you could do would be a multi-spec, with a defmethod for every function/key combination you pass to update*
2016:11:01 20:26:16 madstap Got it, thanks
2016:11:02 09:55:16 vandr0iy Hi, #clojure-spec! came to ask: what's the difference between clojure.spec/map-of and clojure.spec/every-kv?
2016:11:02 12:46:27 bhagany @vandr0iy map-of checks all the entries of the map, while every-kv samples a subset of the entries
2016:11:02 12:47:19 bhagany every-kv is designed to be used with maps that may be large
2016:11:02 13:06:01 lopalghost Any thoughts on writing separate specs for input data that needs to be coerced, vs using spec/conformer?
2016:11:02 13:06:56 mpenet coerce separately from your main specs yes, with or without spec
2016:11:02 13:07:11 mpenet it gets very messy otherwise imho
2016:11:02 13:07:59 lopalghost That's what I'm finding to be the case
2016:11:02 13:09:17 lopalghost The only problem is I'm duplicating my validation logic, because I want to be able to detect invalid data at the point of entry
2016:11:02 13:09:44 lopalghost I'm using spec in both cases because the generative testing is super-helpful
2016:11:02 13:12:10 mpenet you can create predicates separately that you re-use on both specs
2016:11:02 13:12:27 mpenet but yes, there will be duplication here and there
2016:11:02 13:36:04 drewr what are you all doing to cleanup after check on a fn with side effects?
2016:11:02 13:36:34 drewr it doesn't look immediately possible with :fn unless I pass context back through the return value
2016:11:02 13:37:03 drewr I suppose I could with-redefs in there, but that doesn't feel right
2016:11:02 14:01:22 alexmiller @vandr0iy additionally, every-kv will not conform its keys or values (because it doesn’t check all of them) whereas map-of will conform all values (and optionally all keys)
2016:11:02 14:02:05 alexmiller @drewr don’t write fns with side effects? :)
2016:11:02 14:02:29 alexmiller seriously though, you might look at the stub and replace options on stest/instrument
2016:11:02 14:03:22 drewr yeah, would love to avoid them, but have to do things other than burn cpu 🔥
2016:11:02 14:03:36 drewr cool, didn't notice those opts on instrument
2016:11:02 14:03:37 alexmiller particularly for the case where you are checking a function that calls a side-effecting function
2016:11:02 14:03:55 alexmiller you can stub or replace it instead
2016:11:02 14:06:14 drewr I didn't think to look at instrument, though, because this goes farther than just args checking
2016:11:02 14:07:00 drewr seems like something that would happen with test.check... ie, a function that somewhere ends up writing a value to a db, and I need to make sure that value is deleted (or the db is removed)
2016:11:02 14:07:12 alexmiller you should instrument, then check
2016:11:02 14:07:18 alexmiller then unstrument
2016:11:02 14:08:51 drewr right now I'm doing the whole enumerate-namespace -> check -> summarize-results dance, sounds like I need to break out of that pattern
2016:11:02 15:01:53 jrheard what’s the alternative to the pattern @drewr gives above?
2016:11:02 15:03:05 jrheard i haven’t used the stub-and-replace options alex mentions, so maybe you guys just mean that using them necessitates breaking out of the pattern he gives
2016:11:02 15:03:17 jrheard just curious 🙂
2016:11:02 15:04:25 drewr @jrheard I'll give you an example when I have one 😂
2016:11:02 15:15:06 lopalghost @gfredericks I think spec/conformer is a pretty good solution to the problem of coercing back and forth, at least
2016:11:02 15:16:19 lopalghost The problem is you lose information about the input, making explain and check less useful
2016:11:02 15:17:02 lopalghost I've tried writing some DSLs to generate specs, but I was never really happy with them
2016:11:02 15:19:24 gfredericks Yeah, I remember an earlier conversation that ended up recommending mapping to new specs rather than just conforming
2016:11:02 15:20:04 gfredericks But spec isn't good at, e.g., describing maps with string keys, though :/
2016:11:02 15:25:47 mpenet anything other than keyword keys (sadly)
2016:11:02 15:27:07 mpenet you end up having to resort to weird, half working, set + map-of hacks
2016:11:02 15:27:46 zane It feels like passing around conformed values is discouraged. Is that correct?
2016:11:02 15:29:20 mpenet I often wonder why namespaced keys couldn't just be a collection of keywords, since that's just values
2016:11:02 15:30:37 mpenet I guess there's a good reason behind this (interning magic leading to better perf ?)
2016:11:02 15:31:12 lopalghost I don't like set + map-of because it makes explanations less clear
2016:11:02 15:31:18 mpenet [:some-ns :some-key]
2016:11:02 15:31:36 lopalghost It might be better to coerce the keys in a separate step
2016:11:02 15:31:46 mpenet @lopalghost more importantly it breaks link between key and value
2016:11:02 15:31:58 lopalghost Right
2016:11:02 15:32:01 mpenet it just guarantees the key is here
2016:11:02 15:37:28 mpenet so yeah either coercion, or it might be better to just have a custom predicate to validate such maps
2016:11:02 16:30:23 alexmiller you can use every on maps to describe the map entries as k-v tuples
2016:11:02 16:31:49 mpenet neat, I didn't think of that one
2016:11:02 16:32:29 alexmiller but whatever or’ed entry types you need
2016:11:02 16:33:06 alexmiller for non-kw key maps, this can be a good approach
2016:11:02 16:33:52 alexmiller (that’s how every-kv and ultimately map-of are actually implemented)
2016:11:02 19:45:45 nwjsmith Where can I vote for adding docstrings to specs?
2016:11:02 19:48:55 nwjsmith I think I saw it being discussed here a few weeks ago, and I though “who would ever want to do that?” Turns out I am someone who would want to do that.
2016:11:02 20:14:38 potetm Is it appropriate to give more rationale for that ^ ticket in a comment on JIRA?
2016:11:02 20:15:30 potetm I've thought about it a little bit, and I think there's a really good justification for adding it.
2016:11:02 22:21:09 jasonjckn is there a general recommendation for versioned schema?
2016:11:02 22:21:14 jasonjckn multi spec dispatching on version tag?
2016:11:02 22:21:49 jasonjckn what if i'm evolving my spec over time, deprecating fields, introducing new ones, changing existing ones.
2016:11:03 05:29:28 alexmiller @potetm feel free, although I don’t know that there is much need :)
2016:11:03 05:30:33 alexmiller @jasonjckn backwards compatible changes do not necessarily require a new spec (for example, s/keys will always validate all registered keys, so adding new ones does not require a spec change)
2016:11:03 05:31:03 alexmiller @jasonjckn breaking changes should mean a new spec
2016:11:03 05:32:06 alexmiller I expect this area will receive more attention as its something Rich has thought about a lot
2016:11:03 05:34:03 alexmiller and he’s really serious about not just “changing” specs but actually leaving the existing one and creating a new one (::person2, ::person3, etc)
2016:11:03 05:34:16 jrheard the evils of “using the same name to refer to different things"
2016:11:03 05:34:34 alexmiller hello immutable data
2016:11:03 05:34:38 jrheard 😄
2016:11:03 05:56:53 ikitommi has anyone managed to do the “take a spec, walk it and transform it to another (differently conforming) spec”? I tried, but for s/keys it seems hard/impossible. Have already copied much of the clojure.spec internals into my project for this. With un-keys, it’s easy, with the non-un keys, it’s not: the key names should remain same, with a different implementation.
2016:11:03 05:59:52 ikitommi would be nice, if there was a map-kind of Spec in clojure.spec that would allow separate keys and values - I could just swap the values in this case. Could there be such a thing under the hoods of s/keys @alexmiller?
2016:11:03 18:30:30 gfredericks @ikitommi having a consistent meaning for a namespaced keyword is one of the core ideas of clojure.spec
2016:11:03 18:56:20 zane ikitommi's question relates to my earlier one about whether conformed values are more or less intended to remain internal to the function in which they were conformed.
2016:11:03 18:56:26 zane When used idiomatically, anyway.
2016:11:03 19:05:18 ikitommi @gfredericks true that, but for external input, there is a need to conform/coerce values differently, based on capabilities of the wire format. Stuarts thought was to transform raw specs into new specs, with different conformers.
2016:11:03 19:06:17 zane How would that preclude you from using different keys for the new specs, ikitommi?
2016:11:03 19:07:01 ikitommi what if I want to expose a fully-qualified key?
2016:11:03 19:09:32 ikitommi for unqualified keys, I can just create a new spec key, e.g. :mydomain/age => :JSON.mydomain/age, as both can be mapped to the use the :age key.
2016:11:03 19:41:03 kuzmin_m Hi!
I play with spec and found one dangerous point.
I use s/fspec to specify high-order function foo with bar argument.
In several cases I may call foo with bar that have some side effects.
When instrument is enabled then bar will be called more than onсе
because spec will check bar.
For instance I may write smth like that: (foo delete-user-by-id)
and debug it many hours.
I can check only bar arity but stest/check will not work.
2016:11:03 19:53:31 drewr @kuzmin_m I've been dealing with exactly this
2016:11:03 19:54:16 drewr you might not want to use check at all, but instrument with its :replace option, and then call the function normally
2016:11:03 19:58:57 kuzmin_m Yes. But people are careless. If something is breakable it’ll be broken.
2016:11:03 20:00:02 kuzmin_m I may forget specify replace option
2016:11:03 20:00:06 drewr not sure what you mean
2016:11:03 20:02:59 kuzmin_m I need an extraordinary action. I need remember about s/fspec behaviour and specify instrument option.
2016:11:03 20:13:23 kuzmin_m May we write a function that replace s/fspec by fn? when we use instrument?
We don’t call fn and we only check an argument type in this case.
2016:11:03 20:14:52 bfabry @kuzmin_m that would make instrument significantly less useful
2016:11:03 20:16:11 bfabry fwiw, I think there should probably be more explicit handling of functions with side-effects for both fspec and fdef, and I'm hopeful that something is coming
2016:11:03 20:18:11 kuzmin_m Sure! But it only about fspec, not about fdef.
2016:11:03 20:20:09 bfabry they're kinda the same thing though, fdef as far as I understand it is just (s/def name (s/fspec blah))
2016:11:03 20:23:17 kuzmin_m Yes. I just write about one case: s/fspec for function argument.
2016:11:03 20:24:28 bfabry right, I just mentioned fdef because you can use the name of something fdef'd somewhere in place of an fspec and you'll get the same behaviour. but anyway you're right that it would be on fspec where things need to be handled
2016:11:03 20:25:22 kuzmin_m right :+1:
I forget about that behavior 😞
2016:11:03 20:27:01 kuzmin_m It’s just alpha version.
I hope that we can fix this issue.
2016:11:03 21:41:16 mrcnc is there a “best practices” way to run spec tests from lein?
2016:11:03 23:11:08 jrheard i haven’t seen official guidance on the one true specifically correct invocation of clojure.spec.test/check, but when i dig around in github search i usually see people write clojure.test assertions like (is (= 0 ((summarize-results foo) :failures))) or whatever
2016:11:03 23:11:16 jrheard handwaving here, but you get the idea
2016:11:03 23:53:41 kenny It works well with Cursive’s run tests REPL action
2016:11:04 11:54:37 mpenet just wondering: why in multispec we have to repeat the dispatch fn/key in s/multi-spec call itself (since it's already in the defmulti definition)
2016:11:04 12:44:34 alexmiller You can use multispec on cases other than keyword maps too - in that case you need a fn for retagging instead.
2016:11:04 13:50:59 mpenet yup, makes sense
2016:11:04 13:51:10 mpenet it's not obvious from the guide tho
2016:11:04 13:51:21 mpenet but the docstring is clear about it
2016:11:04 17:59:00 sveri Lets say I have two specs A and B and now I have a function that has a paramater that is the union of the two maps so like (keys ::A ::B). How would I define that spec?
2016:11:04 17:59:17 sveri I tried a (merge ::A ::B) but that does not work
2016:11:04 18:01:43 bfabry @sveri did you use core/merge or spec/merge?
2016:11:04 18:02:23 sveri @bfabry Ah, I did not know there was a spec/merge, thank you, I used the core one. With spec/merge it works 🙂
2016:11:04 23:12:30 d._.b I'm feeling dumb. I have a map like
{:abc "abc" ; required key, and must have a value of "abc"
:x {:y nil ; required key, but value can be anything
:z 123 ; same as above}
}
2016:11:04 23:44:54 d._.b I think I must just be missing something obvious
2016:11:04 23:45:48 d._.b If I have like 20 keys which are req-un in a map, all of which should have values checked of any?, that seems like it should be more straightforward than resorting to doseq'ing to generate s/defs
2016:11:04 23:48:58 d._.b Sorry for the repost but, just to reiterate:
{
:abc "abc" ; required key, and must have a value of "abc"
:x {:y nil ; required key, but value can be anything
:z 123 ; same as above
:w :wow ; same as above
:o :out ; same as above
}
}
2016:11:04 23:51:59 d._.b Do I really need to write all 4 of those defs?
2016:11:04 23:53:52 d._.b I was thinking I could (s/every-kv #{::y ::z ::w ::o} any?) or something... No?
2016:11:05 00:50:44 bbloom finally getting around to trying spec in anger
2016:11:05 00:50:55 bbloom is there a standard conform-or-throw function?
2016:11:05 01:27:40 hiredman there is a/assert, but you need to set a property to turn it on
2016:11:05 01:35:41 richiardiandrea @bbloom I so wanted that too, unconditionally, I could not find it and copied over s/assert and named it valid-or-throw😄
2016:11:05 01:39:13 richiardiandrea great I'll tweet you 😉
2016:11:05 12:44:51 dominicm Is there yet a way to "scope" an error done against a whole map?
Given that I have:
{:x 1
:y [2 3 4 5]}
And want to validate that :x must be contained in :y, but I want my error message to state that it is :x that is wrong, not the whole map.
How would I do it?
2016:11:05 13:02:46 alexmiller s/and a function that checks that x is in y?
2016:11:05 13:03:33 alexmiller The error should list that function
2016:11:05 13:03:53 alexmiller As the failing predicate
2016:11:05 14:17:12 dominicm @alexmiller I'm thinking more of the :path of the error (I think that's the key)
2016:11:05 15:56:32 tianshu how can i turn on the :ret check for clojure.spec/fdef.
2016:11:05 19:47:10 mpenet I had deal with that pb recently, If you check on test.check wiki there's a page explaining how to deal with recursive gen and how to limit gen depth/size
2016:11:05 19:51:00 mpenet (I am on my phone, not easy to find)
2016:11:05 22:44:16 bbloom is there a spec somewhere for the ns form or “libspec” data?
2016:11:05 23:43:00 gfredericks @bbloom I think you can poke around to find how the spec for ns is implemented
2016:11:05 23:43:11 gfredericks (doc ns) might even help with that
2016:11:05 23:43:46 bbloom duuuuh i forget that core had all the specs in there now
2016:11:05 23:44:00 gfredericks and clojure.spec/form
2016:11:05 23:44:08 bbloom anyway - i decided to do my own spec for a tiny subset of the complexity of the full ns form
2016:11:05 23:44:18 bbloom pretty narrow use case
2016:11:06 00:50:24 hiredman so, anyone write a spec -> gloss codec compiler yet?
2016:11:06 01:42:06 keithsparkjoy Works fine if I define it all on one line, but include the newline and the repl chokes.
2016:11:06 01:42:15 keithsparkjoy Curious if anyone has experienced this or if it’s just me?
2016:11:06 01:42:50 keithsparkjoy Trying to use the new map literal syntax with clojure.spec, which is why I’m asking here...
2016:11:06 01:43:47 keithsparkjoy The error I get is java.lang.RuntimeException: Unmatched delimiter: )
2016:11:06 14:55:15 madstap @keithsparkjoy Works for me on alpha14
2016:11:06 16:11:32 keithsparkjoy Okay must be something about my environment. Thanks for trying.
2016:11:07 07:36:15 norton Has someone an example where spec instrumentation using the :replace option works? I tried a simple example to override the ranged-rand function with one that always returns the start value … but so far no luck.
2016:11:07 18:11:15 uwo is there anyway to say that a map should contain one or the other key, besides this?
2016:11:07 18:13:21 minimal @uwo you can use or in s/keys, see the docstring
2016:11:07 18:13:59 minimal
The :req key vector supports 'and' and 'or' for key groups:
(s/keys :req [::x ::y (or ::secret (and ::user ::pwd))] :opt [::z])
2016:11:07 18:14:52 uwo @minimal wicked, thanks!
2016:11:07 18:15:10 uwo rtfmftw!
2016:11:08 16:09:07 drewr @norton it's tricky... you have to make sure there's an fdef for the thing you're :replace-ing
2016:11:08 16:43:55 norton @drewr Thank you for the feedback. I posted this question (with more details) to the Google clojure mailing list with an example that is using :replace. Alex Miller responded and corrected my example. It works as expected!
2016:11:08 19:38:04 mpenet s/keys question, I need to express key ::foo is optional, but if it's there ::bar key should be too, since there's no or/and in opt it's a bit tricky I think.
2016:11:08 19:41:04 mpenet I guess i ll split that in multiple s/keys
2016:11:08 19:41:08 zane I don't think you can do all — yes.
2016:11:08 19:41:22 zane That's what I'd do. :relaxed:
2016:11:08 19:41:54 mpenet not sure why and/or is not supported in s/keys :opt, it would be useful
2016:11:08 19:45:57 zane I'd probably replace the #(or … with another (s/keys … so you get enforcement of the values for :foo and :bar.
2016:11:08 19:46:10 mpenet @alexmiller because foo is required like this
2016:11:08 19:46:40 mpenet it's should be optional, but if present there should be :bar too
2016:11:08 19:46:58 zane Ah.
2016:11:08 19:47:01 mpenet @zane yep + s/merge support etc etc
2016:11:08 19:50:00 alexmiller I don't think merge helps at all here
2016:11:08 19:50:19 alexmiller But you could and a predicate that adds this constraint
2016:11:08 19:50:25 mpenet it doesn't, I was mentioning it in the context of a solution with keys vs s/and + pred with contains?
2016:11:08 19:50:53 alexmiller Or or 2 keys specs - one with the pair, one without
2016:11:08 19:51:15 mpenet that's what I am going to end up using, but I wish it was possible to use or/and in opt
2016:11:08 19:51:17 alexmiller Although that may not be very useful
2016:11:08 19:51:38 alexmiller Opt is all optional - it doesn't make sense to have and/or there
2016:11:08 19:52:37 mpenet well it'd make this example way nicer at least, there are other situations like this where it would make sense
2016:11:08 19:52:46 alexmiller Note that this is or, not xor
2016:11:08 19:54:37 mpenet to take the other example mentioned earlier: an option map that could have credentials (optionally), but if ::user is there there must be ::password too.
2016:11:08 22:47:53 bbloom @alexmiller ^ is that a bug?
2016:11:08 22:49:23 alexmiller Is what a bug?
2016:11:08 22:49:32 bbloom sorry, should have elaborated
2016:11:08 22:49:55 bbloom the second evaluation returns {x y} with a :seq node instead of a :map node
2016:11:08 22:50:46 alexmiller Maybe?
2016:11:08 22:51:38 bbloom glad i’m not the only one who couldn’t instantly puzzle out the destructuring specs 😉
2016:11:08 22:51:41 alexmiller Not at a computer so hard for me to look at stuff
2016:11:08 22:52:00 bbloom no worries - i’ll look at it a bit more and maybe file a bug - thanks
2016:11:08 22:56:34 alexmiller I guess the question is whether regex specs like cat should or do work on maps
2016:11:08 22:57:03 zane What's the recommended way to handle specs that are dependent on runtime values?
2016:11:08 22:57:28 alexmiller If so, then seq-binding-form will happen to be satisfied by the map
2016:11:08 22:57:49 alexmiller @zane don't?
2016:11:08 22:58:11 alexmiller Or dynamically generate a spec
2016:11:08 23:10:32 bbloom ok - i’m reasonably sure that is a bug & that the fix is a simple s/spec - making a patch
2016:11:08 23:10:59 bbloom the spec splicing behavior is simultaneously useful and annoyingly subtle 🙂
2016:11:08 23:46:45 alexmiller Yep
2016:11:08 23:48:30 bbloom decided to add a s/and vector? instead, since that seemed more reliable & more accurately spec'd (i think) - if not, happy to tweak
2016:11:08 23:48:55 bbloom clj-2055
2016:11:09 14:33:07 erwin
user => (defn foo [] :clojure.spec/invalid)
ExceptionInfo Call to clojure.core/defn did not conform to spec:
:clojure.spec/args (foo [] :clojure.spec/invalid)
clojure.core/ex-info (core.clj:4725)
user => (defn foo [] (do :clojure.spec/invalid))
#'user/foo
2016:11:09 14:34:25 erwin this is maybe a repl only problem?
2016:11:09 14:36:52 erwin nope also fails in source code
2016:11:09 15:36:02 dominicm I think this is a known bug.
2016:11:09 15:36:31 dominicm I seem to remember mention of there not being much of a good solution to it in spec though.
2016:11:09 15:59:14 alexmiller @bbloom We will have some better solution for regex ops in vectors - comes up a lot in macro dsls
2016:11:09 21:35:36 yonatanel What is a good example of using tags other than human-readable s/explain result?
2016:11:09 23:52:17 alexmiller Gen overrides
2016:11:09 23:52:22 alexmiller Conformed values
2016:11:10 09:30:38 yonatanel Is there something like prismatic schema coercions but with clojure.spec?
2016:11:10 09:35:48 dominicm Conformers?
2016:11:10 10:18:07 yonatanel Cool, thanks!
2016:11:10 11:51:37 vikeri Do I always have to specify my generators with with-gen in the same spot as where I define the spec itself? Or could I assoc a generator later in say, a ns that is only used for testing? I am aware of that stest/check takes the :gen argument but I would like to have it while working in the REPL as well.
I could solve this by overwriting the sdef but would like to know if there was a standard way.
2016:11:10 13:23:16 alexmiller You can use instrument to override generators at test time
2016:11:10 15:12:36 vikeri Does anyone else encounter infinite loops when trying to spec a lazy sequence even though one uses every instead of coll-of?
2016:11:10 15:18:45 jrheard in clj or cljs?
2016:11:10 15:20:21 vikeri cljs
2016:11:10 15:21:10 jrheard yeah, occasionally when i instrument (or is it check?) certain bits of my code in cljs, i see spec hang indefinitely
2016:11:10 15:21:17 jrheard can’t reliably reproduce it
2016:11:10 15:21:23 jrheard <— unhelpful sorry
2016:11:10 15:21:56 jrheard been a month or two since i tried to dig into it last time
2016:11:10 15:30:11 vikeri Alright, I had a spec that was failing, when I updated my spec it worked again...
2016:11:11 00:20:20 bbloom i just ran in to a situtation where (s? any?) matched the individual characters of a string
2016:11:11 00:20:36 bbloom which i guess is not incorrect, but it’s certainly not what i expected 😕
2016:11:11 00:20:56 bbloom what’s the best way to treat strings as scalars?
2016:11:11 00:34:47 bbloom oh - no maybe something else has gone wrong
2016:11:11 00:35:04 bbloom nevermind...
2016:11:11 11:24:32 joshkh noobie question here. can spec be used to force a map into a particular shape? specifically, i'd like to drop all keys in a function's argument that aren't defined in :req-un/:opt-un
2016:11:11 13:46:43 yonatanel @joshkh Just yesterday I learned about conformer. Maybe it will help you.
2016:11:11 13:50:20 alexmiller @joshkh not much in spec that will help you (beyond using a conformer that calls select-keys or whatever way you'd do it without spec)
2016:11:11 13:50:21 joshkh i read about conform but that appeared to return the original data if it conforms, or ::invalid if it doesn't. somehow i missed conformer. thanks 🙂
2016:11:11 13:50:57 alexmiller there is an extension to keys that we've talked about that might help eventually
2016:11:11 13:53:00 joshkh ah, okay. it's not a show stopper for me. i can easily select-keys on the map that's passed to my function. i was just wondering if there was a way to leverage my already written specs.
2016:11:11 13:53:15 joshkh thanks @alexmiller and @yonatanel
2016:11:11 17:21:48 sophiago wondering if i can get some quick input, which might be no better than "just give it a shot incrementally, it's not all or nothing," and @alexmiller you might be well-suited to answer this if time permits: i wasn't paying attention to spec until i saw rich's sale's pitch (so to speak) last night and a lightbulb click re: debugging that symbolic algebra library with the custom numeric tower you looked at a while ago. he made several points that applied well to it: leveraging dynamic typing rather than out of laziness and accountability to stakeholders, the latter i couldn't really claim from having it "working fine" on its own as a library and with a couple clones and then breaking when i myself wrote the edge case that uses it with a library of lazy-seqs. to cut to the chase, i had to table the project and now and am weighing the time commitment of beginning to mark it up with spec vs. circling back to debug traditionally (possibly having to handroll my dynamic dispatch excluding the problem types as in math.numeric-tower). asking because you may vaguely remember the code i think it may be an ideal case because lazy-seqs both expose edge-cases in typing and are a nightmare to debug manually plus given the numeric tower is built with protocols i'm wondering how much that could cut down on my time commitment in writing tests. but then again, i'm not even sure if it would make sense to write the tests out of context in the library of making use of the library that broke it just to get the spec/explain data on it. apologies for the length, but any general advice for using spec to debug edge cases in custom typing would likely be useful given where i'm starting from. thanks!
2016:11:11 17:24:28 alexmiller spec can certainly be helpful in debugging
2016:11:11 17:25:16 alexmiller I’m working on a project right now where I have spec’ed most of the fns and run with instrument on while working at the repl and it’s been a huge time-saver to discover immediately when something doesn’t align
2016:11:11 17:26:08 alexmiller (and actually writing the specs of course clarified thinking and design significantly)
2016:11:11 17:26:31 alexmiller it’s impossible for me to judge how the effort involved in writing the specs compares with other debugging approaches for your particular problem
2016:11:11 17:26:56 sophiago yeah, obviously i knew there was no answer to that question. and it's an incremental thing so i just try it out if i want
2016:11:11 17:26:59 alexmiller however, the idea with spec is that if you do take the effort to write the specs, you can reap many benefits from it
2016:11:11 17:27:09 alexmiller yes, it’s not all or nothing
2016:11:11 17:27:18 alexmiller I’d start at the simplest, most basic functions
2016:11:11 17:27:23 alexmiller use instrument
2016:11:11 17:27:36 alexmiller try check to see if it flushes out any bugs at that layer
2016:11:11 17:27:37 sophiago but more, advice on going that route given i've already exposed the bugs and the particular situation?
2016:11:11 17:28:18 sophiago i guess the first question is whether to spec it at the level of the library that broke it or the broken library?
2016:11:11 17:28:31 alexmiller I’d spec the library
2016:11:11 17:28:56 alexmiller but that’s just my intuition
2016:11:11 17:28:59 sophiago but i know sort of what tests to write i suppose
2016:11:11 17:30:55 sophiago like if i know essentially creating rationals using convolution (simplifying things a bit) seems to be where it's broken then i can spec it around that, but inside the typing library
2016:11:11 17:32:43 sophiago i suppose mainly to separate the aspects of recursive functions and lazy-seqs from the actual type conversion causing the bugs and then get that spec/explain data on those cases
2016:11:11 17:34:03 sophiago then hopefully approach similar libraries this way from the ground up going forward. it's true how we don't use unit tests, but also...a library isn't "working fine" just because you tested it manually in the repl on its own
2016:11:11 17:35:01 sophiago (i may have another one to spec out too...good thing he made the learning curve seem slight. once he got to regex i was like, hmm benefits surely outweigh costs for me
2016:11:11 17:39:14 sophiago i guess i'll come back after diving in, but one thing that comes to mind initially is whether i could use it to test for casting to java numerical types. i can't remember the syntax for one talk, but i have an exact idea of which ones are safe and which are causing the problems and i know you currently just have int? and float? for colls. i assume that must be extensible somehow, though
2016:11:11 17:43:39 sophiago actually maybe that's enough? i'd have to check the source. but if i can say (s/cat (s/and [int? float?] )) or something like that and have it return false for rations and BigInts that would satisfy my contract
2016:11:11 17:46:24 alexmiller they’re just functions - use any predicate you like
2016:11:11 17:46:32 alexmiller check precisely what they mean though
2016:11:11 17:46:48 alexmiller int? is satisfied by any fixed precision Java integer type (byte, short, int, long)
2016:11:11 17:47:02 alexmiller float? is a fixed precision floating point (float, double)
2016:11:11 17:48:03 alexmiller syntax would be like (s/cat :num (s/and int? float?))
2016:11:11 17:48:17 sophiago sounds perfect then
2016:11:11 17:48:30 alexmiller but you could pull the combo out into a separate spec too: (s/def ::fixed-num (s/and int? float?))
2016:11:11 17:48:36 sophiago by nature your two specific numerical predicates exclude all the yucky java stuff
2016:11:11 17:48:45 alexmiller then just (s/cat :num ::fixed-num)
2016:11:11 17:49:12 alexmiller I assume yucky == arbitrary precision? :)
2016:11:11 17:50:17 sophiago don't really have problems with the last one in this case, but the first two i would have excluded in diy dynamic dispatch
2016:11:11 17:51:45 sophiago much of the point of having my own ratios was preventing overflow. the canonical power series code in haskell overrides their division for that reason as well
2016:11:11 17:53:09 sophiago like gcd to 1 == 1N ???
2016:11:11 17:53:12 sophiago no thanks
2016:11:11 17:54:45 sophiago and when you said overriding core math was "not so easy" subtyping was my problem having done it well before you made that statement. i suspect this leads into issues with subtyping, but then...hard to tell when everything is elegant yet opaque...hence s/explain
2016:11:12 17:50:43 wei is there documentation for the new namespaced maps creation/destructuring features?
2016:11:12 18:02:07 wei working out how to use fully-qualified keys in our maps to work with spec, without making our code too verbose. how would I create an alias for keyword namespaces without requiring the namespace like (require [foo.bar :as foo])? sometimes it’s impossible to require a certain namespace because of circular dependencies, but I'd still like to use an alias for creating keywords. also is there a valid syntax for referring to the current namespace in the new namespaced map syntax, something like #::{:a 1 :b 2}?
2016:11:12 19:20:44 wei also when I eval this on the repl I get RuntimeException EOF while reading and then RuntimeException Unmatched delimiter: } :
(def a #:test{:a 1
:b 1})
2016:11:12 19:20:52 wei while this works
(def b #:test{:a 1 :b 1})
2016:11:12 19:48:40 alexmiller You can use create-ns and alias to create an alias
2016:11:12 19:49:13 alexmiller The #:: syntax should work for current ns
2016:11:12 19:50:04 alexmiller Not sure on the reader eof - which repl are you using
2016:11:12 20:07:43 wei thanks for the reply @alexmiller. using lein repl on 1.9-alpha14
2016:11:12 20:11:31 wei #:: not working for me in some cases
foo.bar=> #::{:a 1}
RuntimeException EOF while reading, starting at line 1 clojure.lang.Util.runtimeException (Util.java:221)
RuntimeException Unmatched delimiter: } clojure.lang.Util.runtimeException (Util.java:221)
2016:11:12 20:22:16 gfredericks feels like a bug to me
2016:11:12 20:22:18 gfredericks I reproduced
2016:11:12 20:35:43 sophiago um, is there a reason i could be getting a file not found exception for clojure/spec__init.class and clojure/spec.clj after switching to the 1.90-alpha14? i ran lein deps and even downloaded the zip in case that was necessary. do i need to run lein install in the 1.9 folder? i thought lein would have handled that
2016:11:12 20:46:40 wei @sophiago try lean clean first. if you’re still having problems, could you try including spec in a new project as a sanity check?
2016:11:12 20:47:40 sophiago oh you know what
2016:11:12 20:48:02 sophiago i'm running it from a checkout dependency. so probably need to run lein install in the original directory
2016:11:12 20:49:35 sophiago hmm, ok. lein install seems to have worked, but still getting that error in the other project
2016:11:12 20:49:42 sophiago i'll try to clean it
2016:11:12 20:50:24 sophiago hmm nope
2016:11:12 20:50:53 sophiago oh, i guess i probably just need matching versions of clojure in both? that makes sense
2016:11:12 20:51:16 gfredericks you can only have one version of clojure at a time
2016:11:12 20:51:43 gfredericks they don't have to match, but the dependent will use the depender's version
2016:11:12 20:52:16 sophiago ah so that should explain it
2016:11:12 20:53:35 sophiago ok, all good. back to bug i'm getting started with spec to flush out 🙂
2016:11:12 20:53:53 sophiago should have been able to think through that without talking out loud, but it's been a long week
2016:11:12 20:54:02 sophiago protest marching outside my door rn...
2016:11:13 11:05:45 wei what do I need to do to get fdef to throw errors on invalid return values?
(defn x [] "hello")
(s/fdef x :ret number?)
=> (x)
"hello"
2016:11:13 11:44:18 yonatanel Also, is there a way to require a bunch of same-namespaced keys without specifying the namespace on each one?
2016:11:13 14:08:14 yonatanel Is there a best practice for conformers, like one that can transform a string to java.util.UUID - should I call it ::uuid, ->uuid etc?
2016:11:13 14:50:20 alexmiller @wei not supported. ret and fn specs are checking that a function works, which is seen as a testing time activity supported only by stest/check
2016:11:13 14:52:42 alexmiller @yonatanel: re multimethods, certainly no opposition to it, but needs some work
2016:11:13 14:55:31 alexmiller @yonatanel: re ns keys, use create-ns to create the namespace, then alias to alias it. There is a pending patch to remove the first step.
2016:11:13 14:55:49 alexmiller Re conformers, no convention
2016:11:13 18:13:46 wei @alexmiller what would you recommend for checking inputs at runtime using spec? s/valid? inside of a :pre?
2016:11:13 18:24:45 madstap @yonatanel There's a version of alias in https://github.com/gfredericks/schpec that creates the namespace, then aliases to it.
2016:11:13 18:25:42 wei nevermind, figured it out, the guide does mention :pre/:post
2016:11:13 18:27:17 wei what are some tradeoffs between s/check-asserts and :pre/post?
2016:11:13 18:28:47 alexmiller Check-asserts can be turned on and off or even compiled out
2016:11:14 06:24:18 ikitommi When creating custom Specs, what are the rules for describe*? eval’ing it should return the Spec?
2016:11:14 06:31:25 ikitommi Could the describe* of Specs be used as a serialization format over the wire (sending specs over Transit etc.)?
2016:11:14 13:52:35 alexmiller You should call s/form
2016:11:14 13:53:16 alexmiller That is intended to be a form that can serialized a wire (all symbols should be resolved etc)
2016:11:14 13:53:47 alexmiller There are a number of known bugs and patches on it that I have been working through so it's not 100% yet
2016:11:14 13:54:18 alexmiller s/form uses describe* internally but you shouldn't call that directly
2016:11:14 14:26:03 ikitommi thanks @alexmiller and great to hear the serialization support in the scope of spec!
2016:11:14 20:28:50 hiredman I wrote some code the other day that uses s/form to traverse specs and generates a json schema from the spec, it was way cool
2016:11:14 21:41:24 jasonjckn @hiredman gist
2016:11:14 21:41:30 jasonjckn @hiredman link to code?
2016:11:14 21:43:03 hiredman I can't, but it is a lot like a lisp interpreter, type dispatch + dispatch on the first symbol in a seq
2016:11:14 21:44:00 hiredman if you see a keyword, call s/form to lookup the spec, if you get a seq, dispatch on the first symbol in the seq (s/keys, s/or, whatever) if you get a symbol it is some kind of type predicate like uuid? and you map that to some type in json
2016:11:16 11:59:33 gerrit I wonder if there is something built into spec already, that does the following: conform a value and throw an exception with an explanation of what violates the spec in case conform returns invalid. similar to s/assert but always on, as it should be used for internal conforming and validation, e.g. after reading from a database. I came up with this:
any simpler approaches that I am missing or helper libraries that do this already?
2016:11:16 14:04:40 alexmiller That looks good to me
2016:11:16 14:04:52 alexmiller Nothing in spec for it
2016:11:16 22:05:18 bbloom ^ I ran in to precisely that need almost immediately upon trying spec. I think it would be a good addition to the lib if it had a good name and insightful docstring. Maybe s/enforce and a docstring that contrasts with assert
2016:11:16 22:06:07 bbloom although, to play devil’s advocate with myself, i found that when i did want this, i wanted to add extra data to the error via ex-info
2016:11:16 23:47:47 alexmiller Rich considered it and ended up with assert so I don't think he plans to add it afaik
2016:11:16 23:48:40 bbloom yeah, it’s a reasonable assumption that people would use the wrong one and/or it wouldn’t be particularly useful w/o wrapping it w/ extra data
2016:11:16 23:50:18 alexmiller I expect people will make exactly what they want out of the available parts
2016:11:17 00:00:28 seancorfield We’re slowly creating functions that turn explain data into a series of unique failure codes which we have as a documented part of our API. Our use case is essentially
2016:11:17 11:12:35 ikitommi @seancorfield is it possible to share what kind of code lies in the problems->failure-codes?
2016:11:17 15:07:57 ikitommi Shoud (s/form (s/spec integer?)) return (clojure.spec/spec clojure.core/integer?)? now it returns just clojure.core/integer?
2016:11:17 15:19:46 alexmiller That's a valid spec
2016:11:17 15:20:21 alexmiller So, looks right to me
2016:11:17 15:34:12 seancorfield @ikitommi: it walks the problems and maps path to a code if it non-empty else it maps pred to a code (and it special-cases (contains? % key) which you get for a missing key from req/req-un.
2016:11:17 15:39:52 dergutemoritz @ikitommi That's to be expected, s/form returns a data structure, not a spec. The clojure.core/integer? you're seeing is just the symbol, not the var
2016:11:17 16:11:40 dergutemoritz @ikitommi Ah, alright, makes sense 🙂
2016:11:17 16:12:19 dergutemoritz Looks like s/form is not injective!
2016:11:17 16:19:53 dergutemoritz This is also interesting:
(s/form integer?)
; => :clojure.spec/unknown
2016:11:17 17:25:20 yonatanel Why does this not work?
2016:11:17 19:18:54 alexmiller @dergutemoritz: I've done some of the work on it, just needs a bit more input from rich
2016:11:17 19:37:39 joost-diepenmaat is there a way to do something similar to the removed clojure.spec.test/instrument-all before running (but after compiling) tests with lein test?
2016:11:17 19:39:00 joost-diepenmaat oh I guess I can use fixtures for this
2016:11:17 19:41:11 bfabry @joost-diepenmaat (instrument-all) was replaced with (instrument)
2016:11:17 19:42:17 bfabry and yeah, I think (use-fixtures :once #(do (clojure.spec.test/instrument) (%)))
2016:11:17 21:36:50 joost-diepenmaat That's what I'm using as fixture right now
2016:11:17 21:53:17 gfredericks try/`finally`
2016:11:17 22:01:00 hiredman that was my first thought too, but in theory deftest will catch any exceptions from tests, so the only exceptions could be from fixtures, but your other fixtures should already never bubble out exceptions, because fixtures have some odd edge cases when exceptions are allowed to bubble out
2016:11:17 22:01:25 hiredman if I recall fixtures that bubble out exceptions can hang lein test
2016:11:17 23:14:39 spieden am i abusing spec if i use it for type conversions?
2016:11:17 23:15:05 spieden e.g. an external system returns strings containing integers and i use a conformer to parse them
2016:11:17 23:15:28 spieden it’s mostly convenient except that i have to use with-gen everywhere
2016:11:17 23:19:35 seancorfield Channeling Alex: it’s OK to do a type conversion in a spec if you’re OK with all clients of that spec getting the built-in type conversion 🙂
2016:11:17 23:19:49 seancorfield (but in general avoid conformers in specs)
2016:11:17 23:20:00 spieden hmm ok
2016:11:17 23:20:21 seancorfield FWIW, we have a bunch of specs that accept strings and conform them to non-strings, for use in a REST API.
2016:11:17 23:20:38 spieden ok that’s comforting
2016:11:17 23:21:30 spieden i’ll probably want to circle back and override a bunch of the generators anyway
2016:11:17 23:22:09 seancorfield We use this macro to wrap such specs:
(defmacro api-spec
"Given a coercion function and a predicate / spec, produce a
spec that accepts strings that can be coerced to a value that
satisfies the predicate / spec, and will also generate strings
that conform to the given spec."
[coerce str-or-spec & [spec]]
(let [[to-str spec] (if spec [str-or-spec spec] [str str-or-spec])]
`(s/with-gen (s/and (s/conformer ~coerce) ~spec)
(fn [] (g/fmap ~to-str (s/gen ~spec))))))
so you can say (s/def ::age (api-spec ->long (s/int-in 18 121))) where ->long is a conversion that produces ::s/invalid if the conversion fails.
2016:11:17 23:22:58 seancorfield This also allows (s/def ::yes-no (api-spec keyword name #{:yes :no}))
2016:11:17 23:24:03 spieden cool thanks
2016:11:18 06:15:39 bbloom so instrument only works on functions right? right now i’m kinda wishing i could instrument a dynamic var...
2016:11:18 06:15:54 bbloom w/o having to instrument every read/write to the var manually
2016:11:18 06:16:06 bbloom i guess i could add a validator to an atom...
2016:11:18 07:04:55 bfabry gate var alterations to a single function that's specced and instrumented?
2016:11:18 07:05:09 bbloom yeah, that’s what i just did
2016:11:18 07:05:21 bbloom would be nice to avoid the indirection tho 🙂
2016:11:18 16:30:54 gfredericks @bbloom vars can have validators too, can't they?
2016:11:18 17:51:00 bbloom @gfredericks huh, i guess they can - i wonder why i assumed they couldn’t? d’oh.
2016:11:18 17:55:32 gfredericks Bet they can't in cljs
2016:11:19 10:04:35 triss ok so I’ve got some specs for params of my system...
2016:11:19 10:04:57 triss they specify the range of valid values for each of them.
2016:11:19 10:05:23 triss I’d like to map values between 0-127 to values with in those ranges.
2016:11:19 10:05:41 triss Can I leverage my sepcs in anyway whilst doing this?
2016:11:19 10:32:30 triss currently I’m using this map of fns to coerce values in to the required ranges:
2016:11:19 10:42:03 triss can I not use the specs in someway here?
2016:11:19 11:01:35 triss I guess I’m wondering how I might build functionality like SuperCollider’s ControlSpec(http://doc.sccode.org/Classes/ControlSpec.html) around Clojure’s spec...
2016:11:19 11:01:58 triss any thoughts? sorry for the brain dump. hope that’s ok in here
2016:11:19 12:55:49 alexmiller Aside - swap the order of the preds in ::sample
2016:11:19 12:58:04 alexmiller There are lots of ways to either use custom generators or to each piece or to create a generator that applies a function to the default generator for the overall structure (with gen/fmap)
2016:11:19 12:58:27 alexmiller Not sure what's best here but lots of options
2016:11:19 12:59:35 alexmiller There's a talk at the conj in a couple weeks about making music with spec
2016:11:19 14:26:56 triss many thanks @alexmiller… looking forward to music talk (sadly will be digesting via youtube)
2016:11:19 14:29:30 triss I think I need to familiarise my self with generators before I’ll understand what you’ve just suggested! many thanks though - I know what to read up on next
2016:11:19 14:39:38 triss ok more direct question now, if I have:
2016:11:19 14:40:13 triss how can I find ::every‘s minimum and maximum? is it possible?
2016:11:19 15:37:54 alexmiller so there is intent to make this more visible but things are not quite in place yet
2016:11:19 15:38:19 alexmiller specifically, you can call s/form on a spec to retrieve the original form and then grab the 2nd and 3rd values in the list
2016:11:19 15:38:47 alexmiller however, these compound specs (int-in, double-in, and inst-in) do not have the original form atm
2016:11:19 15:39:07 alexmiller there are several paths towards that and I’ve been discussing with Rich the best one to take but it’s not implemented yet
2016:11:19 16:54:36 triss ah. ok that seems simple enough for an integer range… a lot more complex for a range of doubles...
2016:11:19 20:17:10 alexmiller If you write a spec for the double-in form, then it's just a conform away
2016:11:19 21:01:35 twashing I’m not sure if this is a dumb question. But Is there anyway to include clojure.spec'd functions in a generalized test suite?
2016:11:19 21:06:48 twashing Adding spec’d functions to your running test suite, is sort of discussed in this Google Group thread. But I can’t figure how stest/instrument is used outside of the repl, in a test suite.
2016:11:19 21:27:06 alexmiller check the pinned items here
2016:11:19 23:38:51 twashing @alexmiller Lol!! Too right. Ok, let me have a play.
2016:11:19 23:38:53 twashing Thanks!
2016:11:20 02:09:24 amacdougall I'm stumped on this one—how do I declare that a valid arg is anything fnable, not just fn?? The function params are [pred coll], and I'm not sure how it should be fspecced.
2016:11:20 02:49:07 alexmiller ifn?
2016:11:20 03:03:51 amacdougall Thanks, that was the ticket!
2016:11:20 13:25:33 yonatanel Does s/def order matter? Can I compose with yet to be defined specs?
2016:11:20 13:39:43 gfredericks @yonatanel no and yes
2016:11:20 13:43:10 yonatanel It works for me now, at least with keyword spec (instead of predicate function)
2016:11:20 13:43:26 yonatanel @gfredericks What's the no for?
2016:11:20 13:44:37 gfredericks @yonatanel the first of your two questions
2016:11:20 13:48:03 yonatanel Is it possible to write custom explanations? At least for a conformer.
2016:11:20 14:12:18 alexmiller No
2016:11:20 14:20:39 alexmiller Regarding the earlier question on order - do note that specs delay their resolution, but cache it once they do
2016:11:20 14:20:58 alexmiller This mostly matters when working at the repl
2016:11:20 14:21:49 gfredericks what are the implications of that? re-deffing a spec requires re-deffing all specs that reference it?
2016:11:20 14:22:10 alexmiller Yes - if they've been used
2016:11:20 14:23:00 alexmiller This changed in the alpha that did the perf pass
2016:11:20 14:23:41 gfredericks "been used" like in s/fdef in particular?
2016:11:20 14:30:25 alexmiller No, like you've conformed with it or done something to force the delay
2016:11:20 14:40:32 yonatanel Not directly related to clojure.spec, but is there a point in namespaced keywords that are values and not keys, other than being used in datomic as enum? I'm conforming some strings to keywords and I wonder if I should conform them directly to their datomic enum form.
2016:11:20 14:48:27 alexmiller Depends on your use
2016:11:20 16:09:52 yonatanel How can I define a spec that is like another keys spec but one of the keys is optional this time?
2016:11:20 16:14:34 alexmiller Redefine it
2016:11:20 16:15:17 alexmiller Or define the simpler one first, then s/merge with another s/keys that reqs the additional key
2016:11:20 16:16:09 yonatanel @alexmiller If the first one has the key in :opt and the second in :req, will the second one override the first?
2016:11:20 16:16:41 alexmiller There's no overriding - they are both checked and will be satisfied
2016:11:20 16:17:03 yonatanel I see
2016:11:20 16:23:40 yonatanel I can't find a reason to use s/key's :opt if s/keys checks all map keys anyway. The guide says "We’ll see later where optional attributes can be useful" but I couldn't find an explanation.
2016:11:20 16:47:28 alexmiller It's in docs and it's used for gen
2016:11:20 17:22:38 yonatanel Is it possible to turn on nilable for a whole set of keys, for example if we decide to treat nil values as not existing at all in a certain case?
2016:11:20 18:19:35 alexmiller No
2016:11:20 19:08:17 yonatanel Conformer to the rescue: (s/def ::no-nils (s/conformer (fn [x] (into {} (filter second x)))))
2016:11:20 19:08:34 yonatanel Along these lines...
2016:11:20 21:57:13 alexmiller Bleh
2016:11:20 21:58:06 alexmiller With stuff like this youre just treating spec as a meat grinder rather than actually trying to describe your data.
2016:11:20 22:59:52 yonatanel @alexmiller Interesting point. What's a good use of conformer?
2016:11:20 23:07:57 alexmiller building composite specs (like keys*)
2016:11:20 23:08:25 yonatanel I'm doing some coercion from standard web api (json) to internal representation, checking that something is a string, trimming it, checking that it's not blank after trimming. Stuff like that. It looks great in clojure.spec.
2016:11:20 23:09:54 alexmiller as I’ve said many times here - that’s fine. but keep in mind that you are throwing away information when you use conformers like that. When you register a spec like that, you are making a decision for all future users of your spec. That may be fine, but it’s something to be aware of.
2016:11:20 23:14:59 zane There's no way to generate a spec for the conformed values of some other spec, is there?
2016:11:20 23:46:26 bbloom are people generally putting specs inline? or in separate files/namespaces?
2016:11:21 00:31:51 alexmiller @zane no, although that's been asked a lot
2016:11:21 00:32:05 alexmiller @bbloom: I've done both
2016:11:21 00:34:09 bbloom @alexmiller besides core, do you use a separate namespace for the separate file?
2016:11:21 00:34:26 bbloom i guess yes if you want to load them with a require, no if you’re going to load them with load-file from the main ns?
2016:11:21 00:38:53 alexmiller I think it's easier to use a separate ns
2016:11:21 00:39:52 bbloom @alexmiller one weird thing with that, spec, and namespaced keywords in general, is that i find myself doing this:
2016:11:21 01:44:38 twashing and I’m hoping that such a useful function makes it into the core library.
2016:11:21 01:48:33 alexmiller Not planning on it right now
2016:11:21 04:20:43 bbloom i’m using spec in anger to solve a bug in an otherwise un-spec’d namespace right now
2016:11:21 04:20:53 bbloom i haven’t found the bug i am looking for, but i did find three other lurking bugs
2016:11:21 04:21:00 bbloom it’s kinda awesome.
2016:11:21 04:34:53 twashing @alexmiller Curious as to why not. defspec-test, or something similar, seems to be low hanging fruit for adopting spec’d function tests, into a projects overall test suite.
2016:11:21 05:56:32 kenny @twashing the one issue with the macro is it only counts generative tests as one test in the test summary (e.g. You run 1000 generative tests but the summary results only count it as one). I've been meaning to add it but haven't had time.
2016:11:21 05:59:07 kenny If something like it isn't included in Clojure core when 1.9 is released then we can move it into a micro library instead of a gist
2016:11:21 06:43:14 twashing @kenny Right, I was thinking just that. Kewl :+1:
2016:11:21 14:33:22 alqvist @kenny Would make may day if it found its way into 1.9
2016:11:21 14:40:36 alexmiller in general, spec tests have a different shape than example based tests. I think it’s good to step back and think about why or if it’s valuable to force everything into the shape of something that clojure.test can run. This feels to me like the limits of our tool (particularly reliance on lein test to run and report everything) are affecting our ability to think about the problem. Why does there need to be only one kind of test runner? Why does there need to be only one kind of test? Generative tests are (by their nature) longer running and often don’t mix well with example based tests in a single suite. There is more to say on this.
2016:11:21 15:00:25 bplatz Is there a way to use conform without destructuring the values? (i.e. I’d like have an s/or within the spec but not have change the original value).
2016:11:21 15:42:58 alexmiller no, but you can use a conformer of val to undo conforming an or
2016:11:21 15:44:56 alexmiller Rich toyed with s/nonconforming (still there but not in docs) but I suspect it will probably get removed before final release
2016:11:21 15:54:28 dergutemoritz @bplatz You can fudge it by wrapping your or spec like this: (s/and (s/or ...) (s/conformer val))
2016:11:21 15:54:40 dergutemoritz Though this is probably dangerously close to meat grinder territory again 😄
2016:11:21 15:56:03 dergutemoritz Oops, sorry, I overlooked the first reply of yours @alexmiller
2016:11:21 16:12:40 bplatz The use case here, which I’d think is a decent one, is validating and coercing JSON input - but downstream code expects data in a specific way.
2016:11:21 16:13:39 bplatz Thanks for the s/and tip, I’ll use that for the time being until either it gets officially supported or we just use a different mechanism to coerce the data.
2016:11:21 16:51:38 vikeri How does one go about speccing a multimethod? Can the different defmethods have different :args?
2016:11:21 16:59:58 alexmiller I don’t think s/fdef works on multimethods right now
2016:11:21 17:10:11 vikeri @alexmiller Ok, but if I don’t use multimethods but dynamically dispatches inside the function, can I somehow define a relation between the function arguments i.e.:
> If the first argument is a number, then the second should conform to :c/myspec1 , but if the first argument is a keyword then the second argument should conform to :c/myspec2?
2016:11:21 17:10:32 alexmiller there are a couple options
2016:11:21 17:11:03 alexmiller one is to write an fdef with a :fn spec which can describe a relationship between args and ret
2016:11:21 17:11:25 alexmiller another is to use s/multi-spec which can yield a different spec based on a separate multimethod
2016:11:21 17:11:57 alexmiller the latter is really ideal for functions where the spec is open for extension after the fact
2016:11:21 17:12:25 vikeri I checked into multi-spec, but then the input has to be a map right?
2016:11:21 17:12:34 alexmiller no, it can be anything
2016:11:21 17:12:51 alexmiller as long as you give it a multimethod that chooses the proper spec based on the input
2016:11:21 17:13:45 vikeri @alexmiller Ah, just like multimethods the second argument is not a keyword but a function, but in clojure a keyword can be a function… Then that’s exactly what I need.)
2016:11:21 17:13:50 alexmiller yes
2016:11:21 17:14:02 alexmiller you will likely need a custom retag function too, but that’s not a big deal
2016:11:21 17:14:24 vikeri retag?
2016:11:21 17:15:11 alexmiller used during generation
2016:11:21 17:15:16 alexmiller check the docstring for the details
2016:11:21 23:16:44 paytonrules I have a clojure-spec/clojurescript question if anybody can help. I suspect it’s obvious.
2016:11:21 23:17:21 paytonrules I have the following -
2016:11:21 23:18:51 paytonrules I’m sure the problem is between brain and keyboard, but I’m pretty stumped. Shouldn’t this go kerblooie?
2016:11:21 23:20:20 hiredman check-asserts? governs any s/asserts you have in the code, but you don't have any
2016:11:21 23:22:40 paytonrules I thought fdef would instrument the function - thereby adding the asserts automagically
2016:11:21 23:23:08 hiredman nope
2016:11:21 23:24:17 hiredman when testing you can run instrument, which turns on that sort of thing for testing, but you will also get bits of generative testing happening, so it is not something you generally want to turn on
2016:11:21 23:26:54 bfabry @paytonrules you want to run (s/instrument)
2016:11:21 23:26:57 paytonrules I was going by this from fdef docs
"Once registered, function specs are included in doc, checked by instrument,
2016:11:21 23:27:30 paytonrules But apparently ClojureScript doesn’t have the instrument function
2016:11:21 23:27:58 bfabry really? that surprises me
2016:11:21 23:28:01 hiredman instrument is in clojure.spec.test (not sure what translation to that you need to do for clojurescript)
2016:11:21 23:28:14 bfabry oh yeah, sorry, instrument is in a different ns
2016:11:21 23:28:51 hiredman from what I understand, always on checking is a non-goal for spec
2016:11:21 23:29:51 paytonrules Aha - it was the clojure.spec.test that I missed bit that I missed.
2016:11:21 23:32:35 paytonrules Thanks a lot. It would appear then that spec’ing functions that you aren’t actively doing property based testing on, or debugging, may not be a useful endeavor.
2016:11:21 23:36:07 bfabry @paytonrules I'm not sure that's true, I think there's still value in specing things and having instrumentation turned on for development/test
2016:11:21 23:39:14 paytonrules I wonder how much slower things will be turning on instrument in development will be
2016:11:21 23:40:45 bfabry unless you're getting into speccing function arguments I don't expect instrument will be super slow in a development context
2016:11:21 23:42:02 paytonrules Maybe to turn it on and turn it off. I’m making an HTML5 game so if I spec everything it’ll hurt.
2016:11:21 23:52:08 hiredman if you turn on instrument, invoking functions that are spec'ed will also cause those functions to be genertively tested against those specs
2016:11:21 23:56:58 bfabry @hiredman I'm not sure that's right? I think it only generatively tests functions that you pass as arguments, to make sure the argument satisfies the fdef
2016:11:22 13:02:50 alexmiller No, sorry (it's on the list)
2016:11:22 17:11:39 eggsyntax Is there an easy way to get the keys of a map spec pulled from the registry? I feel like there must be an obvious way I'm overlooking. I can just generate one & get the keys from that, but that feels inelegant (& might be incomplete if it had optional keys).
2016:11:22 17:26:53 alexmiller you can use s/form to get the form version of it
2016:11:22 17:27:11 alexmiller when we release form specs, you could then conform with that to extract the keys
2016:11:22 17:27:29 alexmiller but you can work around that for the time being
2016:11:22 17:30:05 eggsyntax That's actually going to be broadly useful for me; I had overlooked s/form until now.
2016:11:22 17:49:41 alexmiller there are a number of remaining issues with it and I have patches for most of those in the queue
2016:11:22 17:50:01 alexmiller in particular all of the coll specs and keys* have broken forms atm
2016:11:22 17:50:22 alexmiller and maybe conformer (can’t remember if that’s been applied yet)
2016:11:22 17:50:44 eggsyntax OK, gotcha, I'll be cautious. Still, it's a great feature!
2016:11:22 17:51:33 alexmiller those will all be fixed
2016:11:22 19:05:09 zane Is there something I can read to better understand query caching and how to optimize queries?
2016:11:22 19:54:27 seancorfield @zane Could you elaborate? That doesn’t sound related to clojure.spec...
2016:11:22 20:46:47 zane That was indeed for the wrong channel.
2016:11:22 20:46:51 zane Meant for #datomic.
2016:11:22 20:47:11 zane Thanks for taking my question seriously anyway, @seancorfield. 😉
2016:11:22 21:03:15 seancorfield @zane Ah, I wondered if it was for JDBC in which case I’d be happy to field it… in #clojure 🙂
2016:11:22 22:06:23 rickmoynihan Does anyone have any tips on using clojure.spec to validate some data conforms to the spec in a clojure.test? Obviously I can do (is (s/valid? ::spec data) but if its invalid I want to see the explain output…
2016:11:22 22:16:15 alexmiller we were waiting for you to do it
2016:11:22 22:34:41 olivergeorge Just a quick bit of feedback. This is a circular dependency thing. I generated some specs based on my relational database schema
tables became (s/keys ...)
fields became simple (s/def :schem.table/field pred)
one-to-many relations became (s/def :schem.table/rel (s/coll-of :schem/table))
many-to-one relations became (s/def :schem.table/rel :schem/table)
2016:11:22 22:36:10 olivergeorge The slightly fiddly bit was the relations
2016:11:22 22:36:49 olivergeorge I couldn't dump these all in one file because the table specs needed to exist before the relation specs could be processed. (more specifically order mattered or it would throw an error)
2016:11:22 22:37:27 olivergeorge I was hoping all (s/def ...) statements were essentially lazy but perhaps that's not the case for performance reasons.
2016:11:22 22:37:42 olivergeorge I might need to produce a simple example to demonstrate this.
CompilerException java.lang.Exception: Unable to resolve spec: :example.order-matters/b, compiling:(order_matters.clj:11:1)
2016:11:22 22:45:45 olivergeorge My work around is to do all table defs first since they reflect relations. s/keys doesn't require the req/opt specs exist before the s/def
2016:11:22 22:46:59 rickmoynihan @alexmiller: presumably the right thing to do is to extend assert-expr to operate on something like 'valid?
2016:11:22 22:47:20 bfabry @olivergeorge I believe you can avoid that by doing (s/spec ::b)
2016:11:22 22:48:03 bfabry although, that's not what the documentation says, so I'm probably confused
2016:11:22 22:59:36 olivergeorge Thanks @bfabry I did wonder if there might be something like that. I'll experiment more.
2016:11:22 23:00:35 bfabry yeah actually doesn't work at all. my bad
2016:11:22 23:00:49 olivergeorge No worries.
2016:11:22 23:02:30 olivergeorge Actually this works: (s/def ::a (s/and ::b))
2016:11:22 23:02:36 bfabry I wonder if there's a concise definition of which spec macros require definition and which don't
2016:11:22 23:02:43 bfabry ha. neat
2016:11:22 23:03:55 olivergeorge Not sure I'd say neat exactly but it'll do the trick in my case.
2016:11:22 23:04:37 olivergeorge I'd be interested in hearing from @alexmiller if this is technically a bug or if "declare before use" is an implementation requirement/assumption
2016:11:22 23:04:59 olivergeorge @bfabry thanks for the inspiration 🙂
2016:11:22 23:05:25 bfabry lol, glad to "help"
2016:11:22 23:05:35 alexmiller I'd say bug, file a jira
2016:11:22 23:05:55 alexmiller Everything should have delays, but there are a few places missing them
2016:11:22 23:06:04 olivergeorge Thanks Alex. Will do.
2016:11:22 23:06:11 bfabry oh well that's concise enough, awesome
2016:11:22 23:08:01 alexmiller That's either really easy to fix or really hard :)
2016:11:22 23:31:07 alexmiller @olivergeorge good enough, I’ll tweak as needed
2016:11:22 23:55:05 olivergeorge Thanks Alex
2016:11:23 10:24:01 yonatanel If I define specs such as :event/type, it's something that will likely be defined by more projects, so is the best practice to have a fuller namespace such as :org.my.event/type? This will deviate from the shorter form of my Datomic attributes which I don't want to prefix with the organization name all over the place.
(I assume specs can collide being in the same process, while in Datomic I control what I put there)
2016:11:23 12:11:27 yonatanel Perhaps the question is about specs of internal vs external data.
2016:11:23 12:52:38 alexmiller Yes, you should use a sufficiently unique namespace
2016:11:23 12:53:54 alexmiller But if you control what namespaces you're using, "sufficiently" can be pretty short
2016:11:23 13:03:20 yonatanel Being paranoid, maybe it could be nice to have separate non-default registry instances, or attaching a spec to a key when calling s/keys somehow.
2016:11:23 13:54:02 alexmiller Not doing that
2016:11:23 15:41:16 gfredericks @yonatanel if you're paranoid then you can just use sufficiently unique namespaces
2016:11:23 15:48:18 rickmoynihan On a similar note, what’s the suggested way to spec a format you didn’t define, that defines the same key with different values/specs… Basically is there an alternative to using s/keys when I have two different specs that share that prefix? Or should I just move the conflicting definitions into different namespaces?
2016:11:23 15:50:04 gfredericks Different namespaces
2016:11:23 15:50:50 gfredericks It just occurs to me that spec can't describe something where a namespaced keyword has varying meaning
2016:11:23 15:51:01 rickmoynihan gfredericks: that’s the problem I have
2016:11:23 15:51:03 gfredericks Which I imagine is rare
2016:11:23 15:51:27 gfredericks Who subjected you to this
2016:11:23 15:52:37 rickmoynihan well maybe I don’t have the problem
2016:11:23 15:53:20 rickmoynihan I mean I can work around it by defining the specs in different namespaces
2016:11:23 15:53:38 alexmiller If you have that problem, you may be doing it wrong :)
2016:11:23 15:53:50 rickmoynihan entirely possible
2016:11:23 15:53:52 rickmoynihan 🙂
2016:11:23 15:54:20 alexmiller if you have unqualified keys, then you can deal with this using s/keys and :req-un with different namespaced specs
2016:11:23 15:54:44 alexmiller if you have qualified keys, then… use the namespaces to differentiate different semantics
2016:11:23 15:54:46 rickmoynihan alexmiller: yeah I think I just need to define some new namespaces for those extra keys
2016:11:23 15:57:52 rickmoynihan out of interest is it possible to do (s/def :foo/spec ,,,) and have that work with keys:req-un?
2016:11:23 15:58:24 gfredericks Looks normal....?
2016:11:23 15:59:01 gfredericks Every registered spec has to have a namespace
2016:11:23 16:00:48 rickmoynihan gfredericks: What I mean is - is the above possible instead of having to create a new namespace e.g. can you do the above instead of needing (ns blah.foo) (s/def ::spec ,,,)
2016:11:23 16:01:53 gfredericks Yes
2016:11:23 16:02:31 gfredericks Code namespaces and keyword namespaces are mostly independent
2016:11:23 16:05:10 rickmoynihan ok that’s what I thought, it’s just not working - presumably for another reason
2016:11:23 16:06:36 rickmoynihan its ok I fixed it
2016:11:23 16:06:37 rickmoynihan thanks
2016:11:23 16:07:44 rickmoynihan code blindness
2016:11:23 16:08:59 gfredericks The eternal bug
2016:11:23 16:10:45 rickmoynihan yup - that’s one thing about software development, you have to be pretty thick skinned as you’re continually told by the computer that “you suck! Another mistake, are you some kinda noob? Oh and another mistake, you really aren’t good enough…” ad-infinitum… I have a hypothesis that this is why people don’t stick it out as a career
2016:11:23 16:11:44 rickmoynihan you basically have to be a bit of a masochist to stick it out
2016:11:23 16:16:38 yonatanel @rickmoynihan My take on this is that you can't yet have "wonderful" mistakes when programming, unlike art where if you use some new material or do a quick sketch it can still look good and expressive.
2016:11:23 16:21:23 rickmoynihan yonatanel: yeah… programming is so precise one bit out and it's a catastrophic failure and how many trillion bits are we handling these days?
2016:11:23 16:21:46 rickmoynihan it’s a miracle of engineering anything works at all
2016:11:23 18:08:29 seancorfield (for me, it’s sheer stubbornness: “I will not be beaten by an inanimate object!” so I must “defeat” the computer 🙂 )
2016:11:23 18:39:34 rickmoynihan a quixotic quest if ever there was one 🙂
2016:11:23 18:47:40 aengelberg The docstring of s/def indicates that one can provide a symbol instead of a keyword as the name for a spec, but I cannot find any examples of that. What use case is that for?
2016:11:23 18:49:48 bfabry @aengelberg s/fdef is equivalent to (s/def symbol (s/fspec ...))
2016:11:23 18:50:22 bfabry same as defn is (def symbol (fn ...))🙂
2016:11:23 18:52:03 aengelberg @bfabry: interesting, thanks. So when would one use specs that have been set to a symbol?
2016:11:23 18:54:00 bfabry well you're implicitly using them anytime you use instrument
2016:11:23 18:54:46 bfabry explicitly? I dunno. I can't think of a good one off the top of my head, but that doesn't mean there's not
2016:11:23 18:57:12 bfabry you could use the registered fspec's to spec a function to be passed in as
2016:11:23 19:00:53 hiredman as of the 1.9 alphas, macro inputs are checked via spec, so because the spec is named with the same symbol as the macro, the ns macro's inputs are checked with that spec
2016:11:23 19:03:30 aengelberg thanks, that helps a lot
2016:11:23 19:07:05 aengelberg Suppose I have a datatype (or {:foo/name-type :keyword, :foo/name :my-foo} {:foo/name-type :string, :foo/name "my-foo"}). How can I write that in clojure.spec, if s/keys forces me to commit to one possible spec for a given key?
2016:11:23 19:08:12 bfabry foo/name will need to be an or spec, then you can enforce the dependency using and with a predicate
2016:11:23 19:09:52 aengelberg Is the fact that :a/b/c gives me a keyword with namespace "a" and name "b/c" an implementation detail?
2016:11:23 19:10:13 aengelberg Because I could technically leverage that with :req-un to have different versions of a namespaced key.
2016:11:23 19:13:51 hiredman I think that is sort of a design decision baked in to spec, a namespaced key should only ever be mapped to the same kind of thing
2016:11:23 19:15:22 hiredman spec is sort of a global schema for data in your program
2016:11:23 19:16:23 hiredman so similar to having a sql table with a schema where you say a column has some type, you don't really say sometimes this column has this type, some times it has this other, depending on where the data comes from
2016:11:23 19:16:57 bfabry I would guess that a symbol with / in its name is undefined in terms of what will happen
2016:11:23 19:18:08 bfabry "Symbols begin with a non-numeric character and can contain alphanumeric characters and *, +, !, -, _, ', and ? (other characters may be allowed eventually)."
2016:11:23 19:18:37 bfabry '/' isn't in that list, so I'd say you're off the edge of the map if you use symbols with it in them
2016:11:23 19:19:51 aengelberg Got it, makes sense. In this case I am working with an existing program that has keys not set up with that philosophy, so I'm just trying to figure out if there's any way I can fall back to more verbose but more flexible behavior a la prismatic schema. Sounds like not.
2016:11:23 19:20:02 aengelberg @bfabry: I was referring to keywords, not symbols.
2016:11:23 19:20:55 bfabry keywords have the same rules as symbols, but can't contain .
2016:11:23 19:20:57 hiredman there was a discussion about / in keywords, and it comes up over and over again because clojure allows a larger range of inputs than it should in many places
2016:11:23 19:21:18 hiredman answer is just don't do that
2016:11:23 19:21:22 hiredman my answer
2016:11:23 19:21:36 bfabry yes, you can make all sorts of symbols and keywords that the documentation doesn't say is permitted, which is very unfortunate
2016:11:23 19:30:49 bfabry basically, explain is not a macro
2016:11:23 19:31:54 hiredman spec tries to transparently treat things that could be specs (like sets and predicates) as specs
2016:11:23 19:32:03 hiredman so those generally work
2016:11:23 19:32:46 hiredman but there are places where it doesn't, like in the message, there, I suspect its and easy fix if you open an issue
2016:11:23 19:33:54 hiredman s/explain must be calling whatever part of the spec protocol implements s/form on the set, and the spec protocol has a fallback on object which returns the unknown keyword
2016:11:23 19:34:55 hiredman there may already be an issue, spec is only out in alphas
2016:11:23 19:38:19 alexmiller @aengelberg re the unknown in explain - that’s a known bug and will be fixed
2016:11:23 19:38:35 alexmiller I’ve worked on it a bit but waiting for some feedback from Rich
2016:11:23 19:38:56 alexmiller I should probably log an actual jira for it so I can just point people to it
2016:11:23 19:39:46 alexmiller need to unmunge the fn
2016:11:23 19:41:06 alexmiller re your prior question, depending on your goal, you can s/def :foo/name with an s/or or you could define different s/keys specs and s/or those
2016:11:23 19:42:17 alexmiller but the latter seems like it would be papering over the true range of variability in the spec for that attribute
2016:11:23 19:42:44 alexmiller if your goal is really coercion, then there may be other options (conformers) to consider
2016:11:23 22:19:26 wei any best practices for ns qualifying keywords that come out of a db?
2016:11:23 22:20:23 wei in particular is there a function for adding a namespace to all bare keys in a map
2016:11:23 22:23:35 rickmoynihan One observation whilst playing with spec is that it seems to be “static” by design, in that it doesn’t seem to encourage or allow you to programmatically build/modify and refine specs. For example one thing I used to do with schema (especially in tests) was define a generic schema for something and then in specific tests where I want to test something more specifically e.g. the presence of a specific test value in the output, I’d simply assoc a new specific value into the schema, and validate against that.
Is this a fair point in understanding the pro’s and con’s of spec vs other schema libraries?
2016:11:23 22:23:40 rickmoynihan (I should say it seems totally fair to me that spec trades off dynamism for something more static and comprehensible)
2016:11:23 22:26:06 hiredman hard to say
2016:11:23 22:26:19 hiredman the combinators are different
2016:11:23 22:26:33 hiredman instead of associng in, you might s/and a new s/keys spec
2016:11:23 22:27:54 hiredman I think there is a bias in spec towards naming things globally, vs some kind of local names or anonymous things
2016:11:23 22:33:31 hiredman I guess I'd say, I don't think spec is particularly static, but the bias towards global names can make it feel that way some times. And the bias is just a bias, you can do that kind of stuff, you might just have to venture off the beaten path of what is provided in the box
2016:11:23 22:36:11 rickmoynihan hiredman: yeah I think you’re right
2016:11:23 23:40:22 aengelberg How can you define mutually recursive specs?
2016:11:23 23:45:45 zane I don't remember having to do anything special.
2016:11:23 23:59:12 alexmiller Just refer to them by (qualified keyword) name
2016:11:23 23:59:34 alexmiller They delay evaluation so should just work
2016:11:24 06:18:08 ikitommi Runtime defined specs/schemas have also a good use case with web apis & dynamic forms. I’m thinking of creating a MapSpec -kinda Spec Record which would allow non-registered (global) specs to be used and which could manipulated as normal Clojure Map. Adding generated specs to registry at runtime might not be a good idea...
2016:11:24 06:25:33 ikitommi Related: toyed with a simple collection spec macro that allows specs to be created from nested maps/vectors/sets. Should help to define partially anonymous map specs. For things like http query-parameters.
2016:11:24 06:29:54 ikitommi emits just recursively s/keys & s/coll-ofs.
2016:11:24 07:11:34 ikitommi Hmm... but I think after runtime modification of a MapSpec, one would still need to have the full source-code of it for the s/form to work. So adding values at runtime would require both the value and it’s source code to be added. Removing keys would be easy, same as merging existing MapSpecs. Maybe not such a good idea after all.
2016:11:24 10:20:29 rickmoynihan ikitommi: Interesting - I was having similar thoughts about the possibility of building more specialised libraries that can consume ::specs - for more dynamic use cases… e.g. you could imagine something like plumatic being able to consume specs in some circumstances- It seems you’re a long way ahead of me 🙂
2016:11:24 10:20:50 rickmoynihan still very much at the getting started stage with spec
2016:11:24 14:24:17 ikitommi rickmoynihan: we are all still spec newbies, exiting times.
2016:11:24 18:24:36 yonatanel @ikitommi Talk to me about that runtime specs you are planning. I've just written down my requirements for this kind of thing yesterday and would love to collaborate.
2016:11:25 00:08:17 triss hi all. I’ve been looking at extracting the minimum and maximum values from specs created with s/int-in and s/double-in...
2016:11:25 00:08:43 triss What I’ve done is create a spec that recognises the form’s produced those functions...
2016:11:25 00:09:40 triss and then created seperate functions to pull the mins and maxs from the conformed maps that I create with these.
2016:11:25 00:10:08 triss can anyone tell me if this is a sane approach?
2016:11:25 00:11:11 triss or am I totally bonkers?
2016:11:25 02:14:12 alexmiller So those will s/form back to their original form in the future (I know they are a mess now). That plus a spec on those forms makes this just a conform away.
2016:11:25 07:16:47 ikitommi @yonatanel plan is just to test (and built utilities) out things that we are using with Schema. For Schema we have the schema-tools library (https://github.com/metosin/schema-tools) having modified versions of the core functions for maps. In top there are special walkers, transformers and matchers. Using those both at design & runtime. At runtime, have used schemas for example for frontend form validation, and the forms/schemas can change based on the user input. With Spec, there could be either more functions/macros to manipulate the Specs (like s/merge) or a new MapSpec record which could be used like a regular map. Maybe someone has tried the map-approach already? What use cases / requirements do you have?
2016:11:25 09:35:16 slipset Sorry if this has been asked/answered already, but initial googling showed nothing
2016:11:25 09:35:27 slipset I have a map like
2016:11:25 09:35:56 slipset
{:timestamp <jodatime instance> :value number?}
2016:11:25 09:36:27 slipset How do I write a spec for the :timestamp that s/exercise understands?
2016:11:25 09:37:18 slipset My first attempt is
(s/def ::timestamp (partial instance? DateTime))
2016:11:25 09:38:04 slipset but s/exercise doesn’t know how to create instances of this.
2016:11:25 09:52:17 slipset ok, so I see there is a inst? which works on java.util.Date. Might get that to work.
2016:11:25 09:56:01 slipset which is part of clojure-1.9 but not of future-spec 😞
2016:11:25 10:16:11 slipset This is what I ended up with:
2016:11:25 21:03:54 leongrapenthin http://dev.clojure.org/jira/browse/CLJ-2013 - Are there plans to fix this for 1.9? It would greatly increase error message quality. Is work on it welcome?
2016:11:25 22:05:06 alexmiller Sure
2016:11:26 03:47:04 lvh Is there a way to specify that I expect the value of a var to be valid according to a particular spec, such that it is automatically tested?
2016:11:26 03:48:43 lvh Sometimes they’re just data structures, but they might also be the result of a higher order function
2016:11:26 04:59:30 lvh Also; why isn’t there a map-of that takes specs instead of preds?
2016:11:26 05:56:11 seancorfield @lvh map-of does take specs...?
2016:11:26 06:04:44 alexmiller @lvh you could just check s/valid? when you set the var. You can use set-validator! on vars but I’m assuming you’re not actually changing it after you set it.
2016:11:26 13:37:44 kestrel7 Is it possible to inspect a spec to get, e.g., the list of keys defined by (s/keys :req [a b c])? I’ve created a spec for a map and want to retrieve the list of keys expected for that map for a purpose other than checking the spec.
2016:11:26 13:57:43 dergutemoritz @kestrel7 clojure.spec/form is what you're looking for
2016:11:26 14:02:36 kestrel7 Perhaps a noob question but from that how do I extract the actual list of keys from the resulting structure, which in this case looks like the following example but could I guess be more complex? Naively I could just say (last (s/form the-spec)) but that’s not going to be robust if the form contents change.
(clojure.spec/keys
:req
[:k.specs.person/title
:k.specs.person/first-name
:k.specs.person/other-names
:k.specs.person/last-name])
2016:11:26 14:03:27 dergutemoritz That's the tricky bit, hehe
2016:11:26 14:03:33 kestrel7 LOL
2016:11:26 14:04:29 kestrel7 I guess I could use core.match somehow. It would be useful to get the spec as data that’s easier to interrogate. Thanks for the help @dergutemoritz
2016:11:26 14:05:34 dergutemoritz You can do something like that for the simple case of a single s/keys spec. However, when stuff like s/merge is involved, it gets a lot trickier
2016:11:26 14:05:57 dergutemoritz It would be useful to have specs for those forms so you could s/conform them.
2016:11:26 14:06:02 kestrel7 For the moment I’m going to cheat:
(def the-keys [::a ::b ::c]
(s/def ::m (s/keys :req the-keys)
2016:11:26 14:06:30 dergutemoritz I'm not sure if anyone has put effort into defining such specs already. I was gonna take a stab at that myself when I find some time
2016:11:26 14:06:43 dergutemoritz @kestrel7 That cheat won't work I'm afraid
2016:11:26 14:07:04 dergutemoritz The keys vector needs to be literal IIRC
2016:11:26 14:07:25 dergutemoritz Because it's processed at macro expansion time
2016:11:26 14:07:43 dergutemoritz Right, just try it in the REPL and you'll see
2016:11:26 14:09:02 kestrel7 @dergutemoritz you’re right 😞
2016:11:26 14:10:20 dergutemoritz @kestrel7 (apply hash-map (rest (s/form your-keys-spec))) should give you a reasonable data structure to work with for bare s/keys specs
2016:11:26 14:19:29 kestrel7 Here’s my workaround for now
2016:11:26 14:19:51 kestrel7 (defn spec-keys
"Return the :req keys from the spec"
[spec]
(let [spec-form (s/form spec)]
(assert (= 'clojure.spec/keys (first spec-form)))
(:req (apply hash-map (rest spec-form)))))
2016:11:26 14:32:15 dergutemoritz Should do the trick
2016:11:26 14:32:37 dergutemoritz You could destructure spec-form right there in the let but I guess that's just cosmetics 🙂
2016:11:26 15:13:09 lvh @seancorfield huh! I knew that compiled but I assumed it was treating the keyword as a predicate
2016:11:26 16:14:51 dergutemoritz Is there a way to unwrap a regex op context that was previously created with s/spec?
2016:11:26 16:16:17 dergutemoritz Hmm regex-spec-impl doesn't look like it offers something like that
2016:11:26 18:21:15 alexmiller I have specs for all the forms and those will be published, but I'm still working through fixing all the bugs in form first
2016:11:26 18:21:39 alexmiller With those, you will be able to conform any spec form to get at its parts
2016:11:26 19:46:44 dergutemoritz Ah heck 🙂 throws his Saturday afternoon hack into the trash bin
2016:11:26 19:52:19 dergutemoritz Well, I'll keep it around, perhaps I came up with some things that could be added to your implementation
2016:11:27 15:17:27 ikitommi the new specize seems to turn any function into a spec. As collections are also functions, this gives bit unintuitive results:
2016:11:27 15:24:42 ikitommi should the collections have special behavior with specize? would not turn into specs? That would help my case, I would like to identify and convert nested vectors and sets into collection specs via my coll-spec macro, but now they can be used as (strangely behaving) specs. Set seems to be the only one of those three that could act as a spec itself. But there could be s/set for it?
2016:11:27 15:33:41 alexmiller What's unintuitive about the above?
2016:11:27 15:34:55 alexmiller Any function that takes 1 arg can be used as a spec
2016:11:27 15:35:26 alexmiller Sets, maps, and vectors satisfy that
2016:11:27 15:35:56 alexmiller It's very intentional that this works
2016:11:27 15:37:51 ikitommi set as a spec means “one of the values” and it has a generator. what does a vector as a spec mean?
2016:11:27 15:46:09 alexmiller Nothing very useful but it would mean "has a valid index"
2016:11:27 15:47:09 ikitommi ok, thanks for the answers.
2016:11:27 17:46:50 yonatanel Is there a way to directly reuse a spec defined as multimethod implementation? I mean just one of them.
2016:11:27 19:43:32 dergutemoritz @yonatanel How do you mean "directly"?
2016:11:27 19:44:16 dergutemoritz Does calling the multimethod with an appropriate value qualify? 🙂
2016:11:27 21:44:20 yonatanel No, I want to define a similar spec by using the one I already defined with one of the multimethod instances, without calling it with a fake map just for the dispatch value. In the end I just defined the common spec as a keyword but I thought there might be a trick to do that.
2016:11:27 21:49:58 alexmiller Why don't you do the reverse? Define the spec and then have the multimethod return it.
2016:11:27 21:52:11 yonatanel @alexmiller That's what I ended up doing.
2016:11:28 14:30:53 vikeri Hmm, I have an issue with generating data for a multi-spec. It works fine for testing with s/valid? but not when using (gen/generate (s/gen ::myspec)). I have the following setup:
(gen/generate (s/gen ::myspec))
;; #object[Error Error: :option/b is not ISeqable]
I’m on cljs FYI.
2016:11:28 15:48:02 alexmiller the problem is your retag function (first)
2016:11:28 15:49:23 alexmiller re the docs "retag is used during generation to retag generated values with
matching tags. retag can either be a keyword, at which key the
dispatch-tag will be assoc'ed, or a fn of generated value and
dispatch-tag that should return an appropriately retagged value."
2016:11:28 15:50:13 alexmiller in this case, I think the value generated by each method spec is probably fine without modification, so you can just use (fn [val tag] val)
2016:11:28 15:50:57 alexmiller note that you can’t easily use the anon function syntax here as it must be a function with arity 2
2016:11:28 15:51:38 alexmiller you could sneakily use it via something like #(first %&) but I find the fn version to communicate much better
2016:11:28 16:07:29 vikeri @alexmiller Awesome, honestly I read the docs but did not quite understand the retagging part… Now it works as expected.
2016:11:28 19:03:55 yonatanel Will specs be re-resolved on Component system restart/refresh?
2016:11:28 19:06:30 zane Specs are re-added to the registry when you use c.t.n.r/refresh and the like, but any deleted specs will not be removed from the registry.
2016:11:28 19:06:54 zane If that helps.
2016:11:28 19:14:44 yonatanel Thanks
2016:11:28 19:39:27 yonatanel Is this a bug? s/merge specs are not conforming in this case:
2016:11:28 19:39:42 yonatanel "hi" should become a keyword
2016:11:28 19:55:26 alexmiller Merge does not flow conformed values like and
2016:11:28 19:55:42 alexmiller You'll get the conformed value of the last spec
2016:11:28 19:58:35 alexmiller Each spec basically has to independently conform for the merge to succeed
2016:11:28 19:59:51 alexmiller If you swap the order in your merge you should get what you want
2016:11:28 20:02:23 yonatanel Don't you want the behavior to be more like map merge? First determine the keys and then conform them.
2016:11:28 20:02:46 alexmiller No
2016:11:28 20:03:02 alexmiller That's not what merge is
2016:11:28 20:06:14 yonatanel OK. I could read the docs that way.
2016:11:28 20:11:33 yonatanel I see bare (s/keys) doesn't conform either.
2016:11:28 20:12:22 yonatanel I wonder how clear these things should be in the docs or if I'm pushing it too far.
2016:11:28 21:01:18 potetm Is there a way to spec a multimethod where you can add to the spec in an open-ended way?
2016:11:28 21:01:49 potetm Just like you can add to the multimethod in an open-ended way.
2016:11:28 22:51:05 mathpunk I had an idea to define a spec, ::item, that specifies that a thing either has an :id key or that it can have a function called id called on it. I'm not sure, though, if that's a job for protocols instead of specs.
2016:11:28 22:56:21 mathpunk The other thing I'm struggling with is, I can use specs when I define them in the namespace in which I want to use them, but I don't understand how to require them from another namespace. I gathered that spec/def registered them globally somehow, but other-name-space/foo does not seem available.
2016:11:28 23:28:07 hiredman it does register them globally, but if you don't load the code it doesn't happen
2016:11:28 23:31:36 alexmiller @mathpunk re the first question - you could do that but you would need some way to tell that a function can do that (and protocols is maybe the best way) - something like (s/def ::item (s/or :has-key (s/keys :req-un [:id]) :has-fn #(satisfies? HasId %)))
2016:11:28 23:32:13 alexmiller where HasId is (defprotocol HasId (id [x]))
2016:11:28 23:32:46 alexmiller re the second, as hiredman said, they are registered globally and can be referred to by (qualified-keyword) name, assuming you loaded the code that registered them
2016:11:29 08:13:37 mathpunk @alexmiller That's a very helpful example, thank you.
2016:11:29 08:22:45 mathpunk @alexmiller @hiredman: As for loading the code that registered them -- I was missing that using ::foo expands to the whole namespace name i.e. including the name of the app. The 'fully' in fully qualified 🙂
2016:11:29 14:32:39 nwjsmith Are there plans to dynaload the shuffle combinator from test.check?
2016:11:29 14:33:26 nwjsmith Would a patch be welcome for such a change?
2016:11:29 14:34:57 alexmiller I think there’s actually a pending patch that has that included in it
2016:11:29 14:35:07 alexmiller but yes, would be happy to add it
2016:11:29 14:35:31 nwjsmith starts digging through Jira to upvote
2016:11:29 14:36:39 alexmiller which is ready for Rich to look at so I expect that will happen whenever he’s looking at tickets next
2016:11:29 14:36:47 nwjsmith Excellent, thanks Alex
2016:11:29 20:09:48 bbloom create-ns and alias seem relatively useful for spec, but clojurescript doesn’t really support these - any good alternative other than verbose keywords?
2016:11:29 20:11:00 alexmiller prob best to ask this in #clojurescript or #cljs-dev
2016:11:29 20:11:22 bbloom ok
2016:11:29 20:31:17 dnolen @alexmiller I think this is good argument for :alias at the ns form
2016:11:29 20:31:20 dnolen it’s already come up before
2016:11:29 20:31:39 dnolen and I believe continued usage of clojure.spec will drive the desire for such a feature
2016:11:29 20:32:17 alexmiller well you’re talking to the wrong person :)
2016:11:29 23:26:56 jfntn Is there a way to add meta data to spec definitions without re-implementing s/def?
2016:11:29 23:41:32 hiredman metadata?
2016:11:29 23:43:51 hiredman specs are "named" by symbols (which can have metadata attached) and keywords (which cannot)
2016:11:29 23:44:53 hiredman but symbols are not interned, so given a symbol 'a if you associated some metadata, another instance of that symbol will not have that metadata
2016:11:29 23:45:27 hiredman by that I mean, even if you did re-implement s/def you would have a hard time putting metadata somewhere
2016:11:29 23:48:32 alexmiller The answer is no (for now) but it would be useful to have at least a doc string and I'd say that's still a possibility
2016:11:29 23:48:59 alexmiller As hiredman mentioned, the tricky part is where to put it
2016:11:29 23:49:21 alexmiller There are some options but none that are obviously good
2016:12:01 00:15:02 stathissideris hello, do function specs and other specs end up in different parts of the registry?
2016:12:01 00:15:12 stathissideris for example, I have a call like that: (foo {::foo 1})
2016:12:01 00:15:40 stathissideris can I safely do (s/def ::foo integer?) and (s/fdef ::foo …)
2016:12:01 00:17:01 alexmiller No, those will collide (and the fdef will not be used)
2016:12:01 00:17:46 alexmiller Def puts keyword keys in the registry and fdef puts symbol keys in the registry
2016:12:01 00:22:03 stathissideris oh so my example should really be (s/fdef foo …) instead, in which case no collision
2016:12:01 00:22:05 stathissideris correct?
2016:12:01 00:32:23 stathissideris I have another one: is there a shorter form for (let [{:keys [::stats/foo ::stats/bar]} {::stats/foo 2}] foo)
2016:12:01 00:33:42 stathissideris @alexmiller btw, I enjoyed your interview for the defn podcast, quite intriguing (all the stuff about secret future plans for clojure)
2016:12:01 00:33:53 stathissideris thanks
2016:12:01 00:47:54 alexmiller @re collision - right
2016:12:01 00:48:07 alexmiller Re destructuring, yes
2016:12:01 00:49:10 alexmiller Do ::stats/keys [foo bar]
2016:12:01 03:28:50 bbloom i’m totally blown away by how i have a namespace with > 100 defns, only 4 fundamental ones of which have specs, and they help me catch tons of errors - super nice
2016:12:01 03:29:15 bbloom the return value of instrument reminds me of just how little spec you actually need to get some value from it
2016:12:01 03:29:23 bbloom vs mandatory type checking everywhere
2016:12:01 05:57:35 naomarik the official spec guide is very well written, had no issues consuming it to once for deep comprehension and also been able to quickly scan it for answers
2016:12:01 05:57:45 naomarik just want to say thanks for that effort 😉
2016:12:01 09:51:22 mishadoff Hey, I experienced an issue in clojure.spec where :fn key in fdef is ignored (clojure 1.9-alpha14)
Here is MVP demonstrating the problem.
If I call (add 1 nil) I got spec error, but when I call (add 1 2) spec says everything is ok, despite the fact add always returns 2
Can anybody help me?
2016:12:01 10:06:19 joost-diepenmaat @mishadoff AFAICT instrument only checks :args (and apparently not :ret or :fn)
2016:12:01 10:06:53 joost-diepenmaat this is surprising to me too but that’s how it’s documented:
2016:12:01 10:07:11 joost-diepenmaat from the instrument docstring
2016:12:01 10:13:41 pithyless @mishadoff @joost-diepenmaat - this has got to be FAQ #1 on clojure-spec 🙂clojure.spec.test/instrument will only check :args (it’s meant to throw errors if you are using the function incorrectly); clojure.spec.test/check will run generative tests and check that the function is implemented correctly.
2016:12:01 14:41:18 nha @ikitommi the link to yada is wrong. Nice article 🙂
2016:12:01 15:05:31 ikitommi @nha thanks! fixed the link.
2016:12:01 15:49:23 camdez Well, I decided to finally give spec a whirl, moved to 1.9.0-alpha14, and promptly discovered that I can’t get a REPL running because one of my dependencies has a non-conforming defn. 😒 Anything I can do here short of replacing / removing the (slightly busted) library?
2016:12:01 15:51:26 alexmiller Most offending libs have newer versions that fix these problems
2016:12:01 15:52:06 camdez @alexmiller: I’m aware—you actually patched the library in question 😄 (Aleph). But a new release hasn’t been cut yet.
2016:12:01 15:52:20 camdez But I’m curious conceptually what the options are here.
2016:12:01 15:52:40 camdez Certainly I can cut my own release.
2016:12:01 15:52:54 alexmiller You could s/fdef a dummy spec over the defn spec
2016:12:01 15:53:15 alexmiller But might be tricky to get that loaded at the right time
2016:12:01 15:53:18 camdez @alexmiller: Gotcha. But I’d basically be overriding it globally, right?
2016:12:01 15:53:26 camdez For all defns
2016:12:01 15:53:40 alexmiller Yes
2016:12:01 15:53:40 camdez Or no?
2016:12:01 15:53:44 camdez k. Thanks.
2016:12:01 15:59:43 camdez Ah, my mistake—there is an alpha release of Aleph with a conforming defn. (Didn’t release lein ancient wouldn’t show that.)
2016:12:01 23:08:32 bbloom i saw some discussion yesterday about docstrings
2016:12:01 23:08:36 bbloom would be super nice to have some mechanism for that
2016:12:01 23:08:45 bbloom i’ve got some metadata on some objects that i’d like to document better than with a comment
2016:12:01 23:09:38 alexmiller Still a possibility
2016:12:01 23:10:21 bbloom cool
2016:12:01 23:12:00 alexmiller There is not an obvious place to hang it atm, but there are some options
2016:12:01 23:12:15 bbloom syntax: the cause of and solution to all of our problems
2016:12:01 23:17:22 alexmiller I don't mean syntax, I mean where to remember it
2016:12:01 23:17:53 alexmiller Could be easily added to s/def
2016:12:01 23:18:18 bbloom oh - i thought the (a?) challenge was no metadata on keywords
2016:12:01 23:32:58 alexmiller You don't have to provide it as metadata (or store it as metadata)
2016:12:02 13:47:31 stathissideris @alexmiller destructuring like that throws an exception: (let [{::stats/keys [foo bar]} {::stats/foo 2}] foo)
2016:12:02 13:47:56 stathissideris :stats/keys works, but that’s not what I meant
2016:12:02 13:48:36 stathissideris because I have this in my require [spec-provider.stats :as stats]
2016:12:02 14:35:11 dergutemoritz @bronsa How's that wrong?
2016:12:02 14:35:42 alexmiller You need to redef bar for that to take effect
2016:12:02 14:35:52 alexmiller Specs are lazily compiled on use
2016:12:02 14:36:00 dergutemoritz Note that bit from the s/keys docstring: "In addition, the values of all namespace-qualified keys will be validated (and possibly destructured) by any registered specs."
2016:12:02 14:36:03 alexmiller And cached
2016:12:02 14:36:23 bronsa @dergutemoritz ah, then it's not a bug :)
2016:12:02 14:36:26 alexmiller Oh, that too
2016:12:02 14:37:19 bronsa leads to surprising results tho
2016:12:02 14:37:19 dergutemoritz I think this is unrelated to the keys destructuring issue @stathissideris ran into
2016:12:02 14:37:42 bronsa @dergutemoritz no, that's the same issue @stathissideris is having
2016:12:02 14:37:58 alexmiller There are some weird corners with unqualified keys
2016:12:02 14:38:01 dergutemoritz OK then I didn't get the connection, yet 🙂
2016:12:02 14:38:30 bronsa the let binding destructuring specs are defined in terms of s/keys, which is why he's getting ::stats/keys validated against his ::stats/keys spec
2016:12:02 14:39:21 dergutemoritz I wonder if this is intended behavior in this particular situation, though 🙂
2016:12:02 14:39:30 bronsa it feels a bit weird that that happens purely because of how those specs are defined (i.e. it depends on an implementation detail)
2016:12:02 14:40:12 alexmiller We've talked about this a bit and I think this is a case where some effort could be taken to detect and warn (or maybe error)
2016:12:02 14:40:54 bronsa realistically this means using ::keys or ::strs as spec is going to lead to those issues most of the times (unless your ::keys spec is literally a coll of simple-syms)
2016:12:02 14:41:00 alexmiller It is in the ballpark of being open in your map specs or kwarg options
2016:12:02 14:41:01 dergutemoritz An option to disable this behavior for particular keys specs might be useful, too
2016:12:02 14:42:15 alexmiller I'll think about it
2016:12:02 14:42:56 bronsa cool, I'll open a ticket later to track this
2016:12:02 14:45:36 dergutemoritz Great find everyone, I'm happy I probably won't have to run into this case myself some day and scratch my head in confusion 😄
2016:12:02 15:25:48 bronsa attached a proposed solution
2016:12:02 15:34:26 alexmiller We are absolutely not doing that
2016:12:02 15:35:19 alexmiller I don't expect to make any adjustments to keys for this, but rather alter how the destructuring spec is written
2016:12:02 15:36:26 bronsa fair enough
2016:12:03 00:43:53 bbloom s/merge makes me so happy.
2016:12:03 00:44:08 bbloom being able to just mix in key sets for various stages of the pipeline
2016:12:03 00:44:09 bbloom beautiful.
2016:12:03 01:01:00 bbloom the go type system has “struct embedding” which i fell in love with when working in go - lets you accomplish something similar by merging struct fields
2016:12:03 18:48:21 bbloom i’m not quite sure yet i grok the every vs coll-of, or every-kv vs map-of stuff yet
2016:12:03 18:48:51 bbloom is the general idea to use coll-of and map-of by default and only switch to every if your specs are too slow and you don’t need conformance
2016:12:03 18:48:52 bbloom ?
2016:12:03 18:50:46 bbloom also - i’m now running in to a situtation where a spec is not-conforming and it’s just consuming a ton of memory producing the explanation data
2016:12:03 19:14:27 arohner @bbloom I haven’t run into that situation, but that’s my guess
2016:12:03 19:16:21 bbloom yeah, i get the feeling explain needs to become lazy somehow - not necessarily with thunks, but maybe with something like a … that you can ask it to expand further
2016:12:03 19:47:41 bbloom hmm, yeah - i have a structure with LOTS of structural sharing and a ton of redundant data - not unlike the clojurescript ast
2016:12:03 19:48:14 bbloom and explain is starting to become non-useful due to the shear volume of output
2016:12:03 19:48:32 bbloom and seemingly exponential time to return
2016:12:03 19:48:36 bbloom or print or whatever
2016:12:03 19:48:43 bbloom not quite able to pin down a reproduction yet
2016:12:03 21:07:52 alexmiller You can set a custom explain printer if that becomes useful
2016:12:03 21:08:31 alexmiller Just setting print-length will affect the default one though
2016:12:03 21:08:38 alexmiller In case that helps
2016:12:03 21:09:26 alexmiller Re your collection question, yes that's a good first approximation
2016:12:03 21:11:10 bbloom if that first-approximation is true, that you should default to coll-of or map-of, then it seems like the more informative doc-strings should be on those, rather than every and every-kv
2016:12:03 21:11:28 bbloom feel free to totally ignore i even said that 😛
2016:12:03 21:13:12 alexmiller Rich always thought people should default the other way
2016:12:03 21:13:26 alexmiller But in general most people seem to disagree :)
2016:12:03 21:13:26 bbloom preferring the sampling kind?
2016:12:03 21:13:31 alexmiller Yeah
2016:12:03 21:13:45 bbloom hm, i was going back and forth on it in my head
2016:12:03 21:14:04 bbloom seeming like sampling is the right thing if the specs are not intended to be used for contract-style checks
2016:12:03 21:14:13 bbloom which seems true
2016:12:03 21:14:25 bbloom but it feels wrong to get non-determinstic behavior by default
2016:12:03 21:14:29 alexmiller I would actually prefer that the doc for both covered all the details, but you know
2016:12:03 21:14:51 alexmiller Rich isn't big on duplication like that
2016:12:03 21:14:52 bbloom the slippery slope of excessive doc strings - i like the short strings
2016:12:03 21:15:19 alexmiller I have a spec reference page in the works for the docs which will help too
2016:12:03 21:15:28 bbloom i wish the see also was metadata tho, so the cross referencing was more automatic
2016:12:03 21:15:36 bbloom and maybe even let you see-also a spec
2016:12:03 21:15:49 bbloom so like the options that are specified could be generated from common doc strings
2016:12:03 21:15:58 alexmiller If wishes were fishes something something
2016:12:03 21:16:28 bbloom heh, indeed - the simple strings are generally fine
2016:12:03 21:16:44 alexmiller Later, shutting down for takeoff
2016:12:03 21:16:46 bbloom aaaaannny way, i guess i’ll lean on the sampling ones & see how that feels
2016:12:03 21:16:54 bbloom cya, have a good flight
2016:12:03 21:52:45 paytonrules How do you spec a multi-method? Multi-spec seems like it should do it, but it only talks about spec’ing a hash with types. More specifically I have the following - but it never validates the :box parameter.
2016:12:03 22:13:34 alexmiller You can't right now
2016:12:03 22:14:05 alexmiller Probably fixable but I haven't looked at it yet
2016:12:03 22:15:37 paytonrules Oh really?
2016:12:03 22:16:24 paytonrules Kinda shocked - I had assumed it was me the whole time.
2016:12:03 22:17:29 paytonrules Might wanna make it clearer in the docs that it’s not supported. All I could find was some confusion on multi-spec.
2016:12:03 22:17:38 paytonrules Thanks for the help tho!
2016:12:03 22:20:14 alexmiller Yeah, multimethod, protocols, primitive typed fns, and inlined fns will not work right now
2016:12:03 22:21:10 alexmiller Inlined and protocols are probably not easily fixable
2016:12:03 22:37:14 bbloom user=> (binding [s/recursion-limit 1 s/coll-error-limit 1 s/coll-check-limit 1] (clojure.test/run-tests ‘ambiparse.calc-test))
That normally returns in ~1 second, but when instrumented, it seems to be infinitely looping. jstack says: https://gist.github.com/brandonbloom/dfa559f81ad1b6d0980c120d86a2e18e
2016:12:03 22:37:22 bbloom something seems to not be respecting the limits
2016:12:03 22:37:40 bbloom or i’m missing a limit
2016:12:03 22:38:19 bbloom amusingly, i’m working on a parser, soooo i’m running in to exactly the same kinds of infinite loop problems in the program i’m trying to debug 😉
2016:12:03 22:58:51 derwolfe Hi - I'm experimenting with spec and am getting an error that I'm not understanding
RuntimeException Var clojure.test.check.generators/large-integer is not on the classpath clojure.spec.gen/dynaload (gen.clj:21)
. Any tip as to what could be the problem? This is happening when I'm trying to exercise a simple spec I've written.
2016:12:03 23:02:28 bbloom looking at the spec impl, it seems like there isn’t on spec checking level/depth - which might be what i want/need, not sure tho
2016:12:03 23:27:11 alexmiller It's entirely possible some spec impl is not respecting the limit
2016:12:03 23:27:32 alexmiller @derwolfe: you need test.check on your classpath
2016:12:03 23:30:13 derwolfe @alexmiller thanks!
2016:12:04 10:08:44 tslocke I’m guessing this comes up a lot - the fact that conformed values flow through the validation process seems to make it difficult to compose specs. Say I’m writing some preconditions for a function, I have to look at the spec definitions to know if I’m going to get, say, raw values, or [tag value] pairs. Am I missing something?
2016:12:04 10:18:28 tslocke OK just discovered s/unform. Adds some verbosity to some specs, but at least removes the dependency.
2016:12:04 12:47:18 tslocke Hmmm doesn’t seem to work with s/coll-of
2016:12:04 20:03:32 gfredericks @tslocke I saw some speculation a while back that spec will eventually have two versions of s/and; not sure if that's exactly what you were referring to
2016:12:04 21:14:38 bbloom definitely related to the collection stuff tho b/c every makes it random, coll-of makes it every time
2016:12:04 21:42:22 bnoguchi @bbloom: ran into the same issue recently as well
2016:12:04 21:42:47 bbloom @bnoguchi were you able to shrink the reproduction down at all? i haven’t been able to yet
2016:12:04 21:47:02 bnoguchi Not quite. I had to hit the brakes on debugging it but am taking a closer look again at it this week.
2016:12:05 04:43:03 jfntn Finally have a good example of an issue I keep running into with s/keys. Here is a polymorphic payload for a couple of websocket messages:
All I can think of is to have the multi-spec on :msg/data instead. But that dispatch function will be much more complex and will need to infer, from the :msg/data alone, the value that was right there in :msg/id.
Am I missing something?
2016:12:05 04:50:51 alexmiller You can base your polymorphism on more than just a key - it's an arbitrary function
2016:12:05 04:51:50 alexmiller So you could base it on both id and something in msg
2016:12:05 04:52:09 alexmiller Or you could do more than one level of multispec
2016:12:05 04:54:41 jfntn Right, I think I understand that and it seems to be the problem. The :msg/id is all I need to determine the type of :msg/data but it’s only available at that root level
2016:12:05 04:55:30 jfntn With a multi-spec for :msg-data I’d need to look at what keys are in there, and in some cases what values, just to derive a tag was just there in the parent map
2016:12:05 05:14:56 jfntn I've heard Rich's rebuttal of this kind of "contextual polymorphism" , but this example feels like something that will come up in practice, especially as people adopt qualified keywords in their apis.
An ad-hoc binding of a map-key to a spec would be a bulletproof one-liner alternative to what would now have to be a complex, possibly buggy dispatch function.
2016:12:05 05:50:28 alexmiller So are you saying that :mag/data has more than one spec? That seems wrong.
2016:12:05 14:38:18 jfntn Yes indeed. The websocket library is embedding the msg/tag and the msg/data in its payload. It takes a [msg/tag msg/data] and we get this map.
2016:12:05 14:38:51 jfntn So we have multiple specs for both the msg itself (omitted other keys that need specs) and the msg/data, both of which ultimately depend on the msg/tag.
2016:12:05 14:40:36 jfntn Curious what’s your intuition on why this is wrong? This is something that comes up a lot in my experience whit what are now unqualified library apis, but it seems bound to happen more as people adopt namespaced kws?
2016:12:05 14:49:39 jfntn In a nutshell, I can define that :msg/id is either a :foo/msg or a :bar/msg and that :msg/data is either a ::foo/spec or ::bar/spec but I can’t enforce that relationship at the msg level
2016:12:05 16:26:16 alexmiller Qualified names should have meaningful stable semantics. Using qualified names should make this happen less if people are using sufficiently qualified names (which they should)
2016:12:05 16:27:15 alexmiller Could you not wrap an s/and around specs on both of these to add a constraint?
2016:12:05 16:30:47 alexmiller Are ::foo/spec and ::bar/spec just s/keys specs? If so, do they really need to be different or is just (s/keys) to validate all attrs sufficient?
2016:12:05 17:06:55 lvh Is the lack of docstrings on spec/def intentional?
2016:12:05 17:07:04 lvh I could of course just add the metadata to the var
2016:12:05 17:13:56 jfntn alexmiller: yes we have different s/keys specs for the data itself
2016:12:05 17:43:30 alexmiller lvh: it was not part of the initial impl. it’s a highly rated request in the jira system and Rich mentioned it as an idea to me long ago. it’s not obvious to me how to best implement it (where to put the meta) for all types of specs (when you consider things like (s/def ::a ::b) as kws don’t have meta). Do you have a var to add meta to?
2016:12:05 17:43:54 alexmiller jfntn: it would help me to see a more detailed example
2016:12:05 17:44:26 jfntn alexmiller, happy to put something more detailed together and ping you later today
2016:12:05 17:48:06 alexmiller cool, I will be in and out today
2016:12:05 20:17:37 lvh alexmiller: Sometimes I do by accident, yes — but much to your point; there’s no real vars in spec most of the time; so presumably that’s not where tooling would want to go look
2016:12:05 21:19:31 alexmiller This is a known bug
2016:12:05 21:25:32 thegeez ok thanks
2016:12:06 02:37:04 settinghead not sure if this have been asked before, suppose i have 2 cases: ["a" 1 true "b" 2 false "c" 3 true] and [["a" 1 true] ["b" 2 false] ["c" 3 true]], my understanding is (s/+ (s/*cat :first string? :second number? :third bool?)) will validate the first case. my question: how to write spec for the second case where each group is enclosed in a vector/list?
2016:12:06 05:10:18 alexmiller another alternative would be (s/coll-of (s/tuple string? number? boolean?) :kind vector?)
2016:12:06 05:11:21 alexmiller I’d probably prefer the non-regex version in this case
2016:12:06 16:09:48 gfredericks Quoting Mr. Miller on clojure-dev:
2016:12:06 16:09:53 gfredericks > That said, current versions of those libs still exist for those wanting it and we could release point updates to those if needed later.
2016:12:06 16:10:35 gfredericks ⇑ something rich didn't address in the criticism of semantic versioning -- sometimes a one-dimensional version-space is insufficient
2016:12:06 16:11:03 gfredericks maybe appending things to a timestamp-based versioning system would be good enough
2016:12:06 16:11:27 gfredericks version 201612061011.2
2016:12:06 16:25:28 alexmiller no thanks :)
2016:12:06 16:26:28 gfredericks @alexmiller I'm curious what alternative you have in mind
2016:12:06 16:29:42 alexmiller I don’t understand what problem you’re trying to solve
2016:12:06 16:30:45 gfredericks @alexmiller making a "point release" off of an older version -- they exact use case you were describing in the email I quoted
2016:12:06 16:30:54 gfredericks s/they/the/
2016:12:06 16:31:20 alexmiller I don’t see a problem with making a point release off of an older version?
2016:12:06 16:31:28 gfredericks in semantic versioning it's clear what's going on
2016:12:06 16:31:44 alexmiller is it?
2016:12:06 16:31:50 alexmiller :)
2016:12:06 16:32:08 gfredericks if we upgrade from 0.8.0 to 0.9.0, but then want to add bugfixes for users stuck on 0.8.0, we release 0.8.1 and nobody thinks that 0.8.1 is a better version of 0.9.0
2016:12:06 16:32:16 gfredericks but if our versions are just timestamps, there's more confusion
2016:12:06 16:32:34 gfredericks relatively clear -- the non-linearity of the releases is clearer
2016:12:06 16:44:44 alexmiller I don't think Rich was saying that we should stop using versions
2016:12:06 16:53:37 gfredericks me neither
2016:12:06 16:54:06 gfredericks he spent a while criticizing the semantics of semantic versioning, and had a slide that suggested "YYYYMMDDHHMM" or something similar as an alternative
2016:12:06 16:54:57 gfredericks and I was thinking about how that sort of versioning system would need some sort of additional feature to support making changes for older releases
2016:12:06 16:56:49 gfredericks I'm probably communicating this poorly
2016:12:06 16:58:44 naomarik maybe something like clojure el-capitan 201611221245
2016:12:06 16:59:12 naomarik attach a name to a major version
2016:12:06 16:59:59 gfredericks the idea of a "major version" was one of the criticisms
2016:12:06 17:01:19 naomarik i think it was cause the number is meaningless… the name is as well but it’s easier to think of a feature set of android nougat vs kitkat than to refer to them as their version numbers
2016:12:06 17:02:09 naomarik and i think the lack of a clear alternative was to spawn discussions like this
2016:12:06 17:14:49 donaldball I took much of the point to be that you would ship both old and new apis in your library (assuming you didn’t choose to fork to a new library name when you changed apis). Thus if you needed to augment the old api with a new feature, you’d release a new version of the whole library.
2016:12:06 17:28:33 sparkofreason Is it possible to attach a generator to a predicate without defining an explicit spec? For example, I want to make a predicate queue?, and generate instances of clojure.lang.PersistentQueue from a spec like (s/coll-of int? :kind queue?), similar to how it works for vector?.
2016:12:06 17:34:21 gfredericks @donaldball I'm not sure that would work in the case I brought up, where your library requires clojure 1.8 but you want to augment older versions for users that haven't upgraded to 1.8 yet
2016:12:06 17:34:44 gfredericks @dave.dixon I doubt it
2016:12:06 18:20:15 sparkofreason @gfredericks Certainly doesn't look like it. clojure.spec.gen/gen-builtins provides a nice shortcut for predicates in clojure.core. It would be nice if this was extensible.
2016:12:06 18:23:56 gfredericks hmm
2016:12:06 18:42:08 alexmiller I suspect making that extensible might also invite a world of conflicts
2016:12:06 18:53:59 kenny @alexmiller Why was that done for the predicates in clojure.core then? Is the purpose of this purely for backward compatibility? It seems like the general workflow is write your predicate, write a spec that is defined as a call to that function, include a generator on that spec, and use that spec but clojure.core varies from this workflow.
2016:12:06 18:54:07 kenny It almost seems like you don’t even need predicate functions at all with spec.
2016:12:06 18:55:26 kenny A call to clojure.spec/valid? could entirely replace predicates
2016:12:06 18:55:33 kenny I think.. 🙂
2016:12:06 18:56:16 alexmiller That doesn't make any sense to me :)
2016:12:06 18:56:46 alexmiller valid? works be evaluating the predicates
2016:12:06 18:58:40 alexmiller The workflow seems exactly the same to me except the generator is already provided
2016:12:06 19:00:27 kenny Right. But why is it written like that in clojure.core instead of providing actual specs for us to use?
2016:12:06 19:00:41 alexmiller Preds are specs
2016:12:06 19:00:59 kenny But there is no way to attach a generator to a pred without defining a spec
2016:12:06 19:01:16 alexmiller Yes
2016:12:06 19:01:49 hiredman the generator is part of the identity of a spec
2016:12:06 19:02:10 kenny But clojure.core is different. I’m wondering why it is different
2016:12:06 19:02:13 hiredman new generator == new spec
2016:12:06 19:02:18 alexmiller The core fns have built in mappings because that lets you build up specs in many cases without needing to write custom gens
2016:12:06 19:02:49 alexmiller Like (s/and int? even?)
2016:12:06 19:04:08 kenny So then why is there no way for us to extend that built in mapping so we can so similar things with our own predicates?
2016:12:06 19:11:13 alexmiller I was not part of that decision process so I can’t say what exactly went into it. but certainly you can create specs by using s/spec or s/with-gen to add custom gens
2016:12:06 19:12:01 alexmiller my suspicion is that it opens a bunch of complexities with respect to libs providing their own competing generators
2016:12:06 19:12:22 alexmiller and maybe that was just a door Rich and Stu didn’t want to open
2016:12:06 19:14:16 kenny Yeah I’m just not sure. It’d be great to know if that was actually what they decided. It seems useful but I am also concerned that it could lead to complexities.
2016:12:06 19:14:50 kenny Technically you could have Spec extend IFn so you can define your predicate on the spec. But that feels a little hacky
2016:12:06 20:12:22 danboykis how do I spec a map whose keys are strings i.e. {"foo" "bar"}?
2016:12:06 20:14:14 alexmiller I don’t know that I have enough info to actually answer your question :)
2016:12:06 20:15:37 danboykis I thought there was maybe some way to instrument it on top of s/keys
2016:12:06 20:16:03 danboykis contains? works
2016:12:06 20:16:06 danboykis thanks
2016:12:06 20:16:48 hiredman the problem with describing things entirely in terms of predicates is you lose the structure
2016:12:06 20:18:35 danboykis hiredman, yep, I was thinking if it becomes to bothersome to do contains? everywhere to convert the string keys to keywords instead
2016:12:06 20:19:10 danboykis i'm not sure how to best handle a case with a large map that has a bunch of strings for keys
2016:12:06 20:20:46 alexmiller one hack is to (s/and (s/conformer clojure.walk/keywordize-keys) (s/keys :req-un [::a]))
2016:12:06 20:21:00 alexmiller (didn’t try this, so could be typos there but you get the idea)
2016:12:06 20:22:45 danboykis alexmiller thanks, I'll play with it
2016:12:06 20:23:54 alexmiller @hiredman fyi, we consider the protocols internal right now and there is a reasonable chance they will still change
2016:12:06 20:24:00 hiredman sure
2016:12:06 20:24:06 hiredman still alpha, etc, etc
2016:12:06 20:24:18 alexmiller protocol might even go away
2016:12:06 20:24:29 hiredman I was disappointed without much work that took to do 🙂
2016:12:06 20:25:13 alexmiller the intention is that most people shouldn’t do it :)
2016:12:06 20:43:11 hiredman I don't want to do it, but I also want to use spec as a data regex (including maps) without having to fiddle with the global registry
2016:12:06 20:51:31 donaldball Do you mean you want the ability to spec a map with unnamespaced keywords without registering corresponding namespaced keywords in the spec registry?
2016:12:06 20:53:17 hiredman maps without keyword keys exist too
2016:12:06 20:57:11 hiredman you can easily say, with predicates, #(and (map? %) (contains? % "foo") (uuid? (get %) "foo")), but then you run in to issues if you want to spec the data
2016:12:06 21:00:11 hiredman you can replace uuid? there with a call to valid? and some spec, but to conform and explain on the original spec it is a black box
2016:12:06 21:01:02 hiredman one solution would be to replace predicates with pairs of predicates and navigators
2016:12:06 22:12:28 eraserhd How could I compute a key for s/keys?
2016:12:06 22:14:00 eraserhd (`is-valid` is effectively (is (s/valid? ...))
2016:12:06 23:22:01 gfredericks @bbloom my guess is the intention is when you test something, you instrument all the stuff it calls to make sure that stuff gets called correctly; checking rets is for when you're testing that function in particular, which you can do with clojure.spec.test/check, or by calling/asserting manually
2016:12:06 23:22:49 gfredericks anyhow if you hate this you could add an alternative to instrument in schpec
2016:12:06 23:24:35 bbloom ugh. i KINDA understand the thought process, but i don’t really buy it
2016:12:06 23:24:59 bbloom i have a recursive function that has a small :ret violation deep inside it and i want to catch the error earlier than the root call from the unit test
2016:12:06 23:26:19 bronsa s/assert in a postcondition ¯\(ツ)/¯
2016:12:06 23:27:04 bronsa (not saying I particularly like it)
2016:12:06 23:33:03 bbloom yeaaaah, i’ll just use s/assert
2016:12:06 23:33:38 bbloom i’m just annoyed that these specs weren’t getting run this whole time i thought they were
2016:12:07 00:35:33 bfabry @bbloom I fully expect someone will write a dev lib that provides an overinstrument or whatever shortly after 1.9 is released
2016:12:07 00:36:34 bbloom it’s not a big deal - it was just 1) surprising and 2) not convincingly justified, despite being unsurprisingly/welcomely accompanied by a rationale
2016:12:07 00:36:55 bbloom gave me feedback, for whatever that’s worth, and moved back on to writing code 🙂
2016:12:07 01:02:31 tetriscodes Can someone tell me if I’m off here?
2016:12:07 01:03:57 tetriscodes I would think that coll-of would call a spec multiple times that has a generator and would produce unique values each time
2016:12:07 01:05:00 alexmiller what is the line that produced the output?
2016:12:07 01:05:40 alexmiller what does point-gen do?
2016:12:07 01:07:06 alexmiller you should never use a random-generating thing in your generator
2016:12:07 01:07:28 alexmiller when you do that, you rob test.check from being able to a) make reproducible results and b) shrink on error
2016:12:07 01:08:10 alexmiller any source of randomness should come from a generator
2016:12:07 01:08:34 alexmiller so something like
2016:12:07 01:08:53 tetriscodes Ok, I’ll swap that with double-in
2016:12:07 01:09:22 seancorfield also with-gen is only going to call the function once to get the generator, right? so this gets a generator for #{(point-gen)} so it calls point-gen just once and gets #{[some, random, values]} and then just generates coords from that single-valued set.
2016:12:07 01:09:49 seancorfield or does the function argument get called every time it needs a generator?
2016:12:07 01:10:02 alexmiller yeah, you really want to use gen/fmap or gen/bind to do this kind of thing
2016:12:07 01:11:27 alexmiller your point-gen could be (s/gen (s/tuple (s/int-in -180 180) (s/int-in -90 90) (s/int-in -20000 0)))
2016:12:07 01:11:32 alexmiller something like that
2016:12:07 01:13:07 tetriscodes Ok, thank you. I’ll do some refactoring
2016:12:07 01:13:19 tetriscodes I’m trying to generate GeoJSON with spec
2016:12:07 01:13:26 alexmiller sounds fun :)
2016:12:07 01:13:33 tetriscodes I’ll post to it when I’m done
2016:12:07 11:41:26 mpenet @tetriscodes I have done something similar as part of one of our internal projects (just for the subset used by twitter)
2016:12:07 11:42:17 mpenet I you need (rev)geocoding data check mpenet/sextant on github
2016:12:07 11:42:34 mpenet It was useful for gen
2016:12:07 14:14:42 vikeri I guess I’m missing something about how cat should be nested, or why does that fail?
2016:12:07 14:16:23 alexmiller If you need to nest regexs, wrap the inner in s/spec to start a new nested sequential context
2016:12:07 14:51:55 jfntn Don’t understand what’s going on here
2016:12:07 14:56:51 vikeri @alexmiller Ok, if I don’t need regexes, is there some other way I could spec a vector of one or possibly two different elements? (s/tuple boolean? (s/? map?)) did not work.
2016:12:07 15:44:36 alexmiller @jfntn the ::foo inside the map is not valid according to the spec you have defined for ::foo
2016:12:07 15:44:55 alexmiller (also, identity is not a valid retag function - you’d hit that with gen if you tried it)
2016:12:07 15:54:23 alexmiller same problem
2016:12:07 15:55:00 alexmiller the ::foo in the map is not a valid value for the ::foo multispec
2016:12:07 15:56:07 jfntn ok so the problem is that the kw that registers the mspec is also present in the map, correct?
2016:12:07 16:05:40 tomc I've got the same spec returning false for s/valid? and "Success" for s/explain-str, for the same data, which I didn't expect to be possible. Is there an obvious reason this is occurring? Can someone point me to something in the docs?
2016:12:07 16:07:11 alexmiller @tomc shouldn’t ever happen. can’t say more without more info
2016:12:07 16:07:31 alexmiller @jfntn yes - s/keys will validate specs for all keys in the map
2016:12:07 16:08:15 tomc @alexmiller ok, let me verify it's not something on my end (which it almost certainly is) and if I can't figure it out I'll give you more info here. Thanks!
2016:12:07 16:09:13 jfntn Makes sense, thank you
2016:12:07 16:12:29 tomc Turns out I had a call to s/keys that didn't conform to s/key's api. If only there were some way to prevent that type of thing 🙂
2016:12:07 16:15:53 alexmiller ha, yes
2016:12:07 16:16:25 alexmiller I have created specs for spec, but it’s somewhat problematic to actually use it without encountering an infinite loop of checking
2016:12:07 16:16:46 alexmiller that said, I’d be happy to see a jira for whatever problem you had as that’s something we could prevent in the code
2016:12:07 16:22:51 tomc Probably not worth it. I'm doing some whacky metaprogramming for reasons not worth getting into, and it resulted in me evaluating code like (s/keys :req-un [nil])
2016:12:07 16:27:09 alexmiller fair enough
2016:12:07 18:42:24 denik how do I spec a hash-map where the keys are integers?
2016:12:07 18:44:48 denik ah map-of!
2016:12:07 19:22:47 alexmiller Yep
2016:12:07 22:25:57 bhauman Thinking about a spec api so that tooling can register listeners for fn spec failures.
2016:12:07 22:29:05 bhauman I would need something like this to reliably report spec errors in tooling like figwheel
2016:12:07 22:31:16 bhauman in JS especially because its extremely difficult and fairly invasive to get any sort of reliable top level Exception handling
2016:12:07 22:33:51 bhauman cc/ @dnolen
2016:12:07 22:52:00 bhauman cc/ @cfleming
2016:12:07 22:58:14 lvh Is there an efficient way to ask for a string of specific length range? The obvious (s/def ::layout-row (s/and string? #(<= 8 (count %) 11))) breaks such-that generation
2016:12:07 23:02:35 dnolen @bhauman another thing to consider is there’s really no harm in patching that for custom tooling
2016:12:07 23:02:44 dnolen vars exist for a reason 🙂
2016:12:07 23:05:43 alexmiller @lvh you will need to dive into gen for that
2016:12:07 23:05:54 lvh That’s what I figured, thanks 🙂
2016:12:07 23:06:13 lvh Is there a way to figure out which spec failed to generate when I see:
java.util.concurrent.ExecutionException: clojure.lang.ExceptionInfo: Couldn't satisfy such-that predicate after 100 tries. {}
2016:12:07 23:06:21 alexmiller no
2016:12:07 23:06:21 lvh (I’m running check over the entire ns)
2016:12:07 23:06:31 lvh Is that gonna change, or is there a deep reason for that
2016:12:07 23:06:57 alexmiller the problem here is that test.check doesn’t have a way to do so (or doesn’t in the version used by spec at least)
2016:12:07 23:07:25 alexmiller basically, spec builds a big composite generator and says gimme and gets back either a sample or a failure
2016:12:07 23:07:41 alexmiller I know Gary has been making some changes to make this better in test.check but I haven’t had time to look at them yet
2016:12:07 23:07:54 alexmiller and it’s possible that we can do better if we leveraged that
2016:12:07 23:08:12 alexmiller there is certainly no desire for that to be information free :)
2016:12:07 23:10:02 lvh Ah, I see 🙂
2016:12:07 23:10:11 lvh Yeah, so I’ve made the same mistake several times now
2016:12:07 23:10:30 lvh in that I forget how to write a clojure.,test test that is actually doing stest/check
2016:12:07 23:11:00 lvh And just do (t/is …) and forget it’s lazy, and then only way later discover that the checks were just not running and I’ve had a bug all along
2016:12:07 23:11:42 alexmiller yeah, I’ve done that too :)
2016:12:07 23:12:10 alexmiller it is intentional that it’s lazy
2016:12:07 23:16:34 lvh Yeah, it makes sense. It’d be fine if there was an stest/deftest IMO
2016:12:07 23:22:15 alexmiller unlikely we will add that
2016:12:07 23:22:42 alexmiller in general, the lifecycle and use of generative or check-based tests is much different than example-based clojure.test tests
2016:12:07 23:23:13 alexmiller I’m dubious that combining them in a single suite makes sense
2016:12:07 23:51:47 lvh Hm; I’ve been doing that by having lein test filter for me based on tags
2016:12:08 09:28:03 artemyarulin Hello, just started to play a bit with spec. Is there any way to make it shorter? Can I somehow embed ::first-name inside ::person without separate def?
2016:12:08 14:09:25 thegeez For the :pre match I want to match the sequence 1 2 3 in that order as the start of a larger sequence. Is there an easier way to write the spec for the :pre part? I tried stuff like #{[1 2 3]}
2016:12:08 18:23:38 bbloom in short, System.TimeZone was broken in a bunch of ways, so they made System.TimeZone2, but people freaked out about that name, so they caved and called it TimeZoneInfo.... leading to substantial confusion
2016:12:08 18:44:52 gfredericks > using numeric modifier is not scalable in the long term
2016:12:08 18:45:10 gfredericks eventually the numbers just get too big
2016:12:08 18:45:27 gfredericks mathematicians tell us that there is no limit to how big numbers can get
2016:12:08 18:47:22 eraserhd I wonder what the smallest number which isn't representable with the amount of information present in the universe is.
2016:12:08 18:48:08 bbloom knowing that number would change the amount of information in the world, hence changing that number
2016:12:08 18:48:27 eraserhd Well, I obviously couldn't know it.
2016:12:08 18:49:03 bfabry I remember watching a ted talk on this, it's interesting. but yeah, what's to stop me saying "the biggest number the universe can represent, times two"
2016:12:08 18:49:34 eraserhd Right, that's a representation.
2016:12:08 18:49:35 bfabry we could just redefine 1 to be "the biggest number the universe can represent" 😛
2016:12:08 18:49:52 bbloom bfabry: the trick is not to abstract over numbers, but to abstract over the operations which generate those numbers
2016:12:08 18:51:01 gfredericks use BB(11111), as scott aaronson suggests
2016:12:08 18:51:20 eraserhd @bfabry If I could find the number by an algorithm, the algorithm would have to require more information to represent than present in the universe to represent.
2016:12:08 18:55:43 gfredericks BB(n) exposes an ambiguity in the question, I suppose -- if a number is platonically well-defined but can't be computed, does that count?
2016:12:08 18:56:10 eraserhd Interesting
2016:12:08 18:56:18 eraserhd I just got to that part
2016:12:08 18:56:37 gfredericks the fun part is I don't even know what I mean by "platonically well-defined"; it's just an emotion
2016:12:08 18:57:55 bbloom as fun as i find numbers, i was hoping somebody had some insights in to the hardest problem in computer science: the paucity of names 🙂
2016:12:08 19:01:01 gfredericks @bbloom what do you think of the renaming you linked to?
2016:12:08 19:01:37 bbloom honestly, i liked the name TimeZone2 - it means i don’t need to look at the docs for TimeZone or TimeZoneInfo to know which does what
2016:12:08 19:01:40 bbloom basically i just ignore TimeZone
2016:12:08 19:02:04 bbloom i can understand the desire to not have to write TimeZone2 in places
2016:12:08 19:02:15 gfredericks because it's ugly?
2016:12:08 19:02:35 gfredericks you feel like you have to memorize a version number for every class/API you learn to use?
2016:12:08 19:02:45 bbloom yeah, but specifically it’s ugly for an ugly reason: it showcases a mistake
2016:12:08 19:02:56 bbloom it’s not my mistake, but there’s still a human error as an accident of history in my code
2016:12:08 19:03:05 bbloom now, that’s fine, b/c everything is human errors and accidents of history
2016:12:08 19:03:19 bbloom my next thought is: ok, so let’s use aliases, which is what rich suggested too
2016:12:08 19:03:25 bbloom but i’m not so sure that’s a good solution either
2016:12:08 19:03:38 gfredericks ns aliases in particular? :rename too I guess
2016:12:08 19:03:48 bbloom yeah, those combined
2016:12:08 19:03:52 bbloom or even just a simple def
2016:12:08 19:04:13 bbloom that helps for migration, which is key, but it still kinda sucks b/c of the nature of context
2016:12:08 19:04:32 bbloom if you and i have a conversation about TimeZone, how do you know which version i mean?
2016:12:08 19:04:55 gfredericks yeah there'll always be a version number at one of the levels
2016:12:08 19:04:59 bbloom by utilizing rename/alias, you wind up in the same place you started with if you had used semver or whatever (shudder)
2016:12:08 19:05:23 bbloom yeah, well ecosystem name is the easy one
2016:12:08 19:05:27 bbloom you just invent a brand name and you’re good
2016:12:08 19:05:41 gfredericks every 5-10 years we declare bankruptcy on all this and start a new programming language
2016:12:08 19:05:50 bbloom like instead of Clojure 2.0 you just fucking call it Twojure or something and you can change whateve ryou want, but at the cost of bifurcating the community
2016:12:08 19:05:54 bbloom yup lol
2016:12:08 19:06:06 gfredericks and then everybody rushes in to write the first http library
2016:12:08 19:06:32 bbloom here’s where i’d use a /giphy lion king circle of life
2016:12:08 19:07:04 gfredericks I don't think there's any Super Happy approach to this :(
2016:12:08 19:07:57 bbloom i’d settle for a reasonable principal by which i can rationalize my decisions and remain mostly happy that i didn’t LOSE a battle with the laws of nature
2016:12:08 19:08:27 bbloom alias & rename give us some powerful tools
2016:12:08 19:08:33 bbloom but i feel like something is missing
2016:12:08 19:08:45 bbloom one problem i’ve run in to a bunch is the impl vs interface namespace problem
2016:12:08 19:08:58 bbloom spec sorta has this problem in that it has some public macros that it’s like “don’t use me directly please"
2016:12:08 19:09:14 bbloom and then there’s core.async that has the impl namespaces and then redefs an indirection to export all the functionss
2016:12:08 19:09:38 bbloom i wonder if there was some way to abstract over aliases and renames that you could more easily address this
2016:12:08 19:09:50 bbloom not sure what that would look like
2016:12:08 19:10:18 bbloom but if we’re going to use names as a tool to achieve accretion (and i see no other way, since everything in symbolic reasoning involves names), then i think we need more tools for names
2016:12:08 19:10:51 bbloom that’s as far as i’ve managed to think here </rant>
2016:12:08 19:11:04 gfredericks rich wanted a global mapping from namespaces to maven artifacts
2016:12:08 19:11:11 gfredericks or seemed to at least
2016:12:08 19:11:17 gfredericks this is sort of a different topic
2016:12:08 19:12:13 bbloom he seemed to be making two points: 1) we’ve got bad abstractions: project files, version numbers, etc and 2) accrete, don’t break, use names to do that
2016:12:08 19:14:05 bbloom if it were up to me, we’d abolish artifact ids etc as much as possible - namespaces have much nicer properties
2016:12:08 19:14:23 gfredericks just publish individual namespaces? with versions?
2016:12:08 19:14:25 bbloom you could probably define some information-preserving algebra of them. ie merge two sets of names, aliases, etc
2016:12:08 19:15:20 bbloom yeah, i think so
2016:12:08 19:16:05 bbloom maybe published dates instead of versions, since we’re turning version trees in to sequences
2016:12:08 19:17:24 gfredericks he said the useful thing about artifacts is that they intentionally tie together particular versions of particular namespaces
2016:12:08 19:17:47 bbloom i mean do they tho? only if you bundle your deps
2016:12:08 19:20:00 nicola Hi all. I need spec for map such as: if key - some pred, then value - spec1, else if key - other pred, then value - spec2?
2016:12:08 19:21:24 gfredericks @bbloom I mean if your artifact contains multiple namespaces, which it probably does, they are free to assume things about each other because you know that you'll get exactly the same version of all of them
2016:12:08 19:25:18 bbloom @gfredericks ah, i understand what you’re saying. yeah, in that sense those are actually tied together and tested together. i think that’s not incompatible with the idea of “eliminating” artifacts or projects tho… you'd just use the context of a project in resolving versions of namespaces, ideally in some content addressable way maybe?
2016:12:08 19:26:26 bbloom so if i have namespace foo and i require foo.bar, then i have a project that says “hey, i’m going to publish namespaces #”foo.*”, then when those get published they get tagged with some artifact bundle in some way, but i don’t actually think about that, i think about the version of the nodes in that dependency graph i point to directly
2016:12:08 19:26:58 bbloom so you resolve internal dependencies within a project using immutable references, like hashes, and then you resolve external dependencies using lamport logic
2016:12:08 19:27:03 bbloom eg by intersecting date ranges
2016:12:08 19:28:56 gfredericks If backwards compatibility is reliable presumably your date range can be open on the right side and intersections are trivial
2016:12:08 19:29:19 bbloom right, but of course people make mistakes, so you’d probably need two features:
2016:12:08 19:29:28 bbloom 1) right bound to say “i no longer trust this maintainer"
2016:12:08 19:29:34 bbloom and 2) a black list of bad versions
2016:12:08 19:29:41 bbloom so your timelines could have right bounds or holes
2016:12:08 19:30:19 gfredericks If intersections are empty then the build tool reports a failure?
2016:12:08 19:30:42 bbloom yup
2016:12:08 19:30:52 bbloom and you always select the most recent version from the intersection
2016:12:08 19:31:15 bbloom then layer on top the standard dependency pinning behavior for prod
2016:12:08 19:32:11 bbloom i want this. somebody make this.
2016:12:08 19:32:12 bbloom 🙂
2016:12:08 19:32:36 gfredericks It can be a first-class feature of twojure
2016:12:08 19:32:36 bbloom alex probably wouldn’t tell us if rich was working on a new tool here 😉
2016:12:08 19:32:57 bbloom Twojure has a nice ring to it
2016:12:08 19:34:16 dspiteself It seemed their primary goal was running the minimal set of tests and generative tests for a given change.
2016:12:08 21:51:34 nwjsmith I would expect that call to gen/sample to give me :thing/entitys with int values for the :thing/id key.
2016:12:08 21:52:13 nwjsmith Instead, it’s as if I hadn’t supplied the override at all. I get a mix of int and uuids.
2016:12:08 21:56:26 hiredman ::id makes it work
2016:12:08 21:57:16 hiredman (dunno what the expected behavior is, if that is a bug, etc)
2016:12:08 21:57:29 hiredman ::id in the override map
2016:12:08 21:57:54 nwjsmith Yeah, it does. This is a minimized example, but I’d like to keep from overriding all ::ids, and just override thing/entity‘s
2016:12:08 22:07:25 nwjsmith @hiredman thanks, I think your suggestion help me identify my problem. I’m pretty sure it’s a bug
2016:12:09 12:12:30 yenda Hi, could someone help me understand why in the above code generating ::engine-variable takes a split of a second and test/check runs forever until hitting a GC overhead ?
2016:12:09 12:16:44 morrifeldman I'd like a constraint that only allows one branch of an or in a derived spec. Is this possible? Here is an example of what I'm trying to do.
2016:12:09 14:06:45 mpenet am I awful thinking about writing an (with-ns ...) macro that does create-ns + in-ns + restore proper ns once out of scope ?
2016:12:09 14:07:46 mpenet getting tired of writing super verbose specs preceeded by (alias baz (create-ns 'foo.bar.baz))
2016:12:09 14:15:44 mpenet the syntax is quite appealing, it would really save a ton of boilerplate
2016:12:09 14:15:51 bronsa you could get e.g. (do (in-ns 'foo) ::foo) to work with reader hackery but no way to get (with-ns 'foo ::foo) to work w/o introducing placeholder objects & resolve aliases at compile time
2016:12:09 14:16:21 mpenet I see
2016:12:09 14:16:38 bronsa realistically, neither is doable
2016:12:09 14:17:04 mpenet Well I might give it a try once I am done with the giant schema -> spec port I am into atm. Sounds fun nonetheless
2016:12:09 16:01:12 lvh @alexmiller Are you a good person to report suggested stdlib macro specs to? If so, I submit http://dev.clojure.org/jira/browse/CLJ-207 for consideration — I just got bitten by that; was expecting :let in the first position to work (kinda feel it should anyway, but OK — if it doesn’t the error message could be a lot more descriptive, and that seems like the kind of thing spec can help with here, plus you’re probably already speccing what for looks like)
2016:12:09 16:01:50 alexmiller jira is a good person to report suggestions to :)
2016:12:09 16:02:06 lvh Does that go on the ticket?
2016:12:09 16:02:10 lvh Or do I re-tag it spec?
2016:12:09 16:02:14 lvh or is that a new ticket
2016:12:09 16:02:16 alexmiller I’d file a new enhancement ticket
2016:12:09 16:02:20 alexmiller and point to this one
2016:12:09 16:02:25 lvh cool, thanks
2016:12:09 16:02:29 alexmiller requesting a spec for for
2016:12:09 16:08:38 yenda I am trying to validate data after edn/parse-string, the field above doesn't validate with the predicate #{true false} even though (s/valid? ::boolean (java.lang.Boolean. "true")) does
:edit actually it does not for "false", only for "true"
2016:12:09 16:12:27 minimal use boolean?
2016:12:09 16:13:12 yenda poor me I'm still on clojure 1.8 😞
2016:12:09 16:15:27 yenda but yeah I can just add the boolean? predicate source to my spec
2016:12:09 16:33:22 yenda nice thanks ! didn't think it was also handling this
2016:12:09 16:50:02 yenda that is so wierd I add a test/check working before and now it throws an exception cider.nrepl.middleware.test$report cannot be cast to clojure.lang.MultiFn. I tried :monkeypatch-clojure-test false in lein profile but it doesn't change anything
2016:12:09 17:14:22 yenda I am trying to figure out what is causing this, it happens both in clojure 1.8 and 1.9
2016:12:09 17:17:33 yenda if I try to run the test right after jacking-in I get the error, if I recompile only the test one or more time and run it it works, if I recompile the whole namespace I get the error even if I make it work once before
2016:12:09 17:30:41 yenda Ok I found the problem I had to explicitely require [clojure.test.check] in my test namespace
2016:12:09 18:36:06 alexmiller @yenda note that in Clojure we use only the canonical Boolean/TRUE and Boolean/FALSE for true/false values so you should never call the Boolean constructor. This is in particular a problem with false as the non-canonical false value will be treated as true (although you’ll see subtle weirdnesses with the non-canonical true value too).
2016:12:09 22:34:31 lvh Is there a way to introspect the generated samples for fspecs? I’m trying to see what sort of stuff it comes up with now to see if it’s worth coming up with a more specific (and ideally: faster to generate) spec
2016:12:09 22:35:13 lvh specifically right now I have a HOF that takes forever to test (for good reason)
2016:12:09 22:36:34 lvh So I’m guessing coming up with lots of ::variants, and then checking if a bunch of combinations of them is still a variant… well, that’s going to be expensive 🙂 I’m not sure if I should just run the generative tests extremely sparingly, or work on producing faster ones (e.g. by helping the generator generate realistic variants faster — I have a bunch laying around anyway...)
2016:12:09 22:41:23 alexmiller you might try changing the :ret spec to (s/coll-of string? :gen-max 3) to see if that helps
The :parent/stored-entity spec is incomplete. I can’t specify that its children are required to be :child/stored-entitys.
2016:12:09 23:01:03 nwjsmith If I use a different key for a :parent/stored-entities children, it means that functions written to work with :parent/entitys won’t work with the :parent/stored-entity, which is a Bad Thing™.
2016:12:09 23:03:34 alexmiller well in this case, you’d probably need stored-entity to be built from scratch rather than by restricting entity
2016:12:09 23:05:52 alexmiller sometimes I feel like the right answer to these kinds of questions is to re-think the modeling approach though
2016:12:09 23:07:19 nwjsmith I’m open to remodelling 🙂. I’m not sure how I could re-write :parent/stored-entity to solve this.
2016:12:09 23:07:43 alexmiller I guess I would ask whether you really need to differentiate stored/unstored
2016:12:09 23:11:45 nwjsmith I think I do. There are some functions that would need to ensure the entity is already on disk. An update-parent-in-place function would need it for example.
2016:12:09 23:12:26 nwjsmith (that example probably counts as vulgarity on Clojurians)
2016:12:09 23:14:47 bbloom spec lacks parameterized abstractions / generics / whatever - trying to encode that invariant is unlikely to work out - i recall some discussion of alternatives at some point in this room
2016:12:09 23:14:52 bbloom one suggestion was just use two different keys
2016:12:09 23:15:27 bbloom for example ::parent/stored-children and ::parent/children
2016:12:09 23:16:00 bbloom what extra data do stored children have?
2016:12:09 23:16:16 nwjsmith same thing, just a :child/id
2016:12:09 23:16:18 bbloom could you have ::parent/children and ::parent/stored where the stored key contains a map of children ids to a stored boolean?
2016:12:09 23:16:26 bbloom or timestamp?
2016:12:09 23:16:47 bbloom see if you can come up with a representation that doesn’t change over time
2016:12:09 23:16:58 bbloom where stored state ~= time
2016:12:09 23:17:00 nwjsmith I’ve considered using something like Om/Datomic’s tempids
2016:12:09 23:17:20 bbloom yeah, or symbols
2016:12:09 23:17:46 bbloom you can always unfold a graph with symbols, ids, whatever
2016:12:09 23:18:39 nwjsmith That way a parent always has an id, but it a stored one has a ‘permanent’ id.
2016:12:09 23:18:53 nwjsmith K, I’m going to explore this a bit more and see what comes of it
2016:12:09 23:19:18 bbloom yeah, try to model it openly where you can accrete data by adding more keys w/o “changing” the thing
2016:12:09 23:19:22 bbloom this way some key always means the same thing
2016:12:09 23:19:33 bbloom it will be easier to reason about and you’ll gain new features you didn’t even expect to have
2016:12:09 23:21:11 bbloom for what it’s worth, i think “parameterized specs” may be useful, but i’m tempted to say we’re better off without them for as long as we can get away with it - if not forever
2016:12:09 23:48:18 alexmiller Alternately, consider a separate predicate that checks whether all things are storeable
2016:12:09 23:48:35 alexmiller That can be combined with the information
2016:12:10 00:39:09 gfredericks @bbloom I can't really spec test.check generators meaningfully without parameterized specs
2016:12:10 00:39:51 bbloom @gfredericks 1) i assume that there’s existing code out there for which that’s true 2) i’d love to hear why and 3) i’m curious if you’d do things differently now to avoid that
2016:12:10 00:40:35 gfredericks @bbloom what's a good generator for the function (defn vector "Returns a generator of vectors of items from g" [g] ...)?
2016:12:10 00:41:15 bbloom i haven’t a clue
2016:12:10 00:41:20 gfredericks right now I just have the args as (s/cat :g generator?) and the ret as generator?, which doesn't remotely capture the relationship between the two
2016:12:10 00:41:26 gfredericks or even the fact that the return value generates a vector
2016:12:10 00:41:48 gfredericks so there's two aspects to this
2016:12:10 00:41:50 bbloom well at least now we can join the Go community in complaining that our “type system” doesn’t have generics 🙂
2016:12:10 00:42:10 bbloom that’s interesting
2016:12:10 00:42:36 gfredericks I can't imagine what would be done differently to avoid this
2016:12:10 00:42:40 bbloom i’d be really curious to hear rich’s thoughts on this
2016:12:10 00:42:52 bbloom b/c i can’t imagine he hadn’t considered it
2016:12:10 00:43:10 gfredericks any comment I've seen about higher-order-ish things is usually along the lines of "specs are for data"
2016:12:10 00:43:53 bbloom sure, the problem comes in once you introduce contextual specs
2016:12:10 00:44:03 gfredericks a corollary of which I have to imagine is "specs aren't useful for things like generators", but I couldn't defend that and haven't heard it explicitly taken that farb
2016:12:10 00:44:04 bbloom which happens instantly once you have functions
2016:12:10 00:44:25 gfredericks yeah you can't spec map completely for the same reason
2016:12:10 00:44:42 gfredericks so I guess you're right that they must have thought about it :)
2016:12:10 00:44:47 bbloom yeah, i think it’s perfectly OK to only spec data “at rest” so to speak
2016:12:10 00:45:13 bbloom in theory it could be extended to spec more things, but you’re not going to serialize a generator to edn
2016:12:10 00:45:40 gfredericks you ultimately can't, because of bind
2016:12:10 00:45:44 bbloom at least not from opaque closures
2016:12:10 00:45:53 bbloom yeah
2016:12:10 00:45:58 gfredericks even fmap I guess
2016:12:10 00:46:36 bbloom i guess “spec doesn’t cover that use case” is fine - i’m just curious if that’s now & more stuff could be spec’d later with more powerful primitives, or if it’s like “eh, this approach only goes so far & you can spec some other way later if you need to"
2016:12:10 00:47:00 bbloom like in theory you could reuse specs in another context, which i guess is what Typed Clojure will do: generate types from specs
2016:12:10 00:47:06 bbloom or otherwise treat specs as types
2016:12:10 00:47:39 bbloom for me, i’ve basically only bothered to spec things that i intend to pprint
2016:12:10 00:47:41 bbloom 🙂
2016:12:10 00:48:07 bbloom basically: “hey spec, save me the trouble of looking at this thing and checking to make sure it looks right"
2016:12:10 00:49:58 gfredericks so a parameterized spec is a function that takes a spec and returns a spec
2016:12:10 00:50:09 gfredericks presumably you could make a spec for that function
2016:12:10 00:50:15 gfredericks and test it by generating specs?
2016:12:10 00:50:19 gfredericks this is really hazy
2016:12:10 00:50:45 gfredericks haskell uses => to talk about this stuff doesn't it?
2016:12:10 00:50:51 bbloom soooorta
2016:12:10 00:51:02 bbloom => is used to constrain this stuff
2016:12:10 00:51:30 bbloom the => basically means to unify things on the left with the environment
2016:12:10 00:51:43 gfredericks oh I guess the parameterization is more basic
2016:12:10 00:52:05 gfredericks is already parameterized
2016:12:10 00:52:18 bbloom so like if you have some type a => b -> c that says you have some function of type b, returning c, and there exists some a… and you can constrain it like: Num a -> b c there exists some a such that a is tagged with the class Num
2016:12:10 00:52:40 bbloom yeah, you don’t need to talk about constraints to get to parameters
2016:12:10 00:52:45 gfredericks does that mean anything interesting if a doesn't appear on the right side of the =>?
2016:12:10 00:52:58 bbloom you could have listof Things :: Num a => [a] if my syntax is correct
2016:12:10 00:53:25 bbloom i believe that the it’s required to be on the right side at least in haskell 87 w/o extensions
2016:12:10 00:53:40 bbloom i used unique letters for the sake of clarity talking about parts of it
2016:12:10 00:53:45 bbloom also, i suck at haskell, so don’t ask me 🙂
2016:12:10 00:54:23 gfredericks hey @bbloom I think I got my lens stuck in a monad transformer can you help.
2016:12:10 00:54:36 bbloom you’re going to have to amputate.
2016:12:10 00:55:18 bbloom but yeah, spec obviously has specs parameterized by specs, like every
2016:12:10 00:55:31 gfredericks are those all special?
2016:12:10 00:55:34 bbloom or cat, or alt
2016:12:10 00:55:47 gfredericks or fspec
2016:12:10 00:55:49 bbloom yeah
2016:12:10 00:56:03 bbloom the issue is that named specs can’t have parameters
2016:12:10 00:56:18 gfredericks I've wondered if (generator integer?) could be a thing
2016:12:10 00:56:20 bbloom there’s no s/fn
2016:12:10 00:56:40 gfredericks presumably it could be implemented with the lower-level machinery
2016:12:10 00:56:56 gfredericks even then you couldn't fully describe gen/vector
2016:12:10 00:57:02 bbloom like in theory you could have (s/fn [val] (s/every-kv integer? val))
2016:12:10 00:57:19 gfredericks this is the "function from specs to specs" idea
2016:12:10 00:57:29 bbloom yeah
2016:12:10 00:57:42 bbloom however, this gets in to an area where i feel the current tradition of type systems is real bad
2016:12:10 00:58:15 gfredericks uh oh
2016:12:10 00:58:22 bbloom positional parameters force you to provide all the values when you call it
2016:12:10 00:58:47 bbloom types are curried in haskell (i think), but still, you don’t actually get a type until you give it all the args
2016:12:10 00:59:11 gfredericks where does that get awkward?
2016:12:10 00:59:27 bbloom b/c once you add a parameterized type, it’s like a virus
2016:12:10 00:59:40 bbloom everything above it needs to be parameterized until you know a concrete type
2016:12:10 00:59:45 bbloom same problem with monads 😉
2016:12:10 01:00:01 bbloom or checked exceptions
2016:12:10 01:00:11 bbloom "oh shit, this function has an effect. ooooh well, let’s change the signature of every function everywhere"
2016:12:10 01:01:13 bbloom if i were designing a type system with generics, i’d make all generic parameters named and optional
2016:12:10 01:01:56 bbloom so you’d have (List {}) and (List {:element String}) - in the former case, the default for :element would be Any
2016:12:10 01:01:56 gfredericks so "the type system knows about effects" seems like a positive thing, it's just the "and I have to type it in all sorts of unrelated places" that's the bad part, right?
2016:12:10 01:02:03 gfredericks would a more advanced inference system fix that?
2016:12:10 01:02:05 bbloom or whatever List declared it to be
2016:12:10 01:02:18 bbloom yeah, that’s the bad part
2016:12:10 01:02:38 bbloom more advanced inference would help, but the badness comes from the syntax
2016:12:10 01:02:49 bbloom haskell avoids this problem a little bit by using currying
2016:12:10 01:03:19 bbloom if you have some function that has an effect, you just leave the monad as the last parameter & then if you change the monad, that’s fine b/c you’re using curring / point-free to avoid mentioning the monad type as much as possible
2016:12:10 01:05:41 gfredericks the monad is the last constraint mentioned before the =>? or it's actually a physical argument?
2016:12:10 01:06:01 bbloom the order ofthings left of the => don’t matter, as far as i know
2016:12:10 01:06:12 bbloom the syntax is super confusing
2016:12:10 01:06:51 bbloom the monad would be an actual argument
2016:12:10 01:07:09 bbloom but you can leave it out of the signature or just say “hey, i return an a"
2016:12:10 01:07:15 gfredericks is that a general thing you deal with with monad transformers?
2016:12:10 01:07:45 bbloom monad transformers are a whole ‘nother bag of goofy ideas 🙂
2016:12:10 01:08:15 bbloom a lot of these problems stem from the fact that they don’t have associative or open structures in their type systems
2016:12:10 01:09:20 bbloom if you want to combine more than 1 monad without manually creating a custom monad, you can layer them and the order in which you layer them matters b/c the type of a monad stack is essentially a cons list of monad types
2016:12:10 01:09:45 bbloom the "extensible effects" stuff comes up with a clever way to encode a set without order in to the type
2016:12:10 01:10:19 bbloom GHC now has some kind of union types with some extension, but at some people they were like sorting types to represent a multi-set or some craziness to achieve “order independence”
2016:12:10 01:10:33 bbloom but that still failed to address openness properly
2016:12:10 01:11:23 gfredericks okay so static types aren't all good yet.
2016:12:10 01:11:37 bbloom lol nope
2016:12:10 01:12:08 bbloom everybody in the ML/Haskell camp AND the Lisp camps were afraid to take a step back and say “hey, what happens if we have maps and sets” 🙂
2016:12:10 14:32:33 cemerick what is the current best idea for how to spec a map with (un-namespaced) symbol keys? The best I've figured is to use every to spec map entries, not very satisfying.
2016:12:10 16:14:10 gfredericks @cemerick I don't know of anything besides an opaque function
2016:12:10 16:14:26 gfredericks I haven't investigated how to use the lower-level facilities to build something more first-class
2016:12:10 16:23:06 cemerick @gfredericks having something equivalent to s/keys (s/syms? :-P) doesn't look hard, but way more work than manually :syms destructuring and conforming each part
2016:12:10 16:31:39 gfredericks I'm more likely to need strings, personally
2016:12:10 16:38:04 cemerick and bob needs numbers, and poor jane is dealing with maps that have namespaced and un-namespaced keywords with the same name carrying different stuff
2016:12:10 16:39:00 gfredericks make a new system and call it spec2
2016:12:10 16:39:14 cemerick stahp
2016:12:10 16:43:05 cemerick your convo with @bbloom above is serendipitous. I was really trying to get spec to do way more than it's meant to last night, notwithstanding my cat problems.
2016:12:10 16:44:16 gfredericks someone should build some sort of adapter to plumatic/schema for fallback needs
2016:12:10 16:44:56 gfredericks teach spec that schemas are specs and teach schema that namespaced keywords are schemas
2016:12:10 16:45:54 cemerick well, part of my frustration is in wanting to get useful generators out of specs, so, things like schema aren't super useful
2016:12:10 16:46:07 gfredericks schema has generator support too
2016:12:10 16:46:14 gfredericks it's just more wiring to do
2016:12:10 16:47:13 cemerick well
2016:12:10 16:47:59 cemerick insofar as I need to do extra work to get corresponding generators (esp. if schema/spec is a separate thing from the generator), it's not super useful
2016:12:10 16:48:37 cemerick I mean, beyond doing the work to have a validating/destructuring spec and corresponding generator
2016:12:10 16:49:17 gfredericks I'm not sure what extra work you're thinking of; I think the overhead to get to generators is comparable in the two systems
2016:12:10 16:50:06 cemerick I'm not sure either, I guess. It's been a while since I touched schema.
2016:12:10 16:50:52 cemerick my recollection was that I ended up bailing and hand-writing my own generators at a certain level of complexity
2016:12:10 16:52:02 gfredericks the big unfortunality is that (every version I've used) has bad errors when there's a missing generator you need to fill in
2016:12:10 16:52:19 gfredericks it just says "Missing leaf generator" or something of that sort and doesn't attempt to tell you which schema caused the problem
2016:12:10 16:52:44 gfredericks but once you build up your stockholm syndrome around that issue then it goes pretty well
2016:12:10 16:53:29 cemerick this callous is my own, there is no other like it
2016:12:10 17:47:33 bbloom @cemerick i too suffer from instantly running in to edge cases and limits when attempting to use new things 🙂
2016:12:10 17:53:10 bbloom gfredricks as well - we’re just those guys who instantly find the dark corners
2016:12:10 18:27:04 cemerick I don't know that I find dark corners any more frequently or quickly than others. Probably just louder than most about it, maybe
2016:12:10 18:30:38 cemerick @bbloom the filesystem expression problem tweet was deeply triggering
2016:12:10 18:30:47 bbloom ¯\(ツ)/¯
2016:12:10 18:30:51 cemerick mmm, guess that's OT here
2016:12:10 19:20:28 alexmiller @cemerick Re your question above, did you look at s/keys with :req-un ?
2016:12:10 19:21:40 alexmiller @gfredricks for string keys, you can use a leading conformer that converts to keywords as a workaround
2016:12:10 19:23:32 alexmiller Assuming you have an information map with keys that are strings ala json
2016:12:10 20:19:01 gfredericks @alexmiller was the s/keys with :req-un the same as your other two comments, or about something different?
2016:12:10 20:19:09 gfredericks I didn't know about the leading conformer idea, so that's useful
2016:12:10 20:57:57 alexmiller It was about the original question of maps with unqualified keys
2016:12:10 21:06:06 alexmiller you can do lots of tricks with leading conformers, like for example calling select-keys to narrow the key set you’re validating
2016:12:10 21:06:17 alexmiller yada yada beware of conformers yada yada
2016:12:10 21:21:11 gfredericks @alexmiller unqualified keys are pretty basic, right? chas had been asking about symbols as keys, which I assume is the same approach as strings
2016:12:10 21:30:47 alexmiller oh, right I heard unqualified keys in my head
2016:12:10 21:31:08 alexmiller but you could use a leading conformer for symbol->keys too
2016:12:10 21:32:20 gfredericks yeah
2016:12:10 21:51:17 bbloom may i ask: why are you using strings or symbols heterogeneously?
2016:12:10 21:51:36 bbloom besides maybe strings ~= keys when doing interop w/ java maps or javascript objects
2016:12:10 22:05:27 gfredericks I like using strings when talking about javascript in clojure
2016:12:10 22:06:09 gfredericks my rule of thumb is not using keywords that don't appear in the source code anywhere
2016:12:10 22:06:46 gfredericks in the past I've used schemas to describe both the internal clojure-fluent version of data but also the external JSON-flavored version
2016:12:10 22:06:52 gfredericks and I found that terribly useful
2016:12:10 22:07:42 gfredericks spec does not fit that pattern nearly as well as p**matic/schema
2016:12:10 22:10:20 seancorfield I’m curious why you don’t use keywordize at the boundary? I find that approach far more natural.
2016:12:10 22:11:43 gfredericks I essentially do, but I prefer it in a less blind way
2016:12:10 22:13:18 gfredericks to be clear, these are mostly just thoughts. I've only tried this in one codebase, but couldn't figure out how to get a nice API for it. most of my thoughts are here: https://github.com/gfredericks/schema-bijections#rationale
2016:12:10 22:13:56 gfredericks so what I want is something like ↑ that library, but less confusing/verbose, and maybe involving spec if that ends up making sense
2016:12:10 22:16:58 bbloom yeah, i definitely support using strings for json rather than keywordize everywhere
2016:12:10 22:17:39 bbloom does raise some questions about how spec should work for json tho
2016:12:10 22:17:48 gfredericks I suppose that readme doesn't talk about trying to separate bijections and surjections, which I never figured out how to do
2016:12:10 22:18:46 gfredericks I wish bijections and surjections were first-class concepts
2016:12:10 22:19:17 gfredericks they compose well; surjections are interesting for generators
2016:12:10 22:20:08 gfredericks if I can ever work this into a better library it probably ought to be named jections
2016:12:10 23:26:14 pedroiago is it intentional that s/? unforms a s/cat into a list?
'e.g. (s/unform (s/? (s/cat :a #{1})) {:a 1}) ;=> [(1)]'
2016:12:11 00:45:38 alexmiller That shouldn't be nested like that - there's one bug filed about that and a few other cases we are looking at
2016:12:11 02:43:00 cemerick @alexmiller the challenge is unqualified symbols for keys
2016:12:11 02:44:35 cemerick ah, saw @gfredericks already corrected you on that
2016:12:11 02:47:15 cemerick @alexmiller yeah, I'm familiar with conformers; what I was wanting to do is retain the automagic generator generation, so conformers aren't that. Only the top level is currently symbol-keyed (everything else is sequences of various sorts), so I can easily destructure out for conforming, and manually assoc in the results of generators when doing that.
2016:12:11 02:51:42 cemerick @bbloom I've been angry about JS[ON] not having symbol literals every single day for ~6 weeks.
2016:12:11 02:55:01 cemerick @gfredericks looking forward to jections. I actually want two different destructurings coming out of conform, depending on context: JSON -> basically symbolic representation that makes sense for the particular structure I've spec'd, or (2) a completely desugared representation that does things like tag the matching spec on s/or's, flows through the custom destructurings I've put in place via conformers, etc.
2016:12:11 02:55:48 cemerick wouldn't be surprised if this requires the aforementioned higher-order / spec generics though
2016:12:11 05:17:13 gowder I'm just gonna be proud of myself that I wrote my first actual specs today, while y'all are talking the fancy stuff :-)
2016:12:11 05:47:32 cemerick @gowder I'm proud of you, too 🙂
2016:12:11 17:39:37 thomas Hi, I am trying to spec my function... and I have spec'ed the :args and the :ret... but now trying to spec the :fn...
2016:12:11 17:40:05 thomas and I have added the example spec (which I know is wrong... but now I don't get any errors ...
2016:12:11 18:10:05 gfredericks :ret and :fn only get checked by doing clojure.spec.test/check or something of that sort
2016:12:11 20:55:33 gowder Or instrumenting
2016:12:11 22:09:16 alexmiller No, not instrumenting
2016:12:12 01:10:30 gowder Oh whoops. Instrument just for args, duh. Lazy speed reading of slack posts is never a good idea :-(
2016:12:12 08:15:04 thomas cheers... I'll investigate it some more.
2016:12:12 10:16:45 alexisvincent Any patterns to avoid things like this
2016:12:12 11:35:50 alexisvincent is the general consensus to prefer namespaced keys in maps?
2016:12:12 12:50:47 dm3 it seems so, for best synergy with spec
2016:12:12 13:07:37 alexisvincent Are people generally using :: as a helper
2016:12:12 13:08:30 alexisvincent Its a pain when wanting to use the spec in another ns… You have to specify the full mylonglongname.namespace.thing/name
2016:12:12 13:09:11 alexisvincent becomes so long winded…
2016:12:12 13:09:35 alexisvincent Or are people just registering thing/name
2016:12:12 13:14:30 alexisvincent OH 🙂:: can also be used for aliasing
2016:12:12 13:15:45 dm3 I guess that's what you meant
2016:12:12 13:54:01 alqvist What is the best way of namespacing an existing map? Using a function, not the reader
2016:12:12 14:26:11 alexisvincent @dm3 yep 🙂
2016:12:12 15:15:52 alexmiller @alqvist what do you mean? adding a namespace to the keys of a map?
2016:12:12 15:16:14 alexmiller there is no function to do that right now
2016:12:12 15:34:36 alqvist @alexmiller exactly that - I made a small function for it. Posted it in #clojure
2016:12:12 15:34:50 alexmiller cool
2016:12:12 17:34:25 sparkofreason Is there some preferred way of writing simulation-testable specs for functions with constraints between parameters? I have a function that takes a map and a key. The key must exist in the map for both the input and output. I can make it work by writing a custom generator for :args using bind, was wondering if there was a preferred method.
2016:12:12 18:28:47 alexmiller that is the preferred method
2016:12:12 18:44:03 sparkofreason Thanks, that's basically how I handled it.
2016:12:12 18:46:13 sparkofreason I notice in the video that the generator override was specified directly in the test, rather than in the spec. Don't know that it makes much difference, though giving it in the spec would allow you to reuse the override if that spec was composed with others.
2016:12:12 18:48:18 bbloom is there a spec or predicate for objects that have strict value equality? looking for something like a clojure.core/edn? or something like that
2016:12:12 18:49:03 bbloom edn? isn’t quite right, since that’s about serializability, but would be close enough for my needs
2016:12:12 18:52:33 hiredman that would be nice
2016:12:12 18:52:56 hiredman any? is so close, but fails with NaNs
2016:12:12 18:53:18 bbloom no, any? isn’t close - it includes functions
2016:12:12 18:53:57 bbloom i have a procedure that will go in to an infinite loop if the function isn’t pure & i’ve been bitten a few times by including a function in it
2016:12:12 18:54:07 bbloom so i’m doing like add-watch! does, which is force key values
2016:12:12 18:54:32 hiredman I am pretty sure the generator for any? doesn't generate functions
2016:12:12 18:54:52 bbloom generation aside, the spec is wrong if i allow functions 🙂
2016:12:12 18:55:14 bfabry the generator for any? was way simpler than preferred last time I checked, there was a bug on test.check about it
2016:12:12 18:55:54 bbloom i need a predicate like value-equality?
2016:12:12 18:56:10 bbloom which i guess i could make, but it seemed like it might have been something that existed or should exist
2016:12:12 18:56:21 bbloom i didn’t feel like exhaustively listing what’s in edn in a spec
2016:12:12 18:56:30 alexmiller I think there are some things in test-check with "printable" in their name?
2016:12:12 18:57:02 alexmiller I know we spent a lot of time building up stuff like that in the old data.generative
2016:12:12 18:57:09 alexmiller Certainly open to ideas
2016:12:12 18:57:23 bbloom hmm “printable” is a good hint, i’ll look - thanks
2016:12:12 18:58:33 hiredman the problem with the printable stuff is it generates NaNs
2016:12:12 18:59:43 hiredman and those are generators, not predicates
2016:12:12 18:59:51 bbloom alexmiller: this looks more or less like what i need. would be cool to have a predicate for it
2016:12:12 19:00:09 alexmiller Jira it up
2016:12:12 19:00:58 alexmiller Seems like we could also fix test.check re NaN (which is not currently printable)
2016:12:12 19:05:39 bbloom i think clojure.core/edn? would be the least objectionable formulation of this request
2016:12:12 19:06:36 bbloom well, actually - may not
2016:12:12 19:06:41 bbloom b/c that would be a “deep” predicate
2016:12:12 19:06:59 bbloom i don’t think any such deep predicates exist - it would have O(N) runtime 😕
2016:12:12 19:39:57 alexmiller I think it's probably unlikely we would make that predicate
2016:12:12 19:40:26 bbloom yeah - after realizing it was O(N), i realized that - but a spec makes more sense, right?
2016:12:12 19:41:20 alexmiller Well, definitely a generator, maybe a spec
2016:12:12 19:41:58 bbloom besides the stuff in clojure.core.specs, are there any “standard” specs anywhere? i don’t think so, right? just predicates...
2016:12:12 21:06:43 jeremyraines In this example from the spec guide, is the ability to key into the arguments with :start and :end created by the s/cat above the anon function? Does the destructuring happen such that the first expression given to s/and has an effect on what’s passed to the other?
2016:12:12 21:07:16 bfabry @jeremyraines yes, the s/cat conforms the data. s/and threads the conformed data
2016:12:12 21:07:54 jeremyraines ok, thanks
2016:12:12 21:07:55 zane Super useful for conformers that coerce between types.
2016:12:13 04:39:50 j-po I'm trying to spec something like s-expressions, and my current flailing in that direction involves a recursive s/tuple (something like
), but I'd like for the input not to have to be a vector of vectors. Am I better off just turning the input into such a vector as part of the checking process, or is there another way?
2016:12:13 05:29:45 alexmiller s/cat?
2016:12:13 05:32:49 alexmiller then you can just replace ::sexp (s/* ::sexp) with (s/+ ::sexp) too
2016:12:13 09:22:11 j-po @alexmiller Thanks! When I gen/generate from an s/cat approach, though, I don't get a sequence of sequences, though, but just one flat one.
2016:12:13 09:25:27 vandr0iy I was asking stuff in the beginners channel, but probably this is a better place for doing that.
I posted this snippet of code, claiming that it doesn't work:
And I was told that :req-un wants a literal vector of namespaced keywords, not something that evaluates to one, because of it being a macro (obviously) - which makes sense.
So I went a step further, and macro'd my stuff like this:
and it still doesn't work OOB; while, if I copy-paste the result of macroexpand '(arst :a) in the REPL - it works flawlessly. What's going on here? I'm afraid that there's some rookie mistake here somewhere...
2016:12:13 09:28:03 mpenet you need to use either a macro or eval:
2016:12:13 09:30:07 mpenet and you might actually need to call seq or vec on the result of the get-in, not sure s/keys will happily take a set as :req-un value
2016:12:13 09:33:56 mpenet your macro should work (kinda, you need namespaced keys in your bunch-of-stuff and there's a paren missing there too)
2016:12:13 09:40:35 vandr0iy I'm sorry for adding the macro after the request, I accidentally pressed enter before completing my post.
My keys aren't namespaced - I "namespace" them with the (map #(->% str keyword) sp#) thing.
What puzzles me is: why does this macro work if I try to do (arst :z) - which takes a function from the bunch-of-stuff and spits a spec out of it -
and doesn't if I get a vector of un-namespaced keywords, "namespace" them with the dirty hack described above???
I (macroexpand '(arst :a)), and copy-paste what I get in REPL - and it works, but it doesn't if I just try to do (arst :a).
I'm pretty sure that ~(vec (map #(->> % str keyword) sp#)) yields a vector of namespaced keywords...
2016:12:13 09:43:13 mpenet it doesnt I think, it creates keys with ":foo" content with a leading :, so yeah ::foo but not namespaced per say
2016:12:13 09:43:47 mpenet try running your spec, it will complain about this. Try changing your keys in bunch-of-stuff with ns keys you ll see
2016:12:13 09:52:31 vandr0iy Oh... I see. Was able to make it work using this other horrible hack:
2016:12:13 10:02:16 triss thanks @vandr0iy . I’ll give that a try
2016:12:13 10:43:26 triss that worked. Brilliant thanks.
2016:12:13 10:44:02 triss If I want a generator that produces a wider range of vaues than the ones provided for double-in and int-in how would I go about getting one of those?
2016:12:13 10:44:34 triss all the random numbers I’m getting are pretty small and I need to test against some big ones.
2016:12:13 10:50:02 mpenet test.check has generators that allow you to specify a range
2016:12:13 10:50:14 mpenet large-integer* and double* I think
2016:12:13 10:51:33 triss cheers @mpenet I’ll have alook
2016:12:13 10:53:05 triss (gen/sample (gen/double* {:min 1 :max 200})) still produces really small values
2016:12:13 10:53:23 triss I’d like to see some values closer to 200. Is this possible?
2016:12:13 11:00:40 mpenet Try sampling more values
2016:12:13 11:13:19 triss ah ok.. looks generators probably aren’t going to be efficient enough for the way I was abusing them.
2016:12:13 11:50:46 triss If I wanted a more evenly distributed generator would it be possible to write one?
2016:12:13 12:13:58 luxbock @triss yes but you need to use the generators in test.check
2016:12:13 12:14:08 gfredericks @triss what's a uniform distribution over doubles?
2016:12:13 12:14:57 gfredericks @triss oh I think this is a sizing issue, sorry I didn't read back far enough
2016:12:13 12:15:09 triss @gfredericks ah I can see why that might be awkward. rand seems more even than this though
2016:12:13 12:15:16 gfredericks @triss you should get lots of values close to 200, but gen/sample is deliberately only showing you small examples
2016:12:13 12:15:45 luxbock is there any reason for why s/spec couldn't accept a third optional argument for defining a custom generator? I thin it would make it easier to statically analyze specs and might end up with some cool tooling use cases
2016:12:13 12:16:56 gfredericks @triss test.check has some subtleties with sizing that need to be documented better, and I have 45 minutes free right now so I think I will start on that
2016:12:13 12:17:17 triss @gfredericks I still see a similar distribution. biased to low numbers
2016:12:13 12:17:31 triss ah many thanks @gfredericks will be much appreciated
2016:12:13 12:17:40 gfredericks @triss it's not going to be uniform, but you should definitely get some larger numbers
2016:12:13 12:18:15 gfredericks @triss one reason I asked my original question about what a uniform distribution is, is because doubles themselves aren't uniform -- there are a lot more numbers between 1 and 50 than 50 and 200
2016:12:13 12:18:43 gfredericks so uniform might mean different things in different contexts
2016:12:13 12:18:45 triss oh that’s interesting. because of the way they are represented in the machine?
2016:12:13 12:18:55 gfredericks you could say that
2016:12:13 12:19:10 gfredericks it's kind of inherent in the idea of floating point
2016:12:13 12:19:11 triss I guess I’d like a distribution of Real numbers? does that make more sense?
2016:12:13 12:19:30 triss ^a more even distribution of real numbers
2016:12:13 12:19:37 gfredericks not really, but I think I know what you're trying to get at
2016:12:13 12:19:59 gfredericks @triss you could use large-integer to get this pretty easily
2016:12:13 12:20:04 triss I’d love to see a flatter histogram!
2016:12:13 12:20:05 gfredericks or at least something close
2016:12:13 12:20:11 gfredericks yes that's a good way of putting it :)
2016:12:13 12:20:41 triss ah brilliant will look at large-integer
2016:12:13 12:44:56 gfredericks @triss w.r.t. choose vs large-integer, the short explanation is that test.check has a strategy where it tries small things first and slowly tries larger and larger things; by using choose you're overriding that strategy, and that strategy is the reason you saw an uneven distribution with large-integer
2016:12:13 12:45:43 gfredericks which is best depends on your goals, but I'd say large-integer would be a good default
2016:12:13 12:46:53 triss would you consider using generators outside of testing code foolish?
2016:12:13 12:47:25 triss it seems very much tailored for testing.... (as the namespace indicates I suppose)
2016:12:13 12:49:08 gfredericks @triss clojure.spec uses them for a bit more than test.check does, but still for testing purposes; if you're using them for not-testing-at-all, it will be a little weird but not terrible; understanding the sizing subtleties could be more important though
2016:12:13 12:50:19 gfredericks I suppose based on that jungle music talk at the conj I should expect people will be using generators for all sorts of things
2016:12:13 12:53:02 gfredericks @triss the gen/generate function will probably be useful for you, since it accepts a size parameter
2016:12:13 12:53:43 gfredericks e.g., (repeatedly 200 #(gen/generate (gen/let ...) 200)) should give you the distribution you wanted even using large-integer
2016:12:13 22:53:13 ag Guys, can someone help me with this: I have a long string of lorem ipsums, something like this "pellentesq dapib suscip liguldon posue augquaeti v tort ..." and now I need a spec with a generator that would generate a name of 3 to 5 words long by randomly pulling those words from that lorem-ipsum string. How do I do that?
2016:12:13 23:21:43 adambros
(def s “pellentesq dapib suscip liguldon posue augquaeti v tort”)
(clojure.string/join " " (take 3 (shuffle (clojure.string/split s " “))))
;=> "augquaeti posue v”
@ag does this look right?
2016:12:13 23:21:57 adambros oh you said generator… hmm
2016:12:13 23:26:46 j-po You could make a generator for sequences fitting that format. Composing in string/join is the unknown bit, but it seems doable.
2016:12:13 23:28:31 j-po You could do that in a custom generator, for instance, but then you'd have a custom generator 😕
2016:12:14 09:16:05 artemyarulin Hello. Where I should put specs: next to functions, next to tests, one global file with all specs for my project. Once I’ve seen [module-name]-spec.clj separate file for each module.
2016:12:14 10:14:28 pyr HI! How do you go about generating related values?
2016:12:14 10:14:47 pyr A simple case would be an indexed map where keys need to reappear in values
2016:12:14 12:18:11 gfredericks @pyr are you familiar with gen/let and/or gen/fmap?
2016:12:14 13:37:19 bronsa I was toying earlier with implementing split-with using spec
2016:12:14 14:53:53 alexmiller the regex derivative is keeping track of all possible solutions - in the last case the split could occur anywhere so there are a large number of those (the others do not have that ambiguity)
2016:12:14 14:54:37 alexmiller so these are (algorithmically) very different specs
2016:12:14 15:00:54 Yehonathan Sharvit what is ident?
2016:12:14 16:48:50 alexmiller Keyword or symbol
2016:12:14 17:13:35 ddellacosta folks I’m a bit confused by something right now, and apologies if this is documented somewhere that I missed:
If I have a spec keyword in namespace x (`(def ::foo …)`) and then I refer to it via a ref in a separate namespace (`[x :as y]`), I can’t seem to use that ref’ed keyword in a defmethod (`(defmethod foo :y/foo …)`)
2016:12:14 17:13:45 ddellacosta does that make sense? Am I doing something obviously wrong?
2016:12:14 17:28:44 hiredman ::y/foo
2016:12:14 17:39:27 josesanch This is not working because s/keys is a http://macro.Is there any way to do this?
2016:12:14 18:16:49 bbloom josesanch: i don’t really think spec wants to be used in that way dynamically. i think it’s somewhat designed to ensure some things are static at useful times.... can you do get-mandatory-fields in a more static way and then validate with a dynamic spec NAME rather than a dynamic spec?
2016:12:14 18:17:04 bbloom something like (s/explain-data (get-spec-name llambda) llambda))
2016:12:14 18:17:36 bbloom otherwise, your choices are eval (probably not a good idea) or macros (which are going to be static anyway)
2016:12:14 18:18:01 josesanch Thank you @bbloom. I was thinking about that. In this case I'm validating each field individually
2016:12:14 18:18:48 bbloom in that case, i suggest using the per-field specs (which presumably already exist?) and a simple loop or map or reduce on top
2016:12:14 18:19:07 bbloom ie do custom validation in addition to the spec validation
2016:12:14 18:19:17 bbloom but others may have other suggestions ¯\(ツ)/¯
2016:12:14 18:21:10 josesanch Yes. This way. Thank you very much.
2016:12:14 18:22:06 bbloom nice. this is one of the great features of spec vs a traditional type system: you can use it as a building block for other dynamic validations
2016:12:14 18:28:29 alexmiller also, (s/keys) will validate all keys
2016:12:14 18:28:55 alexmiller you could combine that with a check for whether a map contains all the required keys
2016:12:14 18:29:11 alexmiller that could be a custom predicate that built from a set of keys
2016:12:14 18:30:49 bbloom hm, are you saying like you’d make a wrapper object {:required #{::foo ::bar}, :object {::foo … ::bar …}} and then validate that with a custom predicate?
2016:12:14 18:31:46 alexmiller no
2016:12:14 18:32:22 alexmiller I’m saying you could (s/and (s/keys) #(every? req-keys-set (keys %)))
2016:12:14 18:32:31 alexmiller I probably typoed that, but you get the idea
2016:12:14 18:34:24 bbloom ah ok
2016:12:14 18:56:59 ddellacosta @hiredman yeah I figured it out after a little while…PBKAC. Thanks!
2016:12:15 20:01:55 sveri I wonder if there is a way to make sense of such a spec message: https://pastebin.com/QrU49Thr
2016:12:15 20:02:49 sveri its hard to figure out if I am just missing a key somewhere or if the structure in general (coll-of (coll-of ...)) is broken
2016:12:15 20:21:49 alexmiller I think there might be more than one thing wrong too
2016:12:15 20:24:56 sveri Can you tell me how you extracted the important information so quick?
2016:12:15 20:25:16 alexmiller looking at the line breaks
2016:12:15 20:25:21 sveri It took me a lot of time figuring out the first error, then reload and look for the second one. My problem is, its just a wall of text.
2016:12:15 20:25:24 alexmiller each line is a problem - those are the ends of the lines
2016:12:15 20:25:56 sveri I see, turning off soft wraps would have helped here.
2016:12:15 20:25:56 alexmiller the (large) data value is the distracting part in each line
2016:12:15 20:26:31 alexmiller you can install a custom explain printer too if you want (could actually hide the val, or limit it’s size)
2016:12:15 20:27:08 sveri That sounds good, my application makes use of an external service, that returns a lot of data. Is there an example somewhere on how to make a custom printer?
2016:12:15 20:27:56 alexmiller *explain-out* is a dynamic variable holding the function that prints explain data
2016:12:15 20:28:31 alexmiller ^^ that will work at the repl, but you might need to use with-bindings to make it work in your code
2016:12:15 20:28:47 alexmiller s/explain-printer is the default print function
2016:12:15 20:29:05 alexmiller this example limits *print-length* to 3 when printing big collections
2016:12:15 20:29:13 alexmiller (that’s the most common source of large printed values)
2016:12:15 20:29:59 alexmiller for example (s/explain empty? (range 100)) with the above will not print the full seq
2016:12:15 20:34:07 sveri Awesome 🙂
First question, may I put that on a gist with your comments and
second: when I try your code it throws the following exception during runtime: java.lang.IllegalStateException: Can't change/establish root binding of: *explain-out* with set
2016:12:15 20:49:28 alexmiller Yes and that's what I mentioned
2016:12:15 20:50:05 alexmiller It works at the repl because the repl binds explain-out
2016:12:15 20:50:55 alexmiller But in your code you need to set it in a dynamic scope with something like binding
2016:12:15 20:51:14 alexmiller Or alter-var-root it
2016:12:15 20:51:36 alexmiller I should probably just write a quick blog on it
2016:12:15 20:56:26 sveri @alexmiller Great, I got it now 🙂 thank you very much
2016:12:16 00:49:40 noprompt i think there’s a bug with :clojure.core.specs/binding-form. it’s defined as
which is the smallest example that demonstrates the problem (`{:as m}` conforms to a [:seq ,,,] value). in order to fix this problem the spec needs to be reorganized to place :map spec above the :seqspec.
2016:12:16 00:56:19 alexmiller Same thing I think?
2016:12:16 01:03:10 noprompt ah nice! yep, that’ll do it too! 🙂
2016:12:16 01:03:26 noprompt (although my patch would have been smaller) 😛
2016:12:16 01:05:20 noprompt (brandon’s is better though) 🙂
2016:12:16 01:11:04 alexmiller We really need a vcat which would be better than either
2016:12:16 01:14:07 alexmiller In particular to c
2016:12:16 01:14:22 alexmiller Conform to a vector
2016:12:16 15:59:39 devth where does spec store registered specs? can i list / inspect them?
2016:12:16 16:14:56 bronsa @devth (s/registry)
2016:12:16 16:15:08 devth @bronsa thanks!
2016:12:16 16:15:49 alexmiller you might also find stest/instrumentable-syms or stest/checkable-syms of interest re functions
2016:12:16 16:16:59 devth ah, very cool
2016:12:17 00:52:42 bbloom i’m trying to get a feel for when it makes sense to use multispec vs or/and/merge/etc.... at first i thought it might be the same tradeoff as with multimethods vs if/case/etc. ie: open vs closed primarily… however i’m not quite sure what the generator behavior will be for and/or/merge w/o the retag function
2016:12:17 00:53:58 bbloom what’s folks experiences with small/closed alternatives? like say 3 cases where all of them share some “base” fields and each has a couple unique fields
2016:12:17 01:06:39 bbloom i think in this case i’m looking at now, i can just ignore the “base” notion, but not quite sure
2016:12:17 05:00:55 tjtolton are videos from the clojure-spec workshop at the recent conj up online anywhere?
2016:12:17 05:01:50 tjtolton the datomic workshop videos are incredible, and depending on how my meeting on monday goes, might mean a new enterprise datomic customer!
2016:12:17 13:07:42 luxbock I started working on a convenience macro for defining a function spec and its definition within the same defn like macro
2016:12:17 13:09:10 luxbock and I'm trying to decide it it would make more sense to infer the spec of the arguments from the argument destructuring syntax, or to allow you to destructure the output of conform on the input parameters
2016:12:17 13:11:01 luxbock so an example of the first would be something like [{:keys-req [foo ::a-foo]}] (the argument vector of the function) expanding to (s/cat :thing-with-foo (s/keys :req [::a-foo])) in the :args part of the fdef form
2016:12:17 13:15:11 luxbock and the second one would look something like [({:keys [xxx yyy]} ::a-foo)] which binds xxx and yyy to be things pulled from (s/conform thing-which-is-a-foo) by wrapping the body of the function definition in a let form that does the destructuring for you
2016:12:17 13:15:58 luxbock I guess these things are not mutually exclusive, so I could just write two separate macros
2016:12:17 13:17:15 luxbock both macros expand to (do (fdef ...) (defn ...)) at the top level
2016:12:17 13:18:57 luxbock maybe I should postpone this until I have more experience with using spec the usual way
2016:12:17 13:28:17 luxbock I suppose you could even combine them by always grouping bindings and specs in parentheses, so you could do [{:keys-req [({:keys [xxx yyy]} ::a-foo)]}] which defines the :args as (s/cat :arg-a (s/keys :req [::a-foo])) and assuming ::foo is something like (s/cat :xxx int? :yyy int?) then you can refer to xxx and yyy as such inside the function body
2016:12:17 13:30:05 luxbock if anyone can think of reasons for why this is a bad idea that'd be great so that I won't have to discover this by trying 🙂
2016:12:17 15:58:01 paytonrules @luxbock I would love to see this macro on github when it’s ready
2016:12:17 18:50:28 bbloom been using update-in, get-in, etc happily for quite some time, but seeing such heavy usage of paths in spec makes me wish Rich would someday give a talk including a rant about xpath/xquery 🙂
2016:12:17 18:51:39 alexmiller Why would he rant about that?
2016:12:17 18:52:09 bbloom i don’t know. i would just like to hear his insights/thoughts about it
2016:12:17 18:52:39 alexmiller I've done a bunch of xquery and I think it's a pretty reasonable functional query language with some good stuff in it
2016:12:17 18:52:54 alexmiller Not all good of course, but nothing is
2016:12:17 18:53:26 bbloom yeah, it’s just sorta this forgotten/unloved old good idea to be able to talk about slices of hierarchical data
2016:12:17 18:54:03 alexmiller So you mean good rants :)
2016:12:17 18:54:04 bbloom there’s stuff like lens in the haskell world and nathan marz’s specter thing (was that what it’s called?) but rich’s take on these sorts of things is always so lucid
2016:12:17 18:54:27 robert-stuttaford Rich’s recent NYC talk mentions a ‘path system’ in spec. is there some place i can read about this?
2016:12:17 18:54:34 bbloom i enjoy a solid rant, so i don’t assign the negative connotation by default 😉
2016:12:17 18:54:50 alexmiller It's just the idea that alternatives or components are tagged
2016:12:17 18:54:56 alexmiller So you can talk about them
2016:12:17 18:55:06 robert-stuttaford ok, so one could form a path into a conformed structure using those names, got it
2016:12:17 18:55:13 robert-stuttaford as with get-in and friends
2016:12:17 18:55:22 bbloom @robert-stuttaford it’s not really a “system” so much as it is two things: 1) a vector of keys a la get-in or update-in and 2) makign sure spec gives you good paths whenever you’d need it
2016:12:17 18:55:25 alexmiller Globally qualified spec name + path together let you talk about any piece
2016:12:17 18:55:38 robert-stuttaford yep, got it — just wanted to check it wasn’t more’n that
2016:12:17 18:56:02 robert-stuttaford how stable is the API now? i took a look in the early alphas, and not since
2016:12:17 18:56:05 alexmiller Yeah it's used more in the second half of spec
2016:12:17 18:56:11 robert-stuttaford wondering if the water’s warm enough now 🙂
2016:12:17 18:56:24 alexmiller (That was a joke btw :)
2016:12:17 18:56:37 alexmiller I'd say the API is pretty stable
2016:12:17 18:57:32 alexmiller We are unhappy with the details of gen override and instrument options and those might change slightly
2016:12:17 18:57:52 alexmiller Unhappy with their inconsistency across fns
2016:12:17 18:58:31 robert-stuttaford and i believe you may also be providing exercise with gen overrides
2016:12:17 18:58:45 robert-stuttaford oh, wait, there it is
2016:12:17 19:00:06 bbloom i’ve got rich’s lisp nyc talk on in the background, that’s why i thought of the xpath thing
2016:12:17 19:00:31 bbloom i haven’t giving the generator side of spec a serious workout yet - probably should make a concerted effort to try
2016:12:17 19:41:16 bbloom I’m surprised to see that *print-namespace-maps* is a boolean value and not a number specifying how big of maps to scan in the private lift-ns function
2016:12:17 19:42:00 bbloom after 1.9 ships i’ll want to add lifted namespace map syntax to fipp, but i want to preserve the linear time and constant space overhead promises
2016:12:17 19:42:37 bbloom surprised to see no such guarantee in the default clojure repl - but then i guess printing large values is sorta bad interactively anyway
2016:12:17 20:01:16 dergutemoritz Is it intentional that there are no bigint? and biginteger? predicates + corresponding generators?
2016:12:17 20:09:37 gfredericks might have to do with cross-platform issues
2016:12:17 20:13:00 dergutemoritz @gfredericks Hmm possibly... there is bigdec?, though
2016:12:17 20:14:06 gfredericks Yeah I dunno. Can write your own though.
2016:12:17 21:01:00 alexmiller I think it was just an oversight and I might even have a ticket for it
2016:12:18 17:33:27 mfikes Is it considered bad practice to employ custom generators to produce values that lead to decent perf of the functions being checked? Motivation: You have a function which, for certain valid inputs, would slow down checking considerably. Would it be considered an abuse of s/with-gen for this purpose, or is there a better way, or is this still an open question?
2016:12:18 18:15:52 mfikes A concrete hypothetical example: Testing the 2-arg arity of repeat. You want the spec to allow any number for n, but you want to limit the size of the generated values of n, just for testing purposes.
2016:12:18 18:27:23 dergutemoritz @mfikes Seems like the essay @gfredericks just posted here is relevant to your question: https://github.com/clojure/test.check/blob/master/doc/growth-and-shrinking.md -- it only hints at sizing being relevant for performance (memory usage) but in general I think it makes sense to apply custom generators for this purpose. I don't have authority on blessing it as a good practice, though 🙂
2016:12:18 19:58:44 alexmiller Seems fine to me
2016:12:18 19:59:26 alexmiller You can also supply the generators at test time rather than in the registered spec if you want smaller scope
2016:12:18 19:59:41 alexmiller With gen overrides
2016:12:18 20:09:32 gfredericks :+1: to all that
2016:12:19 18:26:51 naomarik I recall reading advising caution on using conform awhile back. Is this still the case?
2016:12:19 18:29:49 bbloom are you sure it wasn’t caution on using conformer?
2016:12:19 18:30:44 naomarik i can’t be sure
2016:12:19 18:30:51 naomarik what’s the downside in any case?
2016:12:19 18:31:13 naomarik for conformer, if that’s what it is
2016:12:19 18:32:42 bbloom conformer enables your specs to do a little bit of data coercion
2016:12:19 18:33:10 bbloom sometimes that’s useful to make a spec work, but it’s usually better to do that work in a function rather than a spec
2016:12:19 18:33:35 bbloom ie just translate the data and spec the data that exists before and/or after that translation
2016:12:19 18:33:47 bbloom that’s my best attempt as a short explanation, but others here can probably elaborate
2016:12:19 18:34:28 naomarik okay cool, so there’s no problem with conform then in both the latest cljs and clojure right?
2016:12:19 18:34:46 bbloom yeah, definitely not - conform is a non-trivial part of the utility of spec 😉
2016:12:19 18:35:28 naomarik perfect, gonna save me a non-trivial amount of time writing destructuring branches on my multispecs 😉
2016:12:19 18:41:02 seancorfield Paraphrasing Alex: if your spec also coerces data, bear in mind that this coercion will apply to all clients of that spec.
2016:12:19 18:41:30 seancorfield At World Singles, we have a lot of specs that coerce string data to typed data. We’re OK with that 🙂
2016:12:19 18:42:21 seancorfield Specifically, we have specs for our API calls that accepts strings as input and the result of conforming them is domain model values (so string -> long in some cases and string -> keyword in others).
2016:12:19 18:44:04 seancorfield We generally write the specs for the domain model values first, and then write the API specs in terms of those (so an API spec is (s/and string? (s/conformer some-coercion) ::domain/spec-name) and we have a generator that fmaps the reverse coercion over the ::domain/spec-name (either str or name usually).
2016:12:19 18:45:23 seancorfield We have some shorthand coercion specs tho’ (e.g., ->long which does (try (Long/parseLong s) (catch Exception _ ::s/invalid))).
2016:12:19 18:53:37 naomarik aha
2016:12:19 18:53:58 naomarik i guess it’s extremely helpful when dealing with 3rd party APIs
2016:12:19 18:55:20 bbloom i think what seancofield is saying, and by extension alex, is that conformer couples some interpretation of the data to the specification - however somebody else may want a different interpretation. so if you choose to use conformer, you prevent yourself and others from being able to apply a different interpretation
2016:12:19 18:55:51 naomarik is this the reason to use caution and not that it’s in alpha?
2016:12:19 18:55:53 bbloom if you don’t expect to ever need another interpretation, then go right ahead, but just know you’re making that trade off
2016:12:19 18:56:01 bbloom for conformer specifically, yes
2016:12:19 18:56:06 naomarik perfect thanks
2016:12:19 18:56:49 bbloom seancorfield: the other option would be to not use conformer and then to do a secondary pass where you apply coercions, right?
2016:12:19 18:57:24 bbloom i’ve been hacking on this thing: https://github.com/brandonbloom/ambiparse and i made a decision up front to include semantic rules as a primary feature - it’s basically the same idea as conformers
2016:12:19 18:57:40 bbloom but parsing is different than validation - in parsing, you’re pretty certain to have a lot of “trivia” like whitespace etc that you want to discard
2016:12:19 19:02:36 seancorfield @bbloom My objection to that is that you end up doing the work twice: once to determine whether a value is valid and then again to actually convert it — since you always want to convert it if it is valid.
2016:12:19 19:03:25 seancorfield I mean, you could use a regex I suppose to say “It’s valid if it’s just digits and not longer than N” but that’s not the same as actually parsing a string to get a number — you’ll either allow too many digits or not enough.
2016:12:19 19:03:40 seancorfield And if you decide to use a regex, now you have two problems 🙂
2016:12:19 19:03:40 bbloom sure, the guidance is essentially “consider if you might want to NOT convert it in the future” - and if the answer is “nah, i’ll never need not do that” then go right ahead
2016:12:19 19:03:54 bbloom there’s sorta a 3rd approach
2016:12:19 19:04:20 bbloom which is what instaparse does: allow you to specify rewrite rules independently
2016:12:19 19:04:41 seancorfield And it’s easy to write a non-coercing version of these specs: you just call str or name on the conformed result 🙂(s/and ::api-spec/spec-name (s/conformer str))🙂
2016:12:19 19:04:43 bbloom so like you’d have a map of specs to post-conformers
2016:12:19 19:32:13 naomarik the amazement of discovering exercise is like using the repl for the first time
2016:12:19 19:36:51 naomarik is there a way to pprint the output though?
2016:12:19 19:47:16 bbloom you can just call clojure.pprint/pprint
2016:12:19 19:50:50 naomarik yeah i think it’s fine, it’s just pulling out the common namespace part of the keywords but I guess this is normal behavior for pprint
2016:12:19 19:51:38 naomarik like [#:firstpart {:next “blah"}] for things that are :firstpart/next
2016:12:19 19:55:37 richiardiandrea @naomarik yes that's new syntax for namespaced keys in maps (namespaced maps?), I think it has been added in the first 1.9-alphas. They will all print that way
2016:12:19 19:57:07 richiardiandrea ah ok the bot is not on 1.9 yet
2016:12:19 19:57:32 richiardiandrea
(get #:test{:foo 1 :bar 2} :test/foo)
1
2016:12:19 20:00:06 naomarik ahh okay
2016:12:19 21:24:04 kyle_schmidt Do specs belong with my application code or should I put them in their own namespace? Also, should I assert within application code or use the :pre and :post keywords to check my function i/o?
2016:12:19 21:24:33 bbloom i asked the same question about where to put specs - it seems like both options occasionally make sense
2016:12:19 21:25:07 bbloom it seems like if you have a single-namespace that wants to export some specs, go right ahead and put them inline
2016:12:19 21:25:24 bbloom if you have a bunch of namespaces collaborating around some common stuff, maybe put the specs in their own namespace
2016:12:19 21:25:32 bbloom start single-file and go multi-file if you need to
2016:12:19 21:26:02 bbloom the only decision you really need to make up front is what namespace you want them to land in in terms of usage externally
2016:12:19 21:26:51 bbloom there’s also a risk of a circular dependency between the spec namespace and the other namespaces
2016:12:19 21:36:38 seancorfield @kyle_schmidt As for assert / :pre / :post — “it depends”.
2016:12:19 21:38:05 seancorfield I would normally turn on instrumentation as part of my tests, and have certain tests that run check on key functions (ones that can be generatively tested). I generally do not have assert in any production code and only very rarely have :pre or :post.
2016:12:19 21:39:58 kyle_schmidt Thank you for the replies. And just to make sure, is clojure.spec a replacement for clojure.test? Or would I still want to write both?
2016:12:19 21:42:57 zane In my experience in practice you wind up writing both.
2016:12:19 21:43:12 bbloom spec is definitely not a replacement for test
2016:12:19 21:43:54 bbloom you use your specs from tests
2016:12:19 21:44:04 bbloom at least that’s one use of spec
2016:12:19 21:44:52 kyle_schmidt oh interesting. so use specs to aid tests
2016:12:19 21:45:02 zane Ah, I interpreted that question as being whether clojure.test.check replaces clojure.test.
2016:12:19 21:45:43 bbloom @kyle_schmidt: there’s a bunch of spec videos by stu halloway on youtube that are a good intro of actually doing some testing w/ spec
2016:12:19 21:46:12 kyle_schmidt cool thanks I’ll check them out!
2016:12:19 22:45:52 lambdahands Is there a way to spec out a function as part of a collection, such as a map? For example, if I define a spec like:
2016:12:19 22:47:57 lambdahands I think in practice, I would typically just spec out the function I was using as the callback itself, and spec the key as just fn?. It seems simpler, but I wasn't sure if I was thinking about this correctly
2016:12:19 22:49:38 bfabry @lambdahands I get true for that valid? call
2016:12:19 22:50:13 alexmiller yeah, I would expect that to work
2016:12:19 22:50:24 lambdahands Hmm. I am using clojure-future-spec – let me try with the latest alpha as well.
2016:12:19 22:50:25 alexmiller (also ifn? is usually better than fn?)
2016:12:19 22:50:40 lambdahands Ah, thanks for the tip @alexmiller!
2016:12:19 22:51:59 alexmiller @naomarik you can also turn off namespace map syntax for maps if you want (set! *print-namespace-maps* false)
2016:12:19 22:52:41 alexmiller it will default to false in Clojure but true in the repl
2016:12:19 22:56:22 lambdahands Checked out that spec again, and it does work as you said. Strange, I must have been doing something wrong earlier as I was getting an unexpected result.
2016:12:20 03:01:49 gdeer81 this spec (s/def ::string-num (s/and string? #(not= nil (re-find #"^[0-9]+$" %)))) is unsatisfiable to the generator. I wanted to make sure there wasn't a better way to spec out a string of numbers that could be satisfied before I wrote a custom generator
2016:12:20 08:24:36 artemyarulin Hello. Is there a way to create connected spec generators? I have
I want generator that will returns me accounts like {::user “user1” ::pwd “pwd1”}, but with current setup it will return random user and pwd. I wonder - is there any way to somehow connect specs to generate valid accs for my context?
2016:12:20 12:20:45 gfredericks @artemyarulin is this not what you're seeing, or else what do you mean by "valid accs"? https://www.refheap.com/124315
2016:12:20 12:23:44 artemyarulin @gfredericks Well I meant that valid accounts are pairs of user1 pwd1 and user2 pwd2 and not user1 pwd2 with such data. It was solved like this:
2016:12:20 13:24:19 artemyarulin @alexmiller thanks, I’ll have a look
2016:12:20 14:12:09 yogidevbear Hi everyone. I bought myself Living Clojure for a Christmas present (5 more sleeps to go!). Can anyone recommend a good "beginners" intro course to spec (as this isn't included in the book)? (Other than the standard docs)
2016:12:20 14:13:31 yogidevbear I'm hoping to find something very beginner-friendly
2016:12:20 14:14:02 yogidevbear Thanks in advance 😄
2016:12:20 14:49:24 tjg @yogidevbear I haven’t watched this myself, but I like Lambda Island in general, and their intro to spec is a freebie:
https://lambdaisland.com/episodes/clojure-spec
2016:12:20 14:57:18 mpenet @artemyarulin seems like a s/and with the part containing the s/keys you're interested in with it's own gen might fit better imho
2016:12:20 14:58:46 mpenet I had to do something like this for user profile data that had to return valid/realistic data (gender matching firstname, profile pic matching "fake" user etc)
2016:12:20 15:00:59 mpenet in short, check out :gen in s/keys
2016:12:20 15:01:18 mpenet ./back to xmas stuff
2016:12:20 16:49:29 joshjones @yogidevbear This is not a guide, but it’s been the most helpful for me to think of it this way — the question that helps get things going is: What am I trying to spec? So, I wrote this out in a table on my board, and it’s helped me immensely. First of all, remember that for all intents and purposes, every predicate is a spec. It answers yes or no when given something.
If it’s an associative structure, like a map, you’ll use keys, map-of, and merge.
If it’s “data”, you will use a set as a predicate (`#{}`), your own functions, built-in predicates like string?, pos-int?, and so on, or for ranges, int-of, double-of, and inst-of (for dates)
If it’s a sequential structure (lists, vectors), you’ll use coll-of (where every element is validated by the same predicate), tuple (where each element may have a different predicate), or the regex operations which can be used to combine patterns of specs within the sequence: cat, *, +, ?, alt, keys*, &, and spec (for nesting structures)
You will want to fdef some function specs as well.
So, for me the hard part initially was taking this huge mass of functionality and finding “where do I start?” The above is a very rough, incomplete guide to what you should be using in the spec library to get started, based on what you specifically are trying to spec. Testing, instrumenting, generating data is another thing, but first learn the above and I think it will not be hard to transition from there.
2016:12:20 17:41:48 mpenet to expand on the related key/val before, this ends up something like this:
at least the gen and spec parts are separated this way
2016:12:20 17:47:15 mpenet you can do the same with gen/let or gen/one-of + 1 gen/return per pair
2016:12:20 21:04:00 yogidevbear Thanks @tjg and @joshjones :+1:
@joshjones I think I'm going to have to save your response somewhere so I can digest it further down the line once I'm a bit more familiar with Clojure and some of the terms you're using 😉
2016:12:20 22:24:03 jiangts i’m a bit of a newcomer to spec, but when defining specs can we use other specs instead of predicates? I’m testing in CLJS and it seems that it’s allowed, but then exercise on coll-of seems to break.
2016:12:20 22:25:50 jiangts The following is throwing an error in ClojureScript 1.9.293
2016:12:20 22:28:02 jiangts ah, checking a bit closer it seems that specifying :kind set? does not work
2016:12:20 22:30:41 gdeer81 ✅ I spec'd out a function that takes a set and makes a power set, wanted to know if I'm doing this right https://www.refheap.com/124325
2016:12:20 22:40:52 gdeer81 the funny thing is when you write an :fn spec for a function, now you have two functions to debug.
2016:12:20 22:43:18 bsima hm, spec can't capture side-effects, can it?
2016:12:20 22:46:35 gdeer81 like if you had a function that fired rockets into space and you wanted test.check to be able to fail a case where there weren't rockets in the air?
2016:12:20 22:47:35 bsima yeah, like there's no way to capture rockets-fired = true in the spec definition
2016:12:20 22:48:05 bsima which makes sense, b/c spec isn't trying to be a type system like Haskell's or anything.
2016:12:20 22:50:45 gdeer81 unless your function returned a map like {:num-rockets 5 :target "moon" :fired true} but then its actually your function that captured the side effect
2016:12:20 22:50:55 bsima yeah good point
2016:12:20 22:55:54 joshjones @bsima you cannot capture effects outside a function (AFAIK), but you can capture the return value and its relationship with the args in the :fn portion of the fdef of course
2016:12:20 22:57:56 joshjones I suppose you could use a :postcondition in the definition of the function, or check some outside state inside the function, since you have access to the state anyway.
2016:12:20 23:10:15 gdeer81 @bsima I spec'd out a function that updates a global atom https://www.refheap.com/124326 doesn't smell right
2016:12:20 23:17:17 joshjones Since state is involved, there's no (good) way to avoid race conditions when verifying that the updated value is correct. Code elsewhere could have modified the atom.
2016:12:20 23:20:18 gdeer81 also, the generator is pretty abusive and will throw large numbers at it until it breaks 😆
2016:12:20 23:29:45 gdeer81 @joshjones well the scary thing is that it has passed test.check over 3k times now. I guess I'd have to use pmap to see if I can ferret out race conditions
2016:12:20 23:35:27 joshjones i am guessing check runs the function sequentially, so no danger of seeing race conditions in this case
2016:12:21 00:00:30 bsima gdeer81 that's cool
2016:12:21 00:01:39 gdeer81 I also used pmap to run it over a vector of 99,999 numbers, still no failures
2016:12:21 00:02:40 bsima maybe you could use the funcool "cats" library to handle side effects, and check for right/`left` return types in the spec? idk, haven't used it much
2016:12:21 07:24:35 jiangts using conformer, wondering how to conform a given input efficiently without conforming then unforming
2016:12:21 08:57:08 dergutemoritz @jiangts Can you give an example or elaborate your problem some more? I don't see why you'd need to conform and unform using conformer
2016:12:21 09:10:17 jiangts @dergutemoritz when you coerce with just conform you get a parsed version of the data, including the paths on ors or keys for example. I’d like a coerced version of the data without those key paths, which can be removed with unform
2016:12:21 09:10:49 dergutemoritz Ah, I see
2016:12:21 09:12:11 dergutemoritz There are multiple ways around that. For example, in case of s/or, something like this will do: (s/and (s/or ...) (s/conformer val))
2016:12:21 09:12:38 dergutemoritz For s/cat, often if you don't care about the path labels, you are probably looking for s/tuple instead
2016:12:21 09:14:17 dergutemoritz But perhaps you should think about whether keeping the path labels is actually the better idea!
2016:12:21 19:47:32 ag
2016:12:21 19:58:15 gdeer81 does it make sense to have a separate file for general specs and then inline specs that are specific to that namespace?
2016:12:21 21:46:40 olivergeorge Has anyone written a destructuring to spec translator? Idle thought really. I was wondering about how to spec namespaces with less overhead... a convention based around arglist destructuring and using variable names which match registered specs could work.
e.g.
2016:12:21 22:47:08 olivergeorge Feel free to comment there if it's a terrible idea or I'm missing key points.
2016:12:21 22:53:46 alexmiller for perf reasons, we would not do that in core
2016:12:21 23:13:58 bsima is there a way to write an fdef that says "this function should take :my/spec as it's argument but only after the arg has conformed with (conform :my/spec arg)" ??
2016:12:21 23:17:21 olivergeorge @alexmiller that sounds sensible. hard to imagine it being unopinionated enough to be suitable as a core feature too. Do you think the idea has merit? I guess the real benefit is tidier code which is superficial.
2016:12:21 23:18:06 olivergeorge @alexmiller there was talk of a "dev version" of clojure with more spec integration... less performant but more errors for the developer. Is that still on the cards?
2016:12:21 23:36:32 alexmiller @olivergeorge: seems like most cases that's not what I would want
2016:12:21 23:37:29 alexmiller On the dev version, possibly although just not front burner right now
2016:12:22 00:10:25 olivergeorge Thanks Alex. Enjoy the break.
2016:12:22 00:17:31 naomarik Was wondering, is it possible given two different representations of data to conform to the same thing?
2016:12:22 00:20:59 naomarik Use case would be for instance taking {:key/id {:some :data}} and turning that into {:key/id id :some :data} given either format.
2016:12:22 00:27:24 naomarik this seems like a silly question, doesn’t make sense other than having to transform the data after the fact.
2016:12:22 01:20:24 naomarik I’m unable to figure this out: given example in Rich’s talk if you have a map of email keys and people, how do you spec that? example {”
2016:12:22 01:21:49 gfredericks (s/map-of ::email ::person) I believe
2016:12:22 01:22:16 naomarik hehe i just found that after failing with s/cat and s/tuple
2016:12:22 01:22:17 naomarik thanks 😉
2016:12:22 03:35:22 alexmiller Well you can do it with s/every and s/tuple (which is how map-of is implemented)
2016:12:22 11:02:16 luxbock was there any reason to break the symmetry of symbols always introducing themselves in destructuring syntax by using {:keys [:foo/bar]} instead of {:keys [foo/bar]}?
2016:12:22 11:38:39 luxbock @olivergeorge I added a comment to your gist with some ideas I've been working on
2016:12:22 13:13:25 joost-diepenmaat @luxbock isn’t {:keys [:foo/bar]} for aliased namespaces and {:keys [foo/bar]} for unaliased ns? So foo is an alias in the first expression and a full namespace in the second?
2016:12:22 13:40:54 alexmiller No, those mean the same thing
2016:12:22 13:41:36 alexmiller You can use the first with ::foo/bar though for aliased
2016:12:22 14:04:19 luxbock oh I did misunderstand how it works, I thought {:keys [foo/bar]} would give me foo/bar but just bar works, makes sense
2016:12:22 15:12:37 sveri After using spec for a while now and especially heavily the fdefs I just noticed something important for me. What I did up to now, was to spec out the parameter maps for every function extensively. While now I see that it is more practical and enough for me to just spec out endpoints (like database / rest) and spec my functions in a different way for maps.
What I do now, is to just add the keys to the spec that I require in that function instead of speccing the whole map.
So (s/cat :product ::product-ns/product becomes (s/cat :product (s/keys ::req-un [::product-ns/date-key]
Just wanted to share this 🙂
2016:12:22 15:59:53 alexmiller :+1:
2016:12:22 18:27:50 bbloom what’s the guidance for spec-ing pseudo-namespaced keywords? i’m thinking like datomic attributes when you do :customer/name or something like that
2016:12:22 18:30:14 alexmiller do you mean “qualified but not referring to an actual namespace” keywords?
2016:12:22 18:30:18 bbloom yeah
2016:12:22 18:30:25 alexmiller go for it
2016:12:22 18:30:40 bbloom even when the qualifier is not at all unique?
2016:12:22 18:30:49 alexmiller but if you plan to ever share that code with other people, use adequately unique qualifiers
2016:12:22 18:30:51 bbloom presumably spec’ing something like :event/type may conflict
2016:12:22 18:30:58 alexmiller yes
2016:12:22 18:30:59 bbloom yeah - ok so it’s the app vs lib thing
2016:12:22 18:31:03 alexmiller yeah
2016:12:22 18:31:06 bbloom ok - fair enough
2016:12:22 18:31:09 bbloom thanks
2016:12:22 18:31:19 alexmiller I just wrote up a side bar for this in Programming Clojure :)
2016:12:22 18:31:49 bbloom yeah, i could sense that - i’m awesome like that
2016:12:22 18:31:59 bbloom just helping you test out your advice
2016:12:22 18:32:04 alexmiller well usually people ask about the things I haven’t yet written :)
2016:12:22 18:32:42 bbloom until you write too much and then everybody decides it is easier to ask you than it is to read the mountain of writings! heh
2016:12:22 18:34:55 alexmiller it’s always simultaneously not enough and too much
2016:12:22 18:35:10 bbloom damn tradeoffs strike again
2016:12:22 19:54:54 noprompt (example down at the bottom)
2016:12:22 19:55:55 noprompt it needs just a little more polish but i’ve been happy with it.
2016:12:22 19:58:01 noprompt i put it to use while implementing the CEK, CESK, and CESK* abstract machines for interpreting the pure lambda calculus and it’s been helpful.
2016:12:22 19:58:41 noprompt spared my sanity a bit and help catch a few bugs where i was wiring things up incorrectly.
2016:12:22 19:59:13 noprompt i’m not sure if it’s worth a full blown library or not but i thought i’d share this to show folks something interesting.
2016:12:22 19:59:47 Yehonathan Sharvit where is the code of the pure lambda calculus interpreter?
2016:12:22 20:00:26 noprompt i’ll gist CEK
2016:12:22 20:01:02 bbloom side note: implementing /CES?K/ machines is crazy fun
2016:12:22 20:01:14 bbloom highly recommended to all
2016:12:22 20:02:39 noprompt @viebel i added it as a comment to the gist.
2016:12:22 20:03:30 noprompt @bbloom it really is. a co-worker and i have been taking a deep dive in to this stuff. it’s a ton of fun.
2016:12:22 20:04:03 noprompt i haven’t gotten up to the timestamped or garbage collected implementations yet though.
2016:12:22 20:04:58 noprompt it’s mostly a matter of reading and converting the math into code.
2016:12:22 20:06:53 Yehonathan Sharvit thx @noprompt. You should write an interactive blog post about CEK stuff:clap:
2016:12:22 20:07:13 noprompt anyway, the basic premise of the macro is that we often use symbols in function parameters to stand for something and we tend to reuse them in several functions.
2016:12:22 20:08:32 noprompt the idea here is to specify what those symbols mean and have a macro which handles the business of creating an fdef, instrumenting, etc.
2016:12:22 20:09:14 noprompt take @bbloom’s GLL parser code for example; he’s got a glossary at the top of one of the namespaces which explains what each of the symbols mean (a good practice btw).
2016:12:22 20:13:15 noprompt @viebel i might consider doing that during my vacation.
2016:12:22 20:15:29 noprompt ugh, i just realized i made a mistake specifying ::lam!
2016:12:22 20:26:48 bbloom @noprompt fun to know somebody is reading that code 🙂
2016:12:22 20:27:49 bbloom i have some ideas to dramatically simplify and speed it up too - but have to find a few days to work on it
2016:12:22 20:33:21 noprompt @bbloom i plan to look at it, and the papers, more seriously soon. i implemented a parser combinator library not to long ago and i had to “cheat” in several places to squeeze out more perf.
2016:12:22 20:45:28 bbloom msg me directly if you wanna chat about it - coding this thing gave me a ton of insights
2016:12:22 20:57:36 manutter51 @bbloom I never heard of CES?K machines, do you have a link I could start with?
2016:12:22 20:59:08 bbloom matt might’s blog is where to go
2016:12:22 20:59:34 bbloom he’s got various articles on them
2016:12:22 21:00:28 manutter51 Thanks
2016:12:22 21:32:57 dottedmag I wrote a stateful transducer (I know, I know, it's a variation on a partition theme, so it has to have some state). How do I spec it? I can do it for every arity, but the resulting specs aren't that useful -- I'd better have a spec for the transducing process, not for a single invocation.
2016:12:23 02:40:57 sophiago hi. i took about a month off working on this library i was specing out and now figured it's a good time to come around and finish it since i'm going to be refactoring a bunch of things and would rather not have to end up manually debugging them. the last thing i was stuck on was defining specs for custom types from defrecords. i was trying something like this:
2016:12:23 03:36:43 seancorfield I don't think it's idiomatic to use ? on a spec name (just on a predicate).
2016:12:23 03:37:06 seancorfield ? usually implies a Boolean result.
2016:12:23 03:37:39 seancorfield (sorry for briefness - on my phone) ^ @sophiago
2016:12:23 03:43:33 sophiago thanks @seancorfield! i'll give it a try
2016:12:23 03:55:56 sophiago @seancorfield that's not compiling:
No value supplied for key: (fn* [p1__302#] (instance? p1__302# Rational))
2016:12:23 03:57:58 sophiago it seems you had the arguments to instance reversed, but i get the same stack trace regardless
2016:12:23 04:08:19 sophiago to provide some background on what i'm trying to test here: i have five binary functions defined in protocols over four types and want to enumerate tests on all possible combinations of types as parameters. i figure i'll just focus on getting that right for now, but the the tricky part is i then need to include certain subtyping relations between them (shouldn't be super complicated) and also make sure i have tests where the results are of each possible type to check the coercion function is working (a bit trickier). i think i should be able to build up the subtypes manually once i can figure out how to define basic ones based on defrecords, seeing as they obey certain numerical laws. and then i think i should just be able to write fdefs for each of the five functions where i provide them as a list of inputs to each parameter and then maybe use a cond to determine what the possible return values are given each pairing. the final part i suppose is making sure exercise generates a certain number of tests for each result, which i haven't really thought about yet. hope that makes sense
2016:12:23 04:10:27 joshjones fwiw regarding the previous part of the conversation, there's already a predicate function called rational? so no need to spec it, it's already there
2016:12:23 04:11:37 sophiago but does that pertain to Java ratios? i'm referring to a record called Rational i defined myself
2016:12:23 04:12:04 joshjones ah, i see .. nvmd 🙂
2016:12:23 04:13:55 joshjones i am guessing you've explored multi-spec and that it's not applicable for your use case?
2016:12:23 04:13:58 sophiago thanks, tho...i didn't know they supported that. last i heard there was just int? and float? so seems odd to have rational? given the level of specificity and little use that type gets
2016:12:23 04:14:13 sophiago no, i haven't looked at multi-spec
2016:12:23 04:14:51 joshjones there are several new predicates in 1.9, such as nat-int?, pos-int? .. all very good set of built-in "specs"
2016:12:23 04:15:20 sophiago ah cool
2016:12:23 04:15:45 joshjones the great thing about spec is that any predicate is a spec, so clojure already has tons of them built right in
2016:12:23 04:16:29 joshjones I have not had a need for multi-spec yet but it is related to spec-ing based on types; I suggest looking at that section in the spec guide and perhaps it may be of some use for your use case.
2016:12:23 04:16:49 sophiago yeah actually i am familiar with muli-spec and it may make sense when i get to enumerating the pairings rather than using fdefs, but don't see how it would help with just testing for types
2016:12:23 04:17:26 joshjones oh well, it was worth a shot 🙂
2016:12:23 04:19:06 sophiago the guide says it's for defining things like types within spec, which makes sense, but as far as just speccing out an existing type the suggestion to use instance? seems like it should work...
2016:12:23 04:23:27 sophiago ah! so sorry. i just realized i was using spec instead of def
2016:12:23 04:24:05 sophiago as mentioned...it's been a month
2016:12:23 04:28:29 sophiago ok, hoping this gives me enough to work with until i come back tomorrow with another level of broken 🙂
2016:12:23 04:29:37 sophiago i.e. from here it should be trivial to define subtypes using selectors already have. not sure about matching return values in my fdefs tho...
2016:12:23 05:22:31 wei is there a way to use spec that works with both maps and datomic's entitymaps?
2016:12:23 05:26:32 bbloom there may be something better, but you can use associative?
2016:12:23 05:26:35 bbloom note that will also find vectors
2016:12:23 05:27:13 bbloom so maybe #(and (not (vector? %)) (associative? %)) is passable, but hacky
2016:12:23 08:41:13 seancorfield My initial reaction is you're way-overspecifying things... maybe...
2016:12:23 08:44:10 seancorfield One simplification would be to have a predicate for int or Rational and use that for the numer/denom and real-part/imag-part -- that would cut a lot out of it.
2016:12:23 08:56:21 sophiago @seancorfield yeah, that's one way to think about it. like maybe spec just isn't meant to test for this level of correctness? like whether my coercion functions are reducing values correctly based on the composition of their inputs and outputs. i just wasn't sure if there was a way to automate some of the pairing? otherwise i'm afraid i could be left with something that's not testing that much...
2016:12:23 08:59:25 sophiago for example, i currently have known bugs with subtyping. hence why i want tests for those. that's easy on its own. but the other issue is correctness of coercion up and down the numerical tower and i'm not sure there's any way to test that (that actually makes sense...that is) since it depends on the actual values and calls to gcd
2016:12:23 09:24:07 sophiago @seancorfield i guess on the one hand, i'm not quite sure what you mean by how i could simply it by using predicates for int and Rational? and otoh, defining the possible types was rather easy even if it does seem like overkill... where i'm stuck is if there's a way to at least partially automate testing type coercion by matching input values to return types. here's the last version of the full code if it helps explain what i'm talking about: https://github.com/Sophia-Gold/Symbolic-Algebra.clj/blob/master/src/symbolic_algebra/core.clj
2016:12:23 09:28:02 sophiago oh wait, Sean...you're the one who wrote that old numeric tower library! funny i'm asking you about this then. i've looked at that code a bunch. this started out as intending to be a Scheme-style numeric tower, but obviously with the emphasis on types rather than functions
2016:12:23 13:41:11 Yehonathan Sharvit What is the idiomatic way to define a spec for a collection of numbers not including Double/NaN?
2016:12:23 13:43:11 alexmiller double-in can be used to spec doubles not including NaN - that can be combined with others
2016:12:23 13:44:13 alexmiller Or spec something generic and add a predicate to exclude NaN (but use Double.isNaN, not anything = based)
2016:12:23 13:45:03 Yehonathan Sharvit currently I have (s/def ::coll-of-numbers (s/coll-of number?)
2016:12:23 13:46:40 Yehonathan Sharvit So your suggestion is to create a predicate (defn not-nan-number? [x] (and (number? x) (not (Double.isNaN x)))?
2016:12:23 13:47:57 Yehonathan Sharvit BTW the function I want to spec is a function that receives a collection of numbers and calculates their average
2016:12:23 13:59:15 alexmiller Actually I would use s/and to make a spec, not a pred
2016:12:23 13:59:40 alexmiller Well, you could do it either way
2016:12:23 13:59:50 alexmiller There are tradeoffs
2016:12:23 14:00:10 alexmiller Depends whether you care about gen too
2016:12:23 14:02:49 Yehonathan Sharvit Yeah. I need gen
2016:12:23 14:03:30 Yehonathan Sharvit do you know if there is a isNaN that is cljs compatible?
2016:12:23 14:04:05 Yehonathan Sharvit othrewise I would need to make my own #?(:clj Double/isNaN :cljs js/isNaN)
2016:12:23 15:12:12 bhagany @viebel in cljs, double-in takes a :NaN? parameter… I’m kind of surprised this wasn’t in clojure first? (I’ve only done spec in cljs)
2016:12:23 15:14:18 bhagany so, looks like you can (s/double-in :NaN? false) and get correct check and gen behavior in both clj and cljs
2016:12:23 16:12:42 bhagany Didn’t have time to complete this thought earlier - I’d probably end up with something like (s/or :int int? :double (s/double-in :NaN? false)). This way also has the advantage of easily including ratios, if you wanted to do that. (`number?` won’t gen ratios)
2016:12:23 18:31:41 sophiago going to try this again during the daylight...and hopefully before too many people have left for the weekend, if not already. i'm looking for some advice speccing out this project:
2016:12:23 18:34:20 sophiago essentially i'm just wondering if there's any way to automate pairing return types based on input values like this or whether i'm just being far too ambitious about what spec can do
2016:12:23 18:36:10 sophiago it was also suggested i might be able to simplify how i've specified all the combinations of types, but i'm not quite sure how to go about that
2016:12:23 18:42:47 sophiago i suppose one way would be to use gen with the type constructors instead of using the selectors to define all these types? then feeding those into the input and of the functions i'm using fdef with? i suppose when it comes to checking cases where it should reduce i may have to settle for just enumerating a large number of tests i can go over visually rather than having it flag the return values as invalid for me, although that would be ideal
2016:12:23 19:37:42 assoc-in Does anyone know if when I instrument a function with a spec that defines :ret as int? and the function I am testing returns a string if the spec error should show up in the repl? I was able to get the error of the return not being an int? when I used spec/check but expected it with instrument as well. Thanks!
2016:12:23 19:38:55 assoc-in Also I am able to get the :arg spec error to show up with invalid inputs so I believe I am instrumenting the function correctly.
2016:12:23 19:38:56 seancorfield instrument only verifies that functions are passed valid arguments, i.e., that the calling code “works”. If you want to verify a function “works” — testing the return and invariants — you need to look at clojure.spec.test/check.
2016:12:23 19:39:12 seancorfield Two different types of testing.
2016:12:23 19:40:08 assoc-in Interesting I figured it would be a nice addition to the development workflow being able to check the return types were correct on the fly like the arguments are checked
2016:12:23 20:04:39 sophiago @seancorfield i was confused by your comment on my snippet last night. were you suggesting i use s/valid with the type constructors rather than specifying so many s/defs using the selectors and instance?
2016:12:23 20:06:03 seancorfield I was suggesting reducing the number of specs so that you didn’t have separate cases for int + int / complex + int / int + complex / etc — I think you’re over-specifying things and making life harder for yourself.
2016:12:23 20:07:27 seancorfield @assoc-in Probably a good idea to watch some of the videos online about clojure.spec so you understand the drivers behind its design — since expecting instrument to run :ret and :fn checks is common when folks first start using spec unless they’ve read / watched a bunch of the design justifications.
2016:12:23 20:09:12 seancorfield @assoc-in Not sure if you’re in the #beginners channel? There’s some discussion right now about spec and someone just linked to a great talk by Stu Halloway on it.
2016:12:23 20:12:17 sophiago @seancorfield what i'm really concerned with testing are subtyping + coercion. so i do need those cases specified for it to be useful and at least it's something i can do with relative ease. the latter...i'm less sure about. it may come down to just visually checking a large search space to see if any aren't reduced or if they cause the coercion functions themselves to throw errors. but at least for that i need a variety of subtype relationships
2016:12:23 20:14:21 seancorfield Ah, I see… Then I don’t really have any suggestions...
2016:12:23 20:15:58 sophiago ah ok. thank anyway. i'm just going to start by having it enumerate a certain number of each type per each function i'm testing with no :fn key or more particular restrictions on :ret
2016:12:23 20:16:54 sophiago for the future...if i do want to use a helper function for :fn do you know the syntax for passing the arg values to that? as opposed to what i tried in that snippet?
2016:12:23 20:17:08 sophiago or ret rather
2016:12:23 20:18:09 sophiago i was trying to do something like this : :ret (coerce-types #(:args :a %) #(:args :b %)))
2016:12:23 23:46:05 hlship Is it allowed to instrument protocol methods? That is, I define a protocol, and want to instrument the created dispatch function.
2016:12:23 23:46:43 hlship So far, this is not working; I’d imagine that it would instrument the function to verify arguments & etc., then delegate to the original function to actually dispatch on type to an instance of the protocol.
2016:12:23 23:47:16 hlship Never mind, it works. Problem w/ a test.
2016:12:23 23:51:12 sophiago @hlship i was going to say...that's exactly what i'm working on right now!
2016:12:23 23:54:03 hlship It’s odd; I get the expected failures from the REPL, but inside tests, I can pass non-confirming values without failure.
2016:12:23 23:54:19 hlship I suspect this would work if it was a normal function, but Protocol methods are their own beasts.
2016:12:23 23:55:08 sophiago hmm...that's odd
2016:12:23 23:55:56 sophiago this is my first go at fully testing a project and i'm having trouble getting the syntax correct it seems
2016:12:23 23:56:24 hlship I can imagine any number of load-order or bytecode generating things that could be getting in the way.
2016:12:23 23:56:44 hlship I’d hate to have to add a function in front of the method, just to get the desired test-time spec help.
2016:12:23 23:57:01 sophiago yeah, i'm not planning on doing that either
2016:12:24 00:18:56 sophiago i still can't figure out how to specify a list of possible s/defs to pass to :args and :ret😕 i keep getting a "No value supplied for key" error
2016:12:24 02:28:27 seancorfield @hlship see http://dev.clojure.org/jira/browse/CLJ-1941 -- known places where instrument doesn't work (includes primitive returning fn and protocols)
2016:12:24 02:33:50 sophiago ah, that says only primitives are affected tho?
2016:12:24 02:34:02 seancorfield Protocols are mentioned in the comments.
2016:12:24 02:34:30 seancorfield Can you share the problematic code re: :args:ret?
2016:12:24 02:35:19 sophiago i'm just trying to pass a list of specs to each:
2016:12:24 02:35:51 sophiago can't imagine this is the correct way to do it?
2016:12:24 02:36:55 seancorfield remove the [a b] s-exp
2016:12:24 02:37:01 seancorfield Then it should work.
2016:12:24 02:37:40 seancorfield As for ::sym -- it can't be all those things at once -- shouldn't that be s/or?
2016:12:24 02:37:58 seancorfield and int? to make it a predicate
2016:12:24 02:38:26 sophiago oh yeah. i was playing around with several expressions trying to get it to work
2016:12:24 02:59:18 seancorfield Basically you need to specify a generator that maps your record constructor over a generator for the values it accepts
2016:12:24 02:59:36 sophiago ah i see
2016:12:24 02:59:58 sophiago i truly did make this much more difficult by trying to go that granular
2016:12:24 03:00:22 sophiago it wasn't something obvious in the guide i missed going over this all day
2016:12:24 03:00:50 sophiago ok, thanks. going to give it a try
2016:12:24 03:02:42 seancorfield Making specs generative can be quite the process... we try to do that with all of ours but sometimes it takes... ingenuity...
2016:12:24 03:06:09 sophiago well i was trying to get it to somehow enumerate tests on the correct output types automatically...
2016:12:24 03:06:38 sophiago this is actually much simpler than i imagined
2016:12:24 03:06:45 sophiago but still not covered in the guide
2016:12:24 03:07:13 seancorfield Gary Fredericks has created a good companion library for spec that helps generate all sorts of things BTW (we use it, specifically, for regex generation)
2016:12:24 03:07:30 sophiago oh i think i saw that
2016:12:24 03:08:07 sophiago test.chuck
2016:12:24 03:08:41 sophiago i just glanced over the functions and don't think it would help in this case though
2016:12:24 03:09:04 sophiago tbh this is not so complicated. protocols are generally really easy to debug
2016:12:24 03:09:43 sophiago i actually thought since it's so systematic it would be good to learn how to spec a whole project so i could apply it to code where i really can't figure out what's going on
doesn't complain that foo is supposed to return string? values
2016:12:24 03:28:37 sophiago well i passed it the whole list anyway 🙂
2016:12:24 03:28:45 sophiago i'll have to check it manually
2016:12:24 03:29:27 sophiago but it will tell me if any subtyping combinations are throwing errors, which i believe was the case before
2016:12:24 03:34:07 sophiago huh. i'm getting the same error...
2016:12:24 10:25:43 sveri I get this error: RuntimeException Var clojure.test.check.generators/keyword-ns is not on the classpath clojure.spec.gen/dynaload (gen.clj:21)when running this code: (gen/generate (s/gen number?))in the REPL. Any ideas what I am missing here?
2016:12:24 11:59:17 gfredericks @sveri do you have a test.check dependency declared?
2016:12:24 11:59:34 sveri @gfredericks No, I dont think so
2016:12:24 11:59:40 gfredericks you need one to use generators
2016:12:24 11:59:43 sveri So this is needed whenever I want to use generators
2016:12:24 18:30:00 sophiago what is clojure.spec/Specize? i'm getting the following error: IllegalArgumentException No implementation of method: :specize* of protocol: #'clojure.spec/Specize found for class: nil
2016:12:24 18:34:36 donaldball IIRC it’s the protocol by which values are coerced into specs, notably by which keywords resolve themselves in the spec registry
2016:12:24 18:48:32 sophiago ah, so it does have to do with trying to exercise protocol functions? because i think i may be experiencing this issue, but am having trouble isolating it to be sure: http://dev.clojure.org/jira/browse/CLJ-1941
2016:12:24 18:49:07 sophiago to be fair, that's the not the error reported in that ticket
2016:12:24 18:53:41 sophiago my case is complicated because the protocol functions i'd like to test all dispatch on records so theoretically should be unaffected. however, i went so granular as to define arguments to their type constructors using s/with-gen and gen/fmap. so i can't tell if some of the errors are actually what i'm looking for, known subtyping bugs, because i'm not sure if the generators that don't do that are affected and am currently getting three different error codes dependent on whether the arguments to the type constuctors are primitives or other records and also (the really odd part) the function itself, which shouldn't make a difference as far as i can tell
2016:12:25 09:18:37 naomarik @bbloom > what’s the guidance for spec-ing pseudo-namespaced keywords? i’m thinking like datomic attributes when you do :customer/name or something like that
I’ve actually been using it like this pretty much exclusively and I feel reading and referring to the spec becomes pretty trivial especially when you have multiple types that have the same name but the namespace disambiguates it (like multiple name fields for instance :customer/name :business/name)
2016:12:25 10:03:52 dvcrn Hey guys! Using clojure-spec for the first time to describe a db schema. Does this look okay or am I doing something obvious wrong?
2016:12:25 10:35:45 naomarik so these are fully qualified keywords
2016:12:25 10:35:58 naomarik which is very useful imo when dealing with multiple entities in the same map
2016:12:25 10:36:20 naomarik helps disambiguate whose ID or name you’re talking about
2016:12:25 10:36:41 dvcrn right 🙂
2016:12:25 10:37:03 dvcrn so I guess fully qualified / namespaced keywords are better
2016:12:25 10:38:05 naomarik i’m not qualified to give an answer on that but i can say at least in my extremely limited usage on an immature project it has helped me discern what data i’m looking at
2016:12:25 10:38:58 dvcrn yeah completely makes sense
2016:12:25 11:24:01 nooga unqualified keywords would usually come from/go on wires
2016:12:26 16:59:32 roelof I have this error message :
(clojure.spec.gen/generate (clojure.spec/gen :paintings2.routes.home/page))
RuntimeException Var clojure.test.check.generators/large-integer is not on the classpath clojure.spec.gen/dynaload (gen.clj:21)
2016:12:26 17:04:57 roelof oke, now I see another error message :
ExceptionInfo Unable to construct gen at: [] for: (conformer ->long) clojure.core/ex-info (core.clj:4725)
2016:12:26 17:06:10 joshjones now you're getting somewhere -- i get why you want to use a conformer, but i've not used a conformer to generate data before, nor have i seen any info on doing so -- only seen info on conforming/validating using a conformer
2016:12:26 17:08:04 joshjones so you just want to generate numbers in a certain range?
2016:12:26 17:08:36 roelof yes, I want to check if the url parameter is between the 1 and 471 or is not there
2016:12:26 17:08:54 roelof is the last case the pagenumber schould be set to 1
2016:12:26 17:09:40 joshjones and the input can be either a string or a number, right?
2016:12:26 17:11:20 joshjones the way you've written the spec for ::page, I do not think there's enough info there for it to do any data generation
2016:12:26 17:12:28 roelof oke, what I try to do it this : The user can use this url : http://localhost:3000/
2016:12:26 17:12:47 roelof and then my code schould take that the pagenumber is 1
2016:12:26 17:58:39 roelof @seancorfield The last question is about spec because josh thinks that my fdef for testing and generating test-data is not good
2016:12:26 17:59:37 joshjones @roelof Use the above generator and revised def for ::page and you'll be able to generate data using ::page
2016:12:26 18:01:39 roelof and I can use that data for checking the page-check function
2016:12:26 18:03:17 roelof @joshjones Am I right that I cannot check what happens if a user enters false data like this : ' /?pag="a"
2016:12:26 18:07:36 seancorfield @roelof Although this is about clojure.spec, there are still a lot of beginner-level issues at play here so I’d suggest keeping to the #beginners channel where folks have opted into helping folks with this level of discussion, rather than swamping the other channels with long back and forth about syntax and basic errors.
2016:12:27 09:37:08 curlyfry Hi, I have the following scenario:
I want to keep these specs in the same file, but I also want to disambiguate the different ::name specs, without giving them different names. Is there any way to do this? Something like ::french/name vs just ::name, but that isn't working.
2016:12:27 10:13:27 mpenet @curlyfry you ca use create-ns and alias for that
2016:12:27 10:22:07 curlyfry @mpenet Cool, thanks! Can I create a concatenation of the "current" ns with a symbol using create-ns?
2016:12:27 13:31:35 roelof I hope this is not a beginner question.
2016:12:27 13:31:48 roelof I want to test a function that has as input something like this : [{ :name "Roelof", :place "internet"} {:name "Jane Doe" :place "unknown" }] and as output {"Roelof", "Jane Doe"} . If I want to test this , can I use the same idea as this :
(def email-regex #"^[a-zA-Z0-9._%+-]
2016:12:27 13:33:22 roelof or what must I fill in if a use a fdef with :args and :ret and :fn
2016:12:27 16:47:00 joshjones Perhaps your question is constructed in a way that makes answering it a bit difficult. For example, “input something like this” is ambiguous — are there always two maps in the input vector, or might there be more? And, do you really mean to have a single k/v pair map as the output, where both the key and value are strings? Or did you mean to make that a vector of the :names from the input? It’s clearing up this confusion that results in the things the admins don’t want in the channel, I think.
But here’s some info that might be valuable to anyone new to spec, just a sample of how to construct what you’re asking for. What you want to do is ask, “what am I trying to spec”? In your case, it’s a function which takes a vector of maps, and returns a vector of strings. Those are your :args and :ret, and a decent sanity check for :fn is that the number of names you return should equal the number of maps you were given. So, here’s a spec based on that:
2016:12:28 20:53:59 joshjones I’m a little confused on fmap vs bind — according to the docs for clojure.test.check.generators/bind: Create a new generator that passes the result of gen into function k. k should return a new generator.fmap also takes a generator, and applies some function to it. The only difference I can see is that in bind, the function is responsible for returning a generator, and in fmap, the function just returns data.
Is there some other difference I’m not seeing?
2016:12:28 21:07:40 gfredericks That's exactly the difference
2016:12:28 21:08:40 gfredericks bind is more powerful than fmap
2016:12:28 21:20:26 sophiago so the idea is this is somewhat like Haskell with generators analogous to monads? that comparison doesn't make the most sense to me tho
2016:12:28 21:56:15 gfredericks Generators are a monad, yes
2016:12:28 21:56:56 gfredericks I expect that's literally true in quickcheck, which test.check is largely based on
2016:12:28 22:15:15 sophiago i suppose in the sense that they're bound to a certain type? but i don't see how the monad laws could even possibly apply to generators
2016:12:28 22:34:15 gfredericks The first three make sense to me. I'll translate them to generators when I get to a keyboard
2016:12:28 23:02:16 gfredericks ↑ I haven't grokked the fourth one but it probably works ¯\(ツ)/¯
2016:12:28 23:02:28 gfredericks @sophiago
2016:12:28 23:05:49 sophiago i don't think that SO post is the best source. i've always only known three laws: left identity, right identity, and associativity. https://wiki.haskell.org/Monad_Laws
2016:12:28 23:06:35 gfredericks looks like it just skips the 3rd one
2016:12:28 23:06:37 sophiago the associativity one in your example is unclear to me
2016:12:28 23:06:41 gfredericks so the first two are equivalent
2016:12:28 23:06:58 sophiago the first two make sense. i didn't know there was a gen/return either!
2016:12:28 23:07:00 gfredericks we could experiment to see if the two seem to do the same thing 🙂
2016:12:28 23:08:13 sophiago i'm confused about the purpose of gen/return from reading the api docs
2016:12:28 23:08:25 gfredericks what it does, or when it's useful?
2016:12:28 23:08:40 sophiago just what it does
2016:12:28 23:09:05 sophiago i suppose returns a generator? lol
2016:12:28 23:09:19 sophiago so maybe i should have asked the second question...
2016:12:28 23:09:45 sophiago like in what case would you bind a generator and then return it?
2016:12:28 23:12:22 gfredericks bind+return is just fmap
2016:12:28 23:12:28 gfredericks so it's easier to use fmap
2016:12:28 23:12:41 gfredericks return is more often useful on its own
2016:12:28 23:12:56 gfredericks I'll go grep my source code to see when I use it
2016:12:28 23:16:05 gfredericks the associativity law looks right to me
2016:12:28 23:16:29 gfredericks it's just double-binding in two different ways; looks suspiciously trivial actually
2016:12:28 23:16:37 sophiago woah, that's a really cool application of generators!
2016:12:28 23:17:15 gfredericks 🙂
2016:12:28 23:17:23 sophiago yes...because >>= is the bind operator 🙂
2016:12:28 23:18:24 sophiago so if you can associate that then you've met the third monad law...really the important one in that it lets you compose monads that encapsulate different typeclasses
2016:12:28 23:19:04 sophiago this would make a good blog post or something to that effect
2016:12:28 23:19:04 gfredericks I would need an example of such a composition to see what that means
2016:12:28 23:20:11 sophiago it doesn't have a corollary in dynamic typing. that's why i always thought algo.monad was quite odd...
2016:12:28 23:20:40 sophiago but it makes more sense with generators because they do in a sense have type signatures
2016:12:28 23:20:40 gfredericks yeah I've never seen the point of a monad library in clojure
2016:12:28 23:21:14 sophiago it's really just a library of macros that needlessly replicate the functionality of a number of common haskell monads
2016:12:28 23:21:14 gfredericks too much you can't do and not enough help from the compiler
2016:12:28 23:22:17 sophiago but this makes sense. you need bind in order to compose generators with different types
2016:12:28 23:24:07 sophiago wow, now i'm really interested in uses of spec beyond things like quickcheck. i haven't seen other examples actually using generators to write code, although i think rich touched on it in his talk i saw
2016:12:28 23:24:37 gfredericks I'm not even sure what that means
2016:12:28 23:24:45 gfredericks generating code?
2016:12:28 23:25:31 sophiago well, i only glossed over your chess example, but it seems you're using them to actually generate moves rather than to just test functions you've written?
2016:12:28 23:27:24 gfredericks no it's still for tests
2016:12:28 23:27:40 gfredericks it is generating moves, but only in service of generating legal positions, which is for testing
2016:12:28 23:28:07 sophiago yeah i looked at it again. at first i thought you were using it to play chess
2016:12:28 23:28:10 gfredericks I don't think test.check generators are necessarily the best fit for non-testing purposes
2016:12:28 23:28:40 gfredericks generally they'll give you distributions that are a bit weird outside of testing
2016:12:28 23:29:41 sophiago yeah, i've only noticed that now that i've started getting them working for whole projects...are they purposefully non-random or that's just a consequence of their implementation?
2016:12:28 23:30:25 gfredericks they are random
2016:12:28 23:30:42 gfredericks just skewed in their distribution, as a heuristic for catching bugs
2016:12:28 23:32:20 sophiago i mean, for example, if i call (gen/sample (s/gen int?) n) the size of the ints increase with n. initially they're quite small...which, as you said, makes sense for testing
2016:12:28 23:32:45 gfredericks yeah, that's half of the weirdness
2016:12:28 23:33:01 gfredericks that's demonstrating that gen/sample uses an increasing size parameter
2016:12:28 23:33:12 sophiago oh
2016:12:28 23:33:33 gfredericks but even if you fix size, which you can do with gen/generate, there are still elements of the "smaller things are worth trying more often" heuristic
2016:12:28 23:37:48 gfredericks I'm trying to do a big documentation overhaul, so I'll try to look for ways to make discovery better too
2016:12:28 23:41:27 sophiago yeah, your test.check docs are most useful for what i personally needed than most of what's in alex's guide. like i could have used gen/nat earlier to avoid divide by zero errors and instead rolled my own
2016:12:28 23:47:25 sophiago anyway, i think it'd be interesting to have something about the generator combinators and category theory. it's an interesting consequence of how they're typed that we're not used to in a dynamic language
2016:12:28 23:48:54 sophiago i'd consider writing something like that for a wiki if i thought i could both make it accurate and substantial enough for anyone to care about 😛
2016:12:28 23:50:10 gfredericks that would be fun to read
2016:12:29 07:08:05 seako also, i would like chime in that i also think it would be very interesting to read about generator combinators and category theory
2016:12:29 14:43:00 stephenmhopper I have a spec that I've written with clojure.spec and I can use it to generate valid Clojure data structures. Can I use it to generate invalid structures?
2016:12:29 14:45:02 gfredericks generating things in a negative way is hard to do in general
2016:12:29 14:45:38 gfredericks I expect clojure.spec makes no attempt to support that, since it's not obvious what strategy it would use
2016:12:29 14:46:24 gfredericks you can use unit testing for that, or custom generators to cover more specific cases
2016:12:29 14:46:51 gfredericks for a lot of cases you might do fine by using gen/any and filtering out things that match the spec
2016:12:29 14:50:35 stephenmhopper I did not know about gen/any, but that should do the trick. I was considering just creating a version of my schema where all required keys are instead marked as optional while also changing the data type / generators for some of the keys, but gen/any is simpler
2016:12:29 14:50:41 stephenmhopper @gfredericks thank you
2016:12:29 14:51:01 gfredericks np
2016:12:29 15:06:44 naomarik are you guys converting database records field names to namespaced keywords?
2016:12:29 15:07:05 naomarik or just specing database stuff with :req-un
2016:12:29 15:13:46 gfredericks I think converting would be cooler, but it takes more work and I haven't worked on something like that yet anyhow
2016:12:29 15:14:08 gfredericks e.g., a namespace per table
2016:12:29 15:14:15 naomarik yes this is what i’d like
2016:12:29 15:14:42 naomarik i spent some time specing all my stuff like that and while i can just use req-un i would rather see it qualified
2016:12:29 15:15:55 naomarik would be great if i didn’t have to rename joined tables either, ie multiple id fields
2016:12:29 15:24:46 gfredericks right
2016:12:29 16:16:00 naomarik there’s anyway to conform a spec given keys with :req-un into the matched qualified keywords?
2016:12:29 16:55:34 joshjones I do not think so; do you have a particular use case for this?
2016:12:29 16:57:57 seancorfield FWIW clojure.java.jdbc supports producing qualified names via the :qualifier option. Doesn't handle joins tho'.
2016:12:29 16:59:40 joshjones You could use s/describe on the spec to get the vector of :req-uns, and then pull the key with the name of the key you want to qualify, grab its namespace, and assoc a “newly created” fully qualified key onto the conformed data
2016:12:29 17:00:05 joshjones If you really wanted to. 🙂
2016:12:29 17:13:38 naomarik @seancorfield ya saw that, will probably do that for non joined data and use an identifier that will handle select AS statements
2016:12:29 17:41:47 joshjones just as an exercise @naomarik :
2016:12:29 18:08:39 joshjones @seancorfield you just mentioned in the #clojure channel about using with-redefs to stub functions for testing; what’s your take on taking a spec’d function and now being able to do this, versus the with-redefs approach?:
(stest/instrument `my-func {:stub #{`my-func}})
2016:12:29 18:09:58 seancorfield @joshjones I haven’t experimented with stubs in clojure.spec yet so I don’t have an opinion.
2016:12:29 18:10:26 seancorfield Mostly we’re spec’ing data structures and using s/conform etc — we’re not spec’ing functions much.
2016:12:29 18:14:55 joshjones ok — I have not tried it for production-level testing either, but it looks useful:
2016:12:29 18:17:31 seancorfield So it stubs the named function to return generated data per spec rather than actually calling it? Interesting. Wouldn’t work for anything that actually required mock state, rather than just stubbed data, but I can see that being useful sometimes.
2016:12:29 18:18:29 seancorfield I haven’t generally found stubs to be as useful as mocks so I don’t know how much I’d use it.
2016:12:29 18:18:35 joshjones right, that’s a good point
2016:12:29 18:19:21 seancorfield I can see it being useful as you’re developing top-down where you stub functions as placeholders until you actually write them.
2016:12:29 18:20:59 seancorfield Right now, I’d actually write the physical stub… with the spec approach you could just get instrument to “write the stub” for you, but then you’d have to re-instrument or un-instrument as you actually wrote each function.
2016:12:29 18:21:07 joshjones is there something you typically use for mocking state? or it just depends on your particular application?
2016:12:29 18:21:32 roelof How can i check in a spec for a key in a nested map ? so this is valid (s/explain ::objectNumber-list [{:body{:artObjects{:objectNumber "sk-c-5"}}}])
2016:12:29 18:21:37 seancorfield Since mocks tend to be pretty tied to your implementation, I generally just hand-roll them.
2016:12:29 18:22:03 seancorfield @roelof Please keep working in the #beginners channel on that.
2016:12:29 18:22:18 seancorfield You need to think carefully about the elements of that list, is all I’ll say.
2016:12:29 18:24:15 seancorfield @joshjones When we mock things, we tend to need to coordinate data across a series of calls — or even just record data across a series of calls — so custom functions that use atoms or refs to track that is our “go to” approach.
2016:12:29 18:25:04 seancorfield For example, we mock a payment provider to track calls and amounts, and pass or fail the transaction based on state.
2016:12:29 18:25:54 joshjones I guess the amount of data you mock is pretty small though right? i.e., the purpose of the mocking at this stage is not for load-testing, which is done in a completely different way.
2016:12:29 18:27:22 joshjones for example, we load test our elasticsearch code using dedicated aws servers, so it’s purely a load test — for testing correctness, etc., small amounts of mock data is used locally to be sure everything works as expected.
2016:12:29 18:29:26 seancorfield Mocking is to verify that a function calls some other function(s) in a particular way and/or that a series of calls to the mocked functions returns data in a particular way.
2016:12:29 20:23:09 henriklundahl @roelof, you spec each value which is associated with a specific key and then specify which keys should be included in each map. Start with the innermost map(s) and work your way out.
2016:12:30 09:00:10 sophiago @seako thanks! i've been busy so just getting around to reading this, but it looks really interesting!
2016:12:30 09:03:18 sophiago and i'll think a little more about writing the post. it's not so complicated from the category theory side, so i don't think i need to worry about my level of knowledge with that...more so with test.check. maybe if i've worked with it a bunch more by the time @gfredericks revamps his docs i can add it to the wiki there
2016:12:31 00:14:12 sophiago for example, in the program i just specced out i originally wanted to actually match the correctness of args and return values, which would have meant a quadratic number of sfdefs. i had a list of i think 11 types and five functions...so if i'm doing the math right that would have meant 605 different definitions? now i'm thinking it could actually be possible to enumerate that entire testing space if i learned to have the specs generate themselves... something to think about
2016:12:31 22:09:36 bbloom is there some way to get more info from this error: "Couldn't satisfy such-that predicate” ? the ex-info map is empty
2016:12:31 22:09:46 bbloom i’m curious what predicate is failing
2016:12:31 22:09:51 gfredericks it's um
2016:12:31 22:10:00 gfredericks probably from an s/and somewhere
2016:12:31 22:10:20 gfredericks test.check master has a mechanism for spec to add more info in this case, but spec would have to be updated as well
2016:12:31 22:10:24 gfredericks it's something rich asked for
2016:12:31 22:11:13 bbloom ok cool - will just bin-search it like i do practically all my debugging 😉
2016:12:31 22:11:43 gfredericks yep 😢
2017:01:01 11:31:11 tianshu does clojure.spec.test/instrument support check :ret and :fn for common function call?
2017:01:01 11:31:46 tianshu currently it seems only work for test.check.
2017:01:01 16:22:00 settinghead I noticed spec tolerates ambiguity (say (s/cat :first (s/* number?) :second (s/* number?)) can interpret [1 2 3] in 4 different ways, but it returns the first way it could find). is there any way to disallow ambiguity?
2017:01:01 16:54:12 english @doglooksgood from the spec guide: "Note that the :ret and :fn specs are not checked with instrumentation as validating the implementation should occur at testing time."
2017:01:01 18:45:24 mingp Those of you who use a lot of spec, what would you say to concerns that spec makes you code a certain way to be able to work with it, e.g. namespaced keywords for map keys? This is something that has worried me and made me hesitant to start using spec, but of course I say this as someone who hasn't used it, so what would those of you who have say?
2017:01:01 18:46:34 mingp Something else I've been wondering. To what extent are you all using function specs vs explicit conform? Do you leave both/either on in production?
2017:01:01 18:47:41 notanon @mingp what is the concern with namespaced keywords?
2017:01:01 18:49:02 mingp My concern is, more broadly speaking, that spec expects me to code a certain way to be able to work with it. In the specific case of map keys, I'm normally in the habit of using regular keywords, but spec would work best if I switch to namespaced keywords instead.
2017:01:01 18:49:24 mingp Again, I say this as not an actual user, so I'm trying to ask actual users if they believe these concerns are valid and if I should worry about them.
2017:01:01 18:49:46 notanon well you can still use unqualified keys, it's fully supported
2017:01:01 18:50:31 notanon the reason for namespaced keywords is just because specs are global, and so must have unique names
2017:01:01 18:50:44 sveri I am not sure if thats a good approach, but, what I do is to use namespaced keywords for my definitions and unqualified names for the maps like (s/def ::foo-map (s/keys :req-un [::bar-key ::baz-key]))
2017:01:01 18:50:45 notanon otherwise you wouldnt be able to reuse specs so easily
2017:01:01 18:51:39 mingp More generally speaking, would you say spec has/hasn't constrained how you code Clojure?
2017:01:01 18:51:39 notanon sveri's example is a good example of a compromise, ::foo-map can be re-used all across the app
2017:01:01 18:52:54 notanon the only downside is that ::bar-key and ::bas-key will not be self-describing but... that's probably not going to be an issue for most use cases
2017:01:01 18:54:15 notanon @mingp i don't see it constraining me personally. it's just a name
2017:01:01 18:54:57 mingp That's good to hear. Thanks for that.
2017:01:01 19:46:43 seancorfield Note that you can use namespace qualifiers on your :req-un keys to distinguish between different uses of the "same name".
2017:01:01 19:48:25 seancorfield (s/keys :req-un [:foo/id]) is still the unqualified :id in the map but it uses the (qualified) :foo/id spec.
2017:01:01 19:50:01 seancorfield So while ::id would be the same spec for all :id fields, :foo/id and :bar/id can be distinct :id specs.
2017:01:01 20:03:48 joshjones @mingp As sean just said, just to be clear -- spec totally works with unqualified map keys. What's required to be namespace qualified are the keywords that name specs themselves.
2017:01:01 20:10:53 notanon a subtle, but important difference. said another way (because it's subtle) only the specs themselves need to be namespaced qualified. the data they're spec'ing does not need to be qualified.
2017:01:01 20:14:40 mingp Thanks @seancorfield @joshjones and @notanon.
2017:01:01 22:25:20 alexmiller spec would like to encourage you to work with maps with qualified keywords, but will support you to a limited degree if you can’t/don’t want to.
2017:01:01 22:26:56 alexmiller from my own work with using specs while developing new code, I have found it has made me more likely to break out predicates and helper functions about my data (which has made the code better)
2017:01:01 22:29:36 notanon I think I've heard or read that the fully qualified keys will make your data 'self-describing' but I really don't understand what that is buying me
2017:01:01 22:50:33 potetm @notanon You get validation on keys you might not even know exist, because they're in a global registry.
2017:01:01 22:54:55 potetm That's only possible because they're in a global registry. And to form a global registry, you need keys that are globally unique.
2017:01:01 22:58:59 notanon hm. i don't think i get that, if I conform or validate based on a spec i've defined, i wouldnt have told it the 'keys i dont know about' are of this spec. and of course i might not want it conform/validate keys i didn't ask it to
2017:01:01 23:04:42 alexmiller s/keys will validate all qualified keys in the map based on the matching spec in the registry
2017:01:01 23:05:23 alexmiller we’re suggesting that you should want this :)
2017:01:01 23:07:45 alexmiller this may seem weird, but it’s a key enabler for the kind of evolution Rich talked about in his Conj keynote https://youtu.be/oyLBGkS5ICk where you are “growing” a system over time
2017:01:01 23:08:52 notanon yeah that makes sense. if i add a key later, it will automatically be validated without me updating the spec
2017:01:01 23:23:40 notanon i guess if i just wanted to validate the presence of keys (only presence) i wouldn't use s/keys. not to mention that i don't think that would be very useful
2017:01:02 09:23:23 roelof now I have to extend them for another call with some things
2017:01:02 09:24:05 roelof Can I just make a copy with it and extend it and named all the keywords then ::data-response
2017:01:02 09:24:16 roelof or is there a better way ?
2017:01:02 11:11:29 luxbock so I'm working on my idea of writing a defn-like macro that infers specs from the (extended) destructuring syntax of the argument vector
2017:01:02 11:12:52 luxbock and I was thinking that in the case of a simple case of {:keys [(foo int?)]}, which of course means that the passed map contains an optional unqualified key of :foo which is an int
2017:01:02 11:14:01 luxbock I need to define the spec for :foo. would it be a terrible idea to just use a randomly generated ns for that such as :foo1235/foo?
2017:01:02 11:15:19 luxbock i.e. the argument vector [{:keys [(foo int?)]}] expands to (s/def :foo1235/foo int?) + (s/cat :m (s/keys :opt-un [:foo1235/foo]))
2017:01:02 11:17:12 luxbock I don't want to end up in a situation where my defn-macro ends up overwriting specs for situations where multiple functions accept maps that contain simple-keywords that can contain different types of values
2017:01:02 16:08:39 roelof nobody who can help me with this one
2017:01:02 17:18:11 sveri @roelof It is hard to tell what exactly you want to do. At least for me. I am pretty sure you can do what you want, but if you need help I guess you need to be more specific and provide some examples.
2017:01:02 17:32:58 seancorfield Also, it seems to be the same question you’ve asked several times now — and it has been answered in the #beginners channel...
2017:01:02 17:33:59 seancorfield I have answered it at least twice now in the #beginners channel and you’ve had input from other people there.
2017:01:02 17:47:23 fadrian I'm doing my best to understand spec. I'm working through a cards example, using my own representation of a card. I've defined my specs as follows:
2017:01:02 17:49:42 fadrian According to the spec spec, my definition for cards should be checking for correct conformance for ::suit and ::rank. But (s/conform ::card "1m") returns "1m", not an invalid as I expect. What's happening here?
2017:01:02 18:31:28 gfredericks @luxbock I'd use the current ns plus an extra random segment as the namespace
2017:01:02 18:33:09 seancorfield @fadrian suits should be a set, not a map.
2017:01:02 18:33:14 seancorfield (line 1)
2017:01:03 13:46:28 curlyfry What is the best way to spec a string that contains a floating point number? Using a regex or some other method?
2017:01:03 13:52:50 mpenet Float/parseFloat comes to mind
2017:01:03 13:53:43 curlyfry So what would the spec look like? In particular, would I have to write a custom generator?
2017:01:03 14:00:31 donaldball Possibly using gen/double* with some constraints
2017:01:03 14:19:45 curlyfry @donaldball Cool, thanks! Unfortunately I'm in cljs-land, so I think I might have to go the regex path anyway
2017:01:03 14:20:35 dialelo maybe js/parseFloat can help then @curlyfry
2017:01:03 14:22:23 curlyfry Right!
2017:01:03 14:24:32 curlyfry Just one more thing:
When you make a spec like (s/def ::double-string #(Double/parseDouble %)) we rely on parseDouble throwing an exception rather than matching a certain predicate, right?
2017:01:03 14:45:23 curlyfry js/parseFloat is a bit too forgiving: (js/parseFloat "1.23abcdef") => 1.23
2017:01:03 14:47:26 mpenet if you end up doing that a lot (specing via regex) you might want to check test.chuck for the gen part
2017:01:03 14:48:31 mpenet especially test.chuck.generators/string-from-regex
2017:01:03 14:49:00 mpenet I've thrown crazy regexes at it and it never failed me (so far)
2017:01:03 15:08:11 joshjones @curlyfry not saying a nicer way doesn’t exist, but that way is straightforward and works well, and it’s the way i’d do it
2017:01:03 15:17:37 roelof @seancorfield I know that you answered it with I think a namespace and a function ( artObjects-id:response ) but when I used it in a sdef I did not work
2017:01:04 18:30:44 zmaril is there a way of hiding a tag in spec? In instaparse, there is <a> = "a" and the hide-tag combinator
2017:01:04 18:33:27 seancorfield Do you mean from an s/or clause?
2017:01:04 18:36:46 zmaril yes or a s/cat clause
2017:01:04 18:38:24 joshjones “hiding” it in what way?
2017:01:04 18:38:41 joshjones in the conformed value, the tag will be the key in the map identifying the value
2017:01:04 18:46:41 zmaril hmmm okay nvm
2017:01:05 04:11:16 bbloom so when we’re not using generative tests, what do people use for utilizing specs from tests?
2017:01:05 04:13:24 bbloom i find s/assert annoying b/c of the check-asserts compile time flag - need to forcibly reload everything that might have asserts when working interactively
2017:01:05 04:15:27 bbloom instrument is nicely dynamic, but i understand why the asserts are static
2017:01:05 04:16:50 bbloom i still don’t really understand why :ret and :fn are not checked when instrumented
2017:01:05 04:17:29 bbloom The guide now says "Note that the :ret and :fn specs are not checked with instrumentation as validating the implementation should occur at testing time.” - but instrument are in the stest namespace, soo presumably you only instrument at test time anyway?
2017:01:05 04:18:27 bbloom I get the desire to not instrument ret and fn for other peoples code in some cases, but honestly, i also want to validate that the libraries i’m calling respect their specs
2017:01:05 04:18:29 bbloom i don’t trust them
2017:01:05 04:19:24 bbloom i’m just going to call s/assert* for manual (non-generative) tests and deal with it if it breaks at some point 😉
2017:01:05 12:53:38 dacopare Hello, I've just started using Spec! I'm trying to use it in a particular situation that I can't see covered in the documentation but I may be wrong.
I have two datastructures:
2017:01:05 12:54:50 dacopare I'd like to have a different spec for each of the different :data items, but from what I can see s/keys will use one spec for one key...
2017:01:05 12:57:00 dacopare So that a deleted item would have different required keys to a created item. Is the problem that my keys are not namespaced? Or is this going against:
> "the map spec never specifies the value spec for the attributes, only what attributes are required or optional"
2017:01:05 12:57:50 dacopare Thank you for any help simple_smile
2017:01:05 13:00:03 manutter51 Yeah, this is exactly the use case for namespaces. You want to re-use the same keyword to mean 2 different things, put them in different namespaces and you’ll be fine
2017:01:05 13:27:29 dacopare @manutter51 the only problem I have with this is that I currently have functions that accept either of these structures, and rely on the presence of the data key.
2017:01:05 13:56:39 thomas Hi, just as an idea/question I had... is it possible the do clever things with date/times in spec. I assume you can say that a certain value should be an date/time instance... but wouldn't it be nice if you could express in a spec that a certain date/time should come before/after a different date/time ?
2017:01:05 13:57:41 thomas for instance {:start #inst 2017-01-01 :end #inst 2017-01-5} and if the end is before the start it fails to validate?
2017:01:05 14:00:37 mpenet I guess you can do that with s/and
2017:01:05 14:01:28 mpenet (s/and (s/keys ....) #(start<end? %)) where the latter function checks the values
2017:01:05 14:03:47 thomas ok cool @mpenet that looks like a good solution.
2017:01:05 14:35:29 alexmiller oh, just another case. the bug is in spec itself, not in what’s in the post.
2017:01:05 14:36:18 Yehonathan Sharvit I never meant that there was a bug in the post
2017:01:05 14:36:29 Yehonathan Sharvit 😊
2017:01:05 14:36:43 alexmiller well, not sure why you’re asking then
2017:01:05 14:37:56 Yehonathan Sharvit I’m asking because after reading your post, people might try to do a “conform unform” loop and they might be surprised by the result...
2017:01:05 15:18:50 martinklepsch I’ve been using collection-check https://github.com/ztellman/collection-check a while ago — is something like that becoming easier with spec? Maybe there are generators etc for ·built-in data structures?
2017:01:05 15:28:00 joshjones @dacopare Given your data, you can still use unqualified keys in your existing data. The spec keys must be qualified, but not the keys in your map. They would look something like this:
2017:01:05 15:45:41 alexmiller @martinklepsch I’m not sure spec adds much over collection-check, which has a pretty specific purpose. spec does allow you to gen from collection specs, but not sure if that’s easier. depends what you’re trying to do.
2017:01:05 15:45:49 thegeez @alexmiller in the blogpost the result for the 3rd "WORK IN PROGRESS" block should be: (s/conform ::seq-binding-form '[a b & r :as s])
{:elems [a b], :rest {:amp &, :form r}, :as {:as :as, :sym s}}
2017:01:05 15:46:35 alexmiller ah, thx
2017:01:05 15:47:25 martinklepsch @alexmiller I basically just want to check if a thing behaves like a set, a vector, a list. Guess collection-check it still is then 🙂 Thought maybe through specs for core it would have become easier to generate operations
2017:01:05 15:50:48 alexmiller yeah, nothing at the level of collection-check
2017:01:05 18:20:10 bbloom one thing i learned from your blog post was that simple-symbol? existed 🙂 i had defined my own unqualified-symbol?
2017:01:05 18:51:30 manderson 500 fake internet points to the person that spots the problem (took me all morning to debug this and thought it'd be more fun to make it a challenge simple_smile )
2017:01:05 18:54:24 bfabry name is optional in your fdef but not in your function?
2017:01:05 18:55:14 manderson Ah, good catch. That's an artifact of copy/paste (my original run-thread! had 2 arities where name was optional). But, that's not the issue that got me...
2017:01:05 18:55:42 manderson fixed it 🙂
2017:01:05 18:56:56 manderson I think that's worth 50 fake internet points, though 😉
2017:01:05 18:57:28 manderson hint: it is related to spec...
2017:01:05 18:57:29 bfabry other than that seems to work for me. except of course there's not really any reason why either sleep or println would throw an exception
2017:01:05 18:57:55 manderson well, the catch exception is for killing the thread with (.interrupt t)
2017:01:05 18:58:26 manderson try pasting the function definitions in the repl first and then hit enter. after that paste the instrument and def separately...
2017:01:05 18:59:14 manderson (just to make sure you are instrumenting on the newly added run-thread! spec)
2017:01:05 18:59:41 bfabry doesn't that need to be `run-thread! for that?
2017:01:05 19:00:41 bfabry ah, nope
2017:01:05 19:01:12 bfabry iunno, nothing particularly interesting is happening for me
2017:01:05 19:01:35 manderson Some more detail:
What this does is runs the my-runnable in the current REPL thread and never returns from run-thread! when I would expect the run-thread! to return immediately with the Thread object and the printlns happen in the background.
2017:01:05 19:02:34 manderson Reason: spec/instrument attempts to validate the my-runnable lambda per my fspec, before calling the function, which starts a continuous loop and the run-thread! fn is never called!
2017:01:05 19:02:58 manderson Lesson learned: don't instrument a "runnable" fn that doesn't terminate...
2017:01:05 19:03:11 bfabry ah. I see. yes don't instrument functions with side-effects
2017:01:05 19:03:26 manderson Yep. Makes perfect sense now, but took me a while to figure it out 🙂
2017:01:05 19:04:28 bfabry yeah I totally forgot, but it has been mentioned before
2017:01:05 19:04:37 manderson Thought I could save someone a potential headache by mentioning here...
2017:01:05 19:04:57 bfabry there was talk of more coming for dealing with spec+side-effects maybe, but I'm not sure if it still is
2017:01:05 19:05:03 manderson Yeah, it's one of those things you know in the back of your mind, but forget when in the thick of debugging a weird error
2017:01:05 20:53:45 bbloom i’m not sure i understand what you say is happening here...
2017:01:05 20:55:01 bbloom then call f or (.start (Thread. f))
2017:01:05 20:56:07 bbloom from what i can tell, the spec-checking-fn is called by Thread.run
2017:01:05 20:56:13 bbloom which seems right to me… sooo what’s the problem?
2017:01:05 20:59:26 joshjones what are you trying to accomplish?
2017:01:05 20:59:46 joshjones When I do this I get a stack trace, is this not what you want?
2017:01:05 21:00:03 bbloom I was trying to understand @manderson and bfabry’s discussion above
2017:01:05 21:03:07 manderson @bbloom problem is with the fspec. It's not intended for fn's with side effects, which a while true looping runnable most certainly is. When instrument is turned on, it attempts to validate the fspec by testing it out with different data to ensure it conforms. This ends up running the loop in the current thread and never actually executing the originally intended function.
2017:01:05 21:03:48 manderson Threw me for a while, because it looked like it was running as intended, but was running in the current thread.
2017:01:05 21:03:56 manderson As part of instrumentation
2017:01:05 21:04:09 bbloom huh? fspec should work just fine for functions with side effects outside of generative testing
2017:01:05 21:04:21 bbloom what “different data” is it using?
2017:01:05 21:06:38 bbloom instrumented functions are only called once - with the same args they are given, and presumably on the same thread they would normally have been called on
2017:01:05 21:07:39 gfredericks @bbloom instrumenting uses random data to validate functions passed as an arg to the instrumented fn
2017:01:05 21:07:59 bbloom wait … what?
2017:01:05 21:08:10 gfredericks e.g., if you spec clojure.core/map
2017:01:05 21:08:15 gfredericks whose first arg is a function
2017:01:05 21:08:18 gfredericks then you instrument map
2017:01:05 21:08:25 gfredericks then you call it with some function
2017:01:05 21:08:37 gfredericks that function will be test.checked as part of the arg-validation step
2017:01:05 21:08:54 bbloom whoaaaa OK now it finally makes some sense why :fn and :ret aren’t checked by instrument
2017:01:05 21:09:09 gfredericks yeah? I haven't connected these two things
2017:01:05 21:09:34 bbloom i was under the impression the purpose of instrument was to make sure i wasn't calling other people’s stuff wrong
2017:01:05 21:09:51 bbloom and that it was also used during generative testing to validate my stuff
2017:01:05 21:10:01 bbloom i had no idea generative data was happening if i did instrument normally
2017:01:05 21:10:02 gfredericks if somebody wrote a clickbaity listicle of facts about clojure.spec this would be the "number seven will shock you"
2017:01:05 21:10:43 gfredericks it's not generatively tested the instrumented function
2017:01:05 21:11:05 gfredericks it's generatively testing fspec args to the instrumented function; so for non-HOFs this doesn't apply at all
2017:01:05 21:11:15 joshjones “number three: ALL spec’d keys in a map are validated, not only those explicitly stated as :req‘d …” 🙂
2017:01:05 21:11:21 manderson yep, i discovered this ^^^ when i had some println's for debugging and saw a bunch junk printed out i wasn't expecting. just didn't connect it with my issue above, but it hits home the point as to why fspec isn't for side effects
2017:01:05 21:11:33 manderson the generative testing of fspec, that is
2017:01:05 21:13:49 gfredericks something like that
2017:01:05 21:14:05 gfredericks imagining it only took 2 args
2017:01:05 21:14:22 gfredericks if you instrument map, that means you're asking for its args to be validated
2017:01:05 21:14:37 gfredericks so you instrument and then from the repl you call (map inc [1 2 3])
2017:01:05 21:14:57 gfredericks so you must be expecting spec to validate that [1 2 3] matches (s/coll-of any?), but what do you expect it to do with inc?
2017:01:05 21:15:09 gfredericks it could merely check that inc is an IFn
2017:01:05 21:15:18 bbloom holy hell - that’s what i would have expected: to just check ifn
2017:01:05 21:15:25 bbloom i expected only instrumentation of var calls
2017:01:05 21:15:29 gfredericks number seven will shock you
2017:01:05 21:15:36 bbloom i would not expect instrumentation of higher order functions - that’s crazy
2017:01:05 21:15:43 bbloom clojure is just not prepared for that
2017:01:05 21:15:45 bbloom heh
2017:01:05 21:16:00 bbloom in the absence of chaperones, proxies, and all that other crazy platform stuff that racket contracts have
2017:01:05 21:16:09 bbloom in the presence of pervasive effects
2017:01:05 21:16:12 bbloom that’s just nuts
2017:01:05 21:17:50 bbloom nnnow i understand - here i am trying to recreate the problem discussed above with thread/start etc, but it was only a problem with higher order functions in there
2017:01:05 21:18:54 joshjones well, that’s what fspec is mostly concerned with — hof
2017:01:05 21:19:11 manderson As example:
(spec/fdef test-fn
:args (spec/cat :str string?
:fn (spec/fspec :args (spec/cat :x string?) :ret nil?)) :ret nil?)
(clojure.spec.test/instrument)
(test-fn "hi" #(println %))
U
O
6K1Y
sqD
4PsKAL4
1wbc5
1kq9bvc
wqgAN0gb
kdKAI1aE24
22r
2
6TUvuD2x8c
95NKQmTX6c7kdTO
98O0dGrT9vr65
PE5712237F7
6
Rwc93xcj510gn7
MOvNSAJl
Received x: hi
hi
=> nil
2017:01:05 21:19:37 manderson So, if #(println %) was instead a fn with side effects (looping, hitting a db, etc), then you can see the problem...
2017:01:05 21:19:57 potetm Interesting. So, long story short: HOFs are "instrumented" via generative testing?
2017:01:05 21:20:20 bbloom yeah
2017:01:05 21:20:38 bbloom seems really weird to me
2017:01:05 21:20:40 potetm I'd never considered it directly. I think deep down I was thinking they would be proxied.
2017:01:05 21:20:42 manderson so, in my case, i was seeing my runnable fn executing, and it looked like it was functioning as normal, except it was running in the current thread instead of in a new thread.
2017:01:05 21:20:47 bbloom yeah proxied makes more sense
2017:01:05 21:21:28 potetm But I understand the argument for non-proxied verification.
2017:01:05 21:21:31 bbloom ie you check it’s ifn? or fn? and then wrap it with clojure.spec.test/spec-checking-fn (private function that does the proxying)
2017:01:05 21:21:38 bbloom i don't
2017:01:05 21:21:46 bbloom this makes instrument much less useful for repl use
2017:01:05 21:22:21 manderson yep, i went back and changed some of my specs to #(instance? IFn %)
2017:01:05 21:22:26 gfredericks rich talked about the proxying approach in a ML thread somewhere
2017:01:05 21:22:37 potetm TBH, I'm not sure what utility there is in specing a side-effecting function
2017:01:05 21:22:50 bbloom before i look for that thread - the reason i wouldn’t expect proxying is b/c it breaks equality potentially
2017:01:05 21:22:58 gfredericks I think he thought it was difficult to design well because of having to track the proxy as it flies around the program
2017:01:05 21:23:59 potetm Yeah that's^ what I was thinking.
2017:01:05 21:24:00 gfredericks once you use proxying, the validation of your function can happen anywhere at any time, not just at the moment of calling it
2017:01:05 21:24:06 potetm "We do verification here and only here."
2017:01:05 21:24:15 gfredericks e.g., with map, it would immediately return a lazy seq
2017:01:05 21:24:29 gfredericks and then blow up later when somebody consumed it? or not blow up because the function call already ended?
2017:01:05 21:24:38 potetm Creates separation in execution.
2017:01:05 21:24:47 gfredericks if you unstrument before consuming the lazy seq what should it do?
2017:01:05 21:24:51 bbloom this is all why my assumption was that instrument only instrumented vars
2017:01:05 21:25:05 potetm right
2017:01:05 21:25:11 bbloom ie it was a better than nothing interactive tool
2017:01:05 21:25:37 bbloom cover the 90% case, rather than attempt the 100% case with fallout such as “can’t use instrument if you fspec any side effecting functinos"
2017:01:05 21:25:39 potetm Yeah the fact that it does that by default and "transparently" is... unexpected.
2017:01:05 21:26:25 bfabry instrument does only instrument vars
2017:01:05 21:26:45 bbloom well, i mean only calls to vars and not use the generators
2017:01:05 21:27:21 bfabry but, instrumentation means validation of arguments to a function, if one of your argument is an fspec, validation necessarily means generative testing
2017:01:05 21:27:49 potetm code is data, data is code bruh
2017:01:05 21:27:50 bfabry so, don't write an fspec for functions that could potentially be side-effecting 🙂
2017:01:05 21:28:52 gfredericks @bbloom I think I'm remembering a long thread on the clojure ml (not -dev) from 3-9 months ago
2017:01:05 21:32:04 bfabry I suppose the other thing you could do is stub out specs for functions that are potentially side-effecting
2017:01:05 21:32:38 potetm That seems to be the intended path^
2017:01:05 21:33:06 bfabry hopefully the next screencast/blog on spec is advice on impure functions 🙂
2017:01:05 21:37:47 potetm Yeah.... it's still interesting to me that they would commit to verifying HOF args. I think see the argument (pun definitely intended), but "if 'x is on classpath' do 'y' else 'z'" seems complex.
2017:01:05 21:38:40 potetm I mean, I think it brings back up the question: "What's the relative utility of specing side-effecting fns?"
2017:01:05 21:39:37 potetm If the answer is "so low it's not worth specing them," then committing to verifying HOFs makes a lot more sense to me.
2017:01:05 21:40:43 bbloom it’s only low value if you view spec-ing as a testing mechanism and not as developer guard rails
2017:01:05 21:41:31 potetm Yeah perhaps. OTOH, if you view side-effects as poison, no guard rail will save you.
2017:01:05 21:41:43 bbloom i don’t view side effects that way tho
2017:01:05 21:42:04 potetm Yeah, Rich has said exactly that I think (in Simple Made Easy)
2017:01:05 21:42:07 bbloom you can say that the function passed to reduce should be effect-free, but it’s frequently useful to gather side data or whatever
2017:01:05 21:42:56 potetm (Not saying it's right or wrong. But the decision makes sense when viewed with that mindset I think)
2017:01:05 21:43:14 bbloom i think the decision makes sense when you consider that conform is trying to say “yup, this thing is valid"
2017:01:05 21:43:20 bbloom vs “as far as i can tell, this thing isn’t invalid"
2017:01:05 21:43:31 bbloom the former requires checking everything, the later doesn't
2017:01:05 21:43:44 bbloom if you want to validate some data off the wire for security reasons, you need the former
2017:01:05 21:44:27 bbloom but that doesn’t work in the presence of higher order functions or mutation - as the racket contracts folks learned, hence proxies, chaperones, etc - and still, they have problems with side effects b/c they don’t have an effect handlers system
2017:01:05 21:45:04 bbloom @gfredericks I realize I’m a heretic in clojure-land, but i abuse side effects quite frequently 😛
2017:01:05 21:46:17 potetm I've never used it that way either 🙂 I've use reductions when I'm curious what happens during reducing, but not side-accumulation.
2017:01:05 21:46:29 joshjones I can also say I’ve only ever used the reducing function to .. well, reduce the collection at hand
2017:01:05 21:46:34 gfredericks @bbloom you're a monster
2017:01:05 21:47:12 joshjones @bbloom can you give an example of a side-effecting reducing function? (like a common use case for what you’re describing)
2017:01:05 21:48:11 gfredericks @bbloom do you sprinkle local atoms all over the place and swap them a bunch and then deref them to figure out what happened
2017:01:05 21:48:26 bbloom i’ve done stuff like that - yeah
2017:01:05 21:48:55 gfredericks okay.
2017:01:05 21:48:58 bbloom if it’s local to a function or a concrete process, it’s totally fine
2017:01:05 21:49:04 bbloom “tree falls in a forrest” and all that
2017:01:05 21:49:17 potetm Anyways, not sure I understand what @bbloom was trying to say about conformers with regard to the decision to validate higher order fns, but it makes some amount of sense to me having thought about it a minute.
2017:01:05 21:49:20 bbloom especially when perf is involved
2017:01:05 21:49:35 bbloom using map/filter/reduce etc in 9 passes over a long sequence is just not OK for some use cases
2017:01:05 21:50:02 bbloom i’d love to be able to tell the compiler “hey, these three things traverse the same data structure, do loop fusion” but that’s just not realistic
2017:01:05 21:50:17 joshjones transducers in the 9 pass case! 🙂 (if applicable)
2017:01:05 21:50:28 potetm Yeah, I'm curious how that's more useful than loop/recur or doseq+accumulator.
2017:01:05 21:50:45 bbloom i do the loop/recur or doseq thing plenty too
2017:01:05 21:50:53 potetm Yeah me too
2017:01:05 21:50:54 bbloom but sometimes yo ujust already have a function, so you just call reduce 🙂
2017:01:05 21:51:02 potetm lol truestory
2017:01:05 21:51:43 bbloom anyway - this is waaaay besides the point
2017:01:05 21:52:15 bbloom point is that valid? wants to return true, but really the best it can do in the presence of fspec is say “maybe true"
2017:01:05 21:52:33 bbloom also, i gotta run
2017:01:05 21:52:38 bbloom thanks all for the discussion
2017:01:06 00:36:02 uwo how would you recommend handling this? The map I’m validating has an attribute whose value is an entity. Depending on the context, at times that entity will have only one required attribute, and at other times it will have many required attributes.
So, given two forms:
(s/def ::form1 (s/keys :req [::attr1]))
(s/def ::form2 (s/keys :req [::attr1]))
;; what we want in form 1
(s/def ::attr1 (s/keys :req [::attr-a]))
;; what we want in form 2
(s/def ::attr1 (s/keys :req [::attr-a ::attr-b]))
It seems to me, we’ll have to create another version of ::attr1, but that seems problematic because we use the ::attr1 key to mean the same entity across the project.
2017:01:06 01:00:20 bbloom there are a few things you can do that depends on your situation
2017:01:06 01:00:39 bbloom the simplest is just make the “sometimes required” fields optional and call that good enough, if it meets your validation needs
2017:01:06 01:01:19 bbloom another thing you can do is to have decorated and undecorated versions of the data, and just put them at two differently named keys
2017:01:06 01:02:21 bbloom so instead of (update m :foo assoc :bar 123), you do (assoc m :foo-bared (assoc (:foo m) :bar 123)
2017:01:06 01:02:37 bbloom then use foo vs foo-bared as appropriate
2017:01:06 01:03:06 bbloom i’ll leave more complex solutions for others to discuss 🙂
2017:01:06 04:30:10 joshjones @uwo The part that interests me is the "depending on the context" -- you might be tempted to do something like this:
2017:01:06 04:31:12 joshjones however, this does not do you any good, as in a context where you need multiple the single will still match ... can you be more specific on "context"?
2017:01:06 04:31:52 uwo @bbloom thanks. I’ve got to be strict about required versus optional because I’ve got to bark at the user if they don’t provide a field in a particular context. I’ll have to mull over what the decorated versus undecorated means
2017:01:06 04:35:04 uwo @joshjones thanks. the “context" is two separate forms, one of the forms has a subset of the fields in the other. Both forms describe the same entities. It’s just one form requires more than the other. I’m loath to change to name (or ns) of the attribute, because it’s really the same entity
2017:01:06 04:35:30 bbloom @uwo is your form necessarily hierarchical?
2017:01:06 04:35:45 uwo yeah, it gets very nested
2017:01:06 04:36:42 bbloom hm, yeah, so this is something was talking about with @gfredericks i believe last week or so: parameterized specs
2017:01:06 04:37:00 bbloom or maybe it was @seancorfield
2017:01:06 04:37:01 uwo I haven’t played it out fully yet, but I think the s/or may work @joshjones
2017:01:06 04:37:32 uwo @bbloom yeah, I was wondering if there was some avenue to parameterization
2017:01:06 04:37:55 joshjones so, what is the criteria, in plain english, for choosing map spec one, versus map spec two? It sounds like you are saying "map spec one has these keys" and "map spec two has these keys" ... but the presence or absence of keys is not really an appropriate way to spec it. If you have a key itself, something like {:data {...} :type "A"} .. then you can identify which "version" of the map you have on hand
2017:01:06 04:38:37 joshjones said another way, you have to bark at the user if they don't provide a field, in WHAT context? how do you know?
2017:01:06 04:39:49 joshjones the s/or won't work, for the reason i described ... the single will always match, regardless of your context
2017:01:06 04:40:09 uwo @joshjones there are two separate forms on different pages. I was simply going to validate against a different aggregate spec on each. Do I understand your question?
2017:01:06 04:40:17 bbloom as a total NON SUGGESTION, but just for funsies: i bet you could hack a (very bad) solution with s/conformer and s/and… idea is you assoc in a parameter and then use arbitrary code to look for it — oh man, that’s an evil hack, i kinda wanna try it.... for science....
2017:01:06 04:40:53 joshjones @bbloom we've established you're a heretic, please cease your evil ways
2017:01:06 04:41:14 bbloom never.
2017:01:06 04:41:15 uwo @joshjones oh oh. lol reading comprehension fail. I just looked at your example the first time around. yeah, you’re right that’s no good 😊
2017:01:06 04:41:53 uwo for science...
2017:01:06 04:41:59 joshjones are the keys in the maps namespace-qualified? or no?
2017:01:06 04:42:03 uwo yes
2017:01:06 04:42:10 bbloom if you use un-qualified keys, you can build up a library of specs for non-recursive parts and then manually recreate all the recursive bits
2017:01:06 04:42:17 bbloom but really, there’s probably a better representation
2017:01:06 04:42:24 bbloom especially a non-recursive one
2017:01:06 04:42:30 uwo hehe. using qualified keys. They’re the same names we use in datomic
2017:01:06 04:43:25 joshjones i suggest two specs then .. it's the most straightforward, and models your use case
2017:01:06 04:43:56 joshjones are you checking the spec as a function arg, or what?
2017:01:06 04:44:05 bbloom i’d like to know more about your use case, maybe we can propose a simplification
2017:01:06 04:44:05 uwo Do I misunderstand spec? Can I have two specs for the same (namespaced) key?
2017:01:06 04:44:27 bbloom no, you can’t, which is an intentional design decision
2017:01:06 04:44:30 bbloom and a pretty good one at that 😉
2017:01:06 04:44:45 uwo cool. that’s what I assumed
2017:01:06 04:44:51 joshjones are you checking the spec as a function arg?
2017:01:06 04:44:58 bbloom the challenge is that specs aren’t really context sensitive in any useful way
2017:01:06 04:45:10 bbloom also probably on purpose, but questionable if you actually do care about context
2017:01:06 04:45:11 uwo @joshjones sorry not sure I follow.
2017:01:06 04:45:20 bbloom if i were you, i’d probably use spec in a context-insensitive way
2017:01:06 04:45:20 joshjones how do you plan to use this spec?
2017:01:06 04:45:23 bbloom ie make everything optional
2017:01:06 04:45:26 bbloom and then check required independently
2017:01:06 04:45:34 bbloom ie don’t use spec for required’s take
2017:01:06 04:45:51 bbloom if your required fields are context sensitive, then the fields are optional from the perspective of spec
2017:01:06 04:46:00 bbloom you can conform and then ALSO traverse yourself to check required
2017:01:06 04:46:00 uwo ah. grab the underlying data structure from the form and then verify it against the spec
2017:01:06 04:46:26 joshjones and something in the data structure identifies it as one version, or the other?
2017:01:06 04:46:30 bbloom sure
2017:01:06 04:46:34 uwo @bbloom hmm. yeah, that’s a possibility
2017:01:06 04:47:14 bbloom my apologies for my goofy hacky mood .... this is real advice: don’t try to force spec to do 100% of the work. use it for whatever % you can get away with, and write your own code to do the rest
2017:01:06 04:47:59 bbloom clojure is still really good at recursive functions 😉
2017:01:06 04:48:06 uwo @joshjones not really no. one form is a proper subset of the other. And, as forms go, validations run while they’re partially filled out, so there’s really no way to tell a difference from the specs perspective. Of course, I know what form I’m on, and so could call with an appropriate spec
2017:01:06 04:48:24 joshjones how do you know what form you're on?
2017:01:06 04:48:36 uwo @bbloom heh. thanks. we already have an implementation that doesn’t leverage spec very much. I was just wondering if I could improve what we have
2017:01:06 04:50:07 uwo @joshjones I think we may be passing around a form key that further identifies it, now that I think about that 😊
2017:01:06 04:50:17 joshjones one of two things should happen:
1) since you know at this point in your code which type of data you're dealing with, then you should have a different spec for each, and validate against that spec.
2) your map itself should contain a key which specifies which type of map this is. in this case, use a multi spec
2017:01:06 04:50:37 bbloom you can use conform at the leaves of your own recursive validation function
2017:01:06 04:52:55 joshjones is the form key a clojure keyword, or something else?
2017:01:06 04:54:05 uwo yes
2017:01:06 04:56:57 joshjones doing the spec, give me a sec
2017:01:06 05:03:30 uwo I can see the multispec approach working. So, I’d need an additional key in the model, which I’m guessing would be transitory (I wouldn’t persist it to the db)
2017:01:06 05:04:18 uwo thanks, by the way. I really appreciate your time!
2017:01:06 05:04:28 uwo @joshjones @bbloom ^
2017:01:06 05:05:30 bbloom yeah, so multi-spec works, but you need to eliminate context for nested stuff
2017:01:06 05:05:46 joshjones you are welcome @uwo
2017:01:06 05:05:52 bbloom one way to do that is to walk the tree and add a key to each node in the tree with the type
2017:01:06 05:06:00 bbloom basically pre-load all the context, so that the spec can be context-free
2017:01:06 05:11:23 bbloom i’m working up an example of that for my own sake - almost ready
2017:01:06 05:16:16 uwo too bad I can’t key a multispec off of a type key in a parent map. Because there are so many (nested) entities on some of the forms, I’d have to annotate each one of them with a dispatch key for the multispec.
2017:01:06 05:16:44 bbloom yeah - that’s what i was saying about spec being context-insensitive
2017:01:06 05:16:54 bbloom it’s also what i was saying about the hack with conformers 😉
2017:01:06 05:16:57 uwo since it’ll be the same key though, I could, like you said, just walk the tree and toss the context around
2017:01:06 05:17:10 uwo (yeah that hack went over my head, sorry 😄 )
2017:01:06 05:18:23 bbloom basically using conformer to automate the annotation
2017:01:06 05:18:27 bbloom it’s a dirty dirty hack 😛
2017:01:06 05:18:40 bbloom conformer lets you change the value that flows through spec
2017:01:06 05:18:51 uwo crazy
2017:01:06 05:19:17 bbloom so you could do something like (s/and (s/conformer (fn [x] (assoc x :context foo))) ::node)
2017:01:06 05:19:22 bbloom now ::node can see :context
2017:01:06 05:28:13 bbloom sending data down the tree
2017:01:06 05:28:24 uwo works with pre/post walk 😄
2017:01:06 05:28:29 bbloom indeed
2017:01:06 05:28:32 joshjones cool
2017:01:06 05:28:48 uwo well thanks again, both! I’m out for the night.
2017:01:06 05:29:11 joshjones one last thought @uwo -- take a step back, and ensure that any added complexity is helpful rather than harmful. nite 🙂
2017:01:06 05:29:40 joshjones as complexity is the enemy of good software, and of clojure itself 😉
2017:01:06 05:29:41 bbloom lol yes, that
2017:01:06 05:29:44 uwo exactly @joshjones
2017:01:06 05:30:03 uwo 👋
2017:01:06 15:08:48 carocad @kenny are you sure that your defspec-test macro works? I tried it with my specs and it always succeeds which is weird because if I use s/check some tests fail 😕
2017:01:06 15:48:09 lmergen could someone elaborate what exactly the difference between :ret and :fn are with fdef ?
2017:01:06 15:48:31 lmergen :fn feels more like a higher level qualifier, while :ret describes a type
2017:01:06 15:48:33 lmergen is that correct ?
2017:01:06 15:50:05 lmergen when i look at the spec documentation, it uses a ranged-rand example, and i see this:
> The :ret spec indicates the return is also an integer. Finally, the :fn spec checks that the return value is >= start and < end.
sounds like the :fn spec passing implies the :ret spec passing ?
2017:01:06 15:50:22 lmergen so then why define :ret as well ?
2017:01:06 15:50:26 alexmiller :ret describes the return value
2017:01:06 15:50:56 alexmiller :fn receives the conformed values of both args and return and can thus validate more complicated relationships between inputs and output
2017:01:06 15:51:15 lmergen aha!
2017:01:06 15:51:32 lmergen ahhh, i see now
2017:01:06 15:51:36 alexmiller In that example, :ret can only say that the return value is an integer, but :fn can say how it relates to the args
2017:01:06 15:51:52 lmergen yes, okay
2017:01:06 15:51:54 lmergen this makes sense
2017:01:06 15:52:00 lmergen and feels extremely powerful
2017:01:06 15:52:04 lmergen thanks alex
2017:01:06 17:06:45 carocad has anyone tested the defspec-test macro that is pinned on this channel? For some reason it always succeeds even if I give it garbage specs 😞
2017:01:06 18:47:32 kenny @carocad It works. Can you give an example?
2017:01:06 19:22:34 carocad @kenny the following should break but it doesnt:
2017:01:06 19:32:24 kenny @carocad The parameters passed to it should be the same as the parameters passed to clojure.spec.test/check -- you need to pass a fully qualified symbol.
2017:01:06 20:10:22 carocad I didnt know there was a gist as well. I added a comment on how to use it for those with not much macro understanding like me 🙂
2017:01:06 20:16:13 schmee what does the retag argument do in multi-spec?
2017:01:06 20:16:17 schmee > retag is used during generation to retag generated values with matching tags. retag can either be a keyword, at which key the dispatch-tag will be assoc'ed, or a fn of generated value and dispatch-tag that should return an appropriately retagged value.
2017:01:06 20:16:58 schmee says the docs, but I think I need an example to understand what that means
2017:01:06 20:18:11 schmee ahh, okay, now I see it 🙂
2017:01:06 21:00:48 carocad @kenny if you dont mind ... could you help me a bit further. I get a weird java.lang.ClassCastException when running lein test. It seems to be a compilation error though 😞
2017:01:06 21:07:29 kenny Try changing failure# to (throw failure#) (line 20 in the gist)
2017:01:06 21:15:10 carocad I still get java.util.concurrent.ExecutionException: java.lang.ClassCastException: clojure.lang.AFunction$1 cannot be cast to clojure.lang.MultiFn, compiling:(clojure/test/check/clojure_test.cljc:95:1)
2017:01:06 21:35:33 alexmiller maybe you’re running into the lein monkeytest problem
2017:01:06 22:12:46 bfabry or maybe I'm misunderstanding what you're saying
2017:01:06 22:14:13 schmee ooooooohhhhh
2017:01:06 22:14:27 schmee I never though about using s/and with s/keys that way 😄
2017:01:06 22:15:06 schmee I’m gonna try that out, thanks!
2017:01:06 22:15:10 bfabry np
2017:01:06 22:16:28 schmee I guess it will be tricky to write generators for stuff like this
2017:01:06 22:16:50 schmee it’s fine if you have one relation like this in the map, but if you have 10 or more...
2017:01:06 22:20:16 bfabry oh totally as soon as you spec something like that you're going to need to do custom generators
2017:01:06 22:21:17 schmee I guess I can write separate generators for the special relations, then generate a sample from the spec itself and merge all the special cases onto that
2017:01:06 22:22:00 schmee that way I can still use most of the default generator and just "add on” the exceptions
2017:01:06 22:22:11 bfabry I don't know much about it sorry. I'm holding out for beta before we switch to 1.9 and start using it in anger. there was a screencast about custom generators though ¯\(ツ)/¯
2017:01:06 22:22:56 schmee once you get in the mindset they’re quite fun to write actually
2017:01:06 22:23:22 schmee it always feels like magic when you get to see it in action 😄
2017:01:06 22:24:47 gfredericks Dependencies in a map can likely be generated by wrapping the map generator in gen/fmap
2017:01:06 23:40:10 crimeminister I was looking for a way to serialize a clojure.spec to return from an API and stumbled across this nifty-looking library: https://github.com/uswitch/speculate
2017:01:06 23:41:08 crimeminister Any idea if similar functionality is likely to ship with spec itself?
2017:01:06 23:53:03 crimeminister Thanks for the link @schmee, it was instructive
2017:01:07 00:37:48 alexmiller @crimeminister: the intention is that s/form will return you something serializable
2017:01:07 00:38:02 alexmiller (As Clojure data)
2017:01:07 00:38:24 alexmiller There are some known bugs with that that are in process
2017:01:07 00:39:02 alexmiller Also there will be specs for spec forms
2017:01:07 00:39:20 alexmiller Which you can conform to retrieve data
2017:01:07 01:06:41 crimeminister Sounds great, thank you @alexmiller
2017:01:07 01:07:45 crimeminister Sounds like that will serve admirably for what I need
2017:01:07 10:21:47 schmee is there a way to access the default gen when overriding a gen in s/spec?
2017:01:07 10:43:07 schmee I guess my question above is still reasonable, but I solved my problem by reading the docs and realising that the retag arg for a multi-spec can be a function
2017:01:07 14:32:34 alexmiller The only way to do it atm is to separate the spec into a var (or different registry spec) so you can refer to it
2017:01:07 15:05:00 stathissideris I’m trying to use :clojure.core.specs/arg-list to parse an arg list, but the clojure.spec/every of :clojure.core.specs/map-bindings means that I can’t fully destructure the arg-list with conform
2017:01:07 15:05:12 stathissideris is there a way to force the every to conform?
2017:01:07 16:50:41 zane Not that I know of, @stathissideris. If you want that behavior you'll probably have to roll your own with clojure.spec/coll-of.
2017:01:07 16:51:39 stathissideris @zane thanks! I was hoping to use the “official” clojure spec to parse the args, but it looks like I’ll have to copy and modify it
2017:01:07 16:52:09 zane That's what it looks like to me.
2017:01:07 16:52:31 zane You could always just validate with clojure.spec/valid? and :clojure.core.specs/arg-list and then write your own transformation function.
2017:01:07 16:55:11 stathissideris my main use case is not validation, I’d like to extract the names of arguments (and be able to handle all cases of destructuring etc)
2017:01:08 12:30:44 luxbock I'm a bit surprised that :clojure.core.specs/seq-binding-form doesn't restrict itself to only acceting vectors, i.e. you can attempt to destructure {(foo) :something} and it fails on because of an assert in clojure.core/destructuring rather than doing so at the spec level
2017:01:08 13:26:16 stathissideris @alexmiller in your recent article on the destructuring spec you mention
> Rather than recursively parsing the binding form, we could simply conform it to receive a more regular structure described in terms of the parts we've defined in the spec.
…but that’s not entirely true because the use of every would prevent conform to recur to deeper levels to find the ::binding-form used within ::map-binding for example. Right?
2017:01:08 15:11:33 lmergen so, i was under the impression that fdef instruments a function and ensures that its args and return values conform to the spec on all invocations, but i think i’m wrong
2017:01:08 15:11:58 lmergen i think i still need to manually ensure conform somewhere, right ?
2017:01:08 15:12:51 lmergen for what it’s worth, i’m porting a schema-based codebase to spec, and want something similar to schema’s ^:always-validate
2017:01:08 15:13:39 lmergen of course i could always use :pre and :post
2017:01:08 15:14:26 mpenet instrument is a dev time thing really (it can trigger gen). To get ret validation you need to call check
2017:01:08 15:15:20 mpenet and yeah, pre/post assertions is the closest to what you want i think
2017:01:08 15:15:36 lmergen yeah but i am one of those people who runs its assertions in production 🙂
2017:01:08 15:18:23 mpenet No, it s not a production usable thing. It ll trigger gen and be super slow/heavy. It doesnt just wrap args with simple validators
2017:01:08 15:18:33 lmergen ok
2017:01:08 15:18:40 lmergen thanks, saves me some headaches 🙂
2017:01:08 15:19:28 mpenet I wish we had another instrument for that personally, but now you have to roll your own basically
2017:01:08 15:20:18 lmergen well, something like schema’s defn overload would be great
2017:01:08 15:20:39 mpenet Yup
2017:01:08 15:30:28 lmergen is anyone aware of some utility that does this ? basically all i want is to call s/explain whenever some pre or post condition is not matched
2017:01:08 15:30:35 lmergen i might make it myself
2017:01:08 15:31:07 lmergen it would make sense to me to have a function like this that has the same signature as fdef
2017:01:08 16:19:39 brabster @lmergen something to make :pre/:post call s/explain sounds useful
2017:01:09 08:12:50 pyr Hi
2017:01:09 09:29:23 pyr does anyone have an example showing how they wired generative tests in their test suite?
2017:01:09 09:29:49 pyr I'm using exercise-fn and instrument at the repl but wanted to know if there was a standard way of wiring that in tests
2017:01:09 09:32:30 mpenet I guess there are a few ways to do this: one would be just to call instrument on "normal" tests, then another could be replacing your test values with generated versions of it, which might or might not be difficult depending on what it is.
2017:01:09 09:33:14 mpenet I haven't done any of it so far personally. Well actually just the instrumentation part on 1 project.
2017:01:09 09:33:44 mpenet clj jdbc and alia both have instrumentation on in the test suite for instance
2017:01:09 09:35:39 pyr cool, will look at that, thanks!
2017:01:09 09:44:18 lmergen so, as an alternative of calling s/explain on all pre or post conditions, what about calling s/assert rather than s/valid ?
2017:01:09 09:45:26 lmergen i know the semantics aren't very clean (:pre is supposed to return true/false rather than an exception), but it does contain a lot more information
2017:01:09 10:36:35 pyr @mpenet Integrating with the output of check works well
2017:01:09 10:36:55 pyr How do you go about generating fixed size arrays quickly, usually?
2017:01:09 12:01:41 pyr indeed, that's much better
2017:01:09 12:02:05 pyr then what you have works out of the box and is much shorter than what I had
2017:01:09 12:02:40 mpenet clojure.spec aliases some of test.check but not all of it
2017:01:09 12:03:10 mpenet I guess the idea is to (maybe) someday promote most of it in spec itself, but I could be wrong
2017:01:09 12:04:10 mpenet test.chuck is also awesome for more gen utils
2017:01:09 12:04:46 mpenet string-from-regex for instance was quite the time saver
2017:01:09 12:19:42 pyr A last question for now, is it possible to specify custom generators only for clojure.spec.test/check
2017:01:09 12:20:23 pyr It's only a matter of visual preference, to be able to have simple specs in my main namespaces and specs with generators for tests
2017:01:09 12:23:30 mpenet Yup, I think so, test/check takes a number of options for this
2017:01:09 12:25:39 gfredericks I think that means you want to use overrides
2017:01:09 12:26:04 pyr @gfredericks yes, but I only see support for those in exercise and exercise-fn
2017:01:09 12:26:31 pyr i'd like c.s.t/check to use them
2017:01:09 12:26:42 pyr wait
2017:01:09 12:26:48 pyr i misread the doc
2017:01:09 12:26:50 pyr sorry
2017:01:09 12:27:17 pyr it seems as though :gen does this for check
2017:01:09 12:54:26 pyr Ah, so the map given to :gen in c.s.t/check is shallow
2017:01:09 12:55:04 pyr i.e: the generator overrides are only taken into account for the called fn. If the fn depends on other functions, they won't use the provided generators
2017:01:09 12:55:33 gfredericks like for stubbing in particular?
2017:01:09 13:03:29 pyr @gfredericks not sure I understand
2017:01:09 13:04:11 gfredericks I just don't know why else, when using c.s.t/check, functions other than the one being tested would use generators
2017:01:09 13:04:48 pyr you're right i'm not articulating my pb properly
2017:01:09 13:05:05 pyr it's when generators rely on other overrided generators that i run into the issue
2017:01:09 14:11:56 pyr I struggled to decouple the generators from the spec because otherwise it makes things harder to follow. My first iteration had with-gen on the specs themselves and that made it much harder to explain what was going on
2017:01:09 14:27:46 lmergen so, if i have a defrecord for which I want to fdef and instrument certain functions defined within that record, how am I supposed to be referring those functions ?
2017:01:09 14:28:05 lmergen (i think this is more related to clojure in general, and what the symbol names for functions inside records are)
2017:01:09 14:29:13 schmee AFAIK all record functions must be specified with protocols, so you can spec the protocol methods
2017:01:09 15:09:38 pyr @lmergen behavior is implemented on top of records through protocols, so as @schmee mentioned, you can specify protocol signatures as you would do functions
2017:01:09 15:18:29 pyr or something to that effect
2017:01:09 15:18:43 pyr unless you're doing something funny with component
2017:01:09 15:20:44 pyr If you are really intent on spec'ing per-type behavior for your protocol, there is another much kludgier way.
extend (https://clojure.github.io/clojure/clojure.core-api.html#clojure.core/extend) takes a map of protocol implementation for types, which takes fns as args. You could implement Lifecycle this way and then spec the individual functions you have in the map
2017:01:09 15:21:09 pyr But again, it seems as though this should not be necessary for Lifecycle
2017:01:09 15:22:20 lmergen ok
2017:01:09 15:22:30 lmergen i understand
2017:01:09 15:22:36 lmergen i'm probably solving this problem at the wrong level
2017:01:09 15:23:07 lmergen and should probably verify whether my entire system satisfies a certain spec, after it has been initialized
2017:01:09 17:05:03 alexmiller I don’t think it was mentioned above, but you cannot spec protocol functions
2017:01:09 17:25:39 schmee why?
2017:01:09 17:36:05 lmergen I suspect that it's an implementation issue, since protocol functions have very different function signatures
2017:01:09 17:38:31 hiredman and previous comments have some discussion of why that is
2017:01:09 17:39:29 hiredman basically certain kinds of "function" calls(including protocols) are compiled differently and spec just handles the generic case
2017:01:09 17:39:36 schmee what the recommended workaround?
2017:01:09 17:39:46 schmee wrap it in a regular function and spec that instead?
2017:01:09 17:40:51 lmergen that's what I ended up doing yes
2017:01:09 21:14:58 olivergeorge Am I missing a cleaner way to write this. Goal is to check that the options list has the required keys which are also declared in the data)
2017:01:09 21:40:19 joshjones @olivergeorge so ::Select.props describes a map, one of whose keys is :options, which is a list of maps, each of which must contain values referred to by keys named :value-key or :display-key, which are in ::Select.props -- is that right?
2017:01:09 22:22:10 olivergeorge Correct
2017:01:09 22:23:13 olivergeorge So I really want to confirm the options based on the map data
2017:01:09 22:23:28 olivergeorge Like this movie inception
2017:01:09 22:24:31 carocad @kenny I gave it a try at simplifying your macro and came up with this:
It can be simplified even more but I like the result so far. Hopefully it can be useful to somebody 🙂
2017:01:09 22:27:24 kenny You want to use do-report, not println otherwise you'll break integrations with clojure.test
2017:01:09 22:29:06 carocad why? I dont know much about clojure.test actually 😕. I figured that the call to t/is already took care of the integration. doesnt it?
2017:01:09 22:29:32 kenny The other important difference is that this is counting every single generative test instead of grouping them as one
2017:01:09 22:30:56 kenny is will handle it but you will get confusing results for when tests fail
2017:01:09 22:41:11 carocad ah I see what you mean. true the output is not as clean as with a normal test since you get the generated symbol but I still find the output better since in the previos macro I was only getting the stacktrace of an error. No idea which function failed nor other info. I dont know if it was a problem in my project though
2017:01:09 22:56:23 joshjones @olivergeorge unfortunately, I think what you have is about as straightforward as you can get with what you want. While spec can nest sequential and associative structures easily, there is not another way (that I'm aware of) to relate various levels other than another predicate which is what you've done. If anything, I might combine the two additional predicates into one function, though you may find it gives you an undesirable level of detail:
2017:01:10 06:22:08 luxbock if I want to use spec for runtime validation, in a situation where my functions accept a large map but only care about a few keys, I should probably create my own validation function rather than using s/valid? because it validates all of the keys in the map even if I'm looking to only validate a subset of them, right?
2017:01:10 10:11:46 mpenet @luxbock I don't think so, it will just check what's in (s/keys ...) since map specs are "open" (can contain extra keys) there's no need to check all the entries.
2017:01:10 10:12:11 mpenet At least that's what I assume given how it's defined
2017:01:10 10:14:03 mpenet if you use map-of, it's another story
2017:01:10 10:21:24 mpenet well that sucks, I wonder why it does this
2017:01:10 10:25:07 mpenet @alexmiller any reason why this should happen?
2017:01:10 10:26:41 mpenet
In addition, the values of *all* namespace-qualified keys will be validated
(and possibly destructured) by any registered specs. Note: there is
no support for inline value specification, by design.
from the doc of s/keys
2017:01:10 10:26:47 mpenet anyway, that's kind of odd ...
2017:01:10 10:32:44 mpenet so using non ns keys will not trigger this
2017:01:10 12:41:45 gfredericks currently test.check has a bunch of assertions sprinkled around for checking that the args to particular functions are generators
2017:01:10 12:42:27 gfredericks which is something that could be somewhat accomplished with spec
2017:01:10 12:43:31 gfredericks so I'm wondering if it's worthwhile to remove those assertions and just have specs; the downside being that error messages are worse when you haven't instrumented clojure.test.check.generators, which might not be something that users would naturally do
2017:01:10 12:43:58 gfredericks should library authors expect that users will turn on instrumentation everywhere to debug a problem, and therefore do less runtime validation than they would otherwise do?
2017:01:10 13:46:27 alexmiller @mpenet the whole idea behind s/keys is to register your attributes with known semantics and have them checked throughout your system. By having it validate all keys, your spec grows automatically with your system. If you want to narrow the scope, you can select-keys on the map before validating
2017:01:10 13:47:38 alexmiller @gfredericks: I'm a little worried about the circularity issues of having specs in test.check but maybe it's fine with the dynamic loading gap
2017:01:10 13:49:12 alexmiller I'd say you shouldn't rely on instrumentation in cases where you always want those checks (and this might be one of those places)
2017:01:10 13:49:46 mpenet it's a bit of implicit vs explicit. So if you use ns keys you can (not that it's a good idea) just use (s/keys :req []) for your map validation
2017:01:10 13:50:01 mpenet I personally find that odd, but I can live with it
2017:01:10 13:50:51 alexmiller You don't even need the :req there, just (s/keys) is sufficient
2017:01:10 13:50:59 mpenet right
2017:01:10 14:28:59 gfredericks @alexmiller I'm pretty sure putting specs in test.check is as feasible as putting them in clojure.core 😉
2017:01:10 14:29:33 gfredericks I can at least run the test.check test suite with instrumentation
2017:01:10 15:26:16 tjtolton This channel needs a "quicklinks" section for newbs like me. I'm looking for materials on integrating core.spec test.check... can anyone offer me any leads? Is the core.spec workshop from the conj online anywhere?
2017:01:10 18:01:57 crimeminister Trying to figure out what the idiomatic way to validate something while "transforming" it, i.e. validate a string using a regex, and then checking that the capture groups have the required values using (spec/cat)
2017:01:10 18:06:25 joshjones are you able to work with actual instant objects, or must it be strings?
2017:01:10 18:07:38 crimeminister Strings would be ideal in that I could avoid a transformation step, but if that would make life easier I'd try it out
2017:01:10 18:17:04 joshjones is your goal to just get validation of dates/times, or are you looking to leverage spec to generate data, etc.?
2017:01:10 18:20:31 crimeminister Primary goal is to get validation, but if I could get the latter as well I'd be pretty happy
2017:01:10 18:24:02 crimeminister If need be I could write a little transformer fn that turned my string data into, say, a map for easy validation, and another transform the map value into a string to assist with data generation
2017:01:10 18:24:55 crimeminister I just worry that it introduces an extra step each time I make a (spec/…) call
2017:01:10 18:28:27 crimeminister Another (bad?) option might be some macro magic to allow spec to take a :pre-transform hook
2017:01:10 18:28:57 joshjones I think what you have now is headed in the right direction. With things like date/time, I always long to use existing libraries. The clojure.instant library (specifically the read-instant-timestamp might be of use) can be used to parse a string into a date/time.
for data generation, spec provides inst-in to generate dates within a range.
2017:01:10 18:30:27 crimeminister Thanks for the suggestion @joshjones. I think I need to take that idea away and hammock on it for a few 🙂
2017:01:10 18:38:55 seancorfield What are folks doing for atoms in terms of spec? I thought I read about an atom-of function but can’t find any context for that now.
2017:01:10 18:39:48 seancorfield (I recall the “don’t spec atoms, only spec their dereferenced values” advice but that doesn’t help when I have a map that contains two atoms 🙂 )
2017:01:10 18:44:26 alexmiller It's possible that we might add something like atom-of
2017:01:10 22:35:32 tjtolton err, okay, it seems kind of weird
2017:01:10 22:36:03 tjtolton its specifying that the return value is not just a boolean, but specifically that the return is true
2017:01:10 22:36:42 tjtolton but how can that be guaranteed since the input is generated randomly based on the spec?
2017:01:10 22:40:01 schmee tjtolton the spec has a bunch of custom generators supplied through gen-overrides
2017:01:10 22:40:45 schmee which ensures that only “valid” domain values get generated
2017:01:10 22:44:50 tjtolton huh. Isn't that beyond the purview of a schema or function spec? Ensuring that data is not only in a given form, but correct according to application logic?
2017:01:10 22:45:16 tjtolton Maybe I just need to read that a bit more carefully
2017:01:10 22:45:29 gfredericks if it's just for testing, then that's not really part of the spec
2017:01:10 22:45:48 gfredericks you might want your generators to be rather more specific/fancy than the specs
2017:01:10 22:46:14 gfredericks more or less because of what you just said
2017:01:10 23:29:22 bbloom heh, i just found a pretty interesting spec bug....
2017:01:10 23:29:49 bbloom apparently if you define a spec for ::keys, it interferes with destructuring’s spec
2017:01:11 00:05:44 cap10morgan or is there an updated way to integrate clojure.spec generative tests into your lein test runs?
2017:01:11 00:13:45 uwo @alexmiller has there been any consideration put toward separating required semantics from s/keys, and creating another function s/required?
2017:01:11 00:58:51 alexmiller No
2017:01:11 09:32:48 tap I have a spec which uses s/fspec (`:args` specified). As you know, this spec requires test.check. Is there a way to temporary skip validating s/fspec so that I can execute s/valid? on this spec at runtime?
2017:01:11 13:12:12 alexmiller You can instrument and supply a spec override to ifn?
2017:01:11 14:02:26 mpenet almost suggested the xform thing, but I am not sure it will work with the advanced stuff (instrumentation)
2017:01:11 14:04:40 cgrand which interaction do you see between a channel and an instrumented defn?
2017:01:11 14:09:10 tjtolton so, perhaps an odd question, but is there a clojure 1.9 build that is production ready? I want to use spec, but being the guy that insists we put an incomplete version of clojure in prod is a bit nerve racking
2017:01:11 14:09:14 mpenet arg gen triggered by fdef for instance
2017:01:11 14:09:49 mpenet so far it has been one of the thing I have found quite confusing/surprising
2017:01:11 14:14:27 mpenet I guess as long as the fn is pure it's ok
2017:01:11 14:14:34 cgrand @mpenet: I see: not instrumentation but validation of a fspec
2017:01:11 14:16:10 mpenet I mixed fdef and fspec earlier tho
2017:01:11 14:16:32 schmee tjtolton clojure 1.9 is still alpha, no word on when the official release will be out yet
2017:01:11 14:18:24 tjtolton gotcha. so, officially, no guarantees on spec in production
2017:01:11 14:18:25 tjtolton but
2017:01:11 14:18:28 tjtolton unofficially
2017:01:11 14:19:30 tjtolton is it a good idea for me to try to get spec into our code now? or should I wait, and just keep using spec in toy projects until the official release
2017:01:11 14:59:13 alexmiller That’s a judgement call only you can make. spec is still alpha and subject to change. We are aware of the desire from many directions to have a 1.9 (non-alpha) with spec available.
2017:01:11 15:02:24 pyr @cgrand yes, it's an option. I was interested in specing promise-chans which should return a specific type of value.
2017:01:11 16:24:06 lmergen @tjtolton we're running spec in production. haven't found any problems, but YMMV
2017:01:11 16:24:37 lmergen (we're also running aleph and manifold alpha's in production, so there's that)
2017:01:11 16:25:48 tjtolton @lmergen cool. I'm going to basically hook up 1.9-alpha and spec to the service that I'm the owner of.
2017:01:11 16:26:16 tjtolton If it breaks, it will be my job to fix it, which I'm cool with.
2017:01:11 16:34:48 lmergen i hope you're not in charge of the authorization service then 😉
2017:01:11 16:44:48 seancorfield @tjtolton: we've been running Clojure 1.9 in production for months with no problems. We've always run alpha and beta builds in production as they've appeared in order to leverage new features early. We first went to production on Clojure 1.3 alpha 7 or 8 back in 2011.
2017:01:11 16:46:20 tjtolton good thing clojure is stable as hell
2017:01:11 16:46:33 tjtolton God I love this language culture.
2017:01:11 18:10:39 seancorfield I think in nearly seven years we’ve seen just two builds with production-affecting bugs. We skipped one of those (just a matter of timing, due to when our production builds fell, either side of that release), and the other one was fixed quickly enough that we just pushed a new production build with the updated Clojure JAR.
2017:01:11 18:25:41 joshjones Like every single thing in life, it comes down to a simple ratio of risk:reward.
If the code won’t run on 1.8, and if the consequences of delaying a production release are worse than the consequences of a production product exploding (i.e., first to get to market on something may be most important than it actually working well), then it might make sense to use a 1.9 alpha in production. Also, if the user impact is low (i.e., this isn’t a mission critical application and several hours/days of downtime won’t kill anyone) then it might also make sense.
However, as a general process, it just doesn’t make sense to run alpha in production. The liability is all on you. I prefer to shift liability elsewhere when it’s pragmatic to do so, and when the clojure core code is considered “production ready” and it fails, the liability has at least partially shifted away from you as the developer, and onto the clojure core team. For products with relatively low use, I’d run an early alpha in production since the track record seems to be good. But for a service which receives several thousands of requests per second, around the clock, and upon which many millions of dollars hang — for us personally, being able to run spec isn’t worth that risk, i.e., we can wait a few months. Each developer and team has to make the call and accept the consequences 🙂
2017:01:11 19:52:42 bbrinck Is it a design goal of clojure.spec (and the core specs in particular) that conform/unform should always round-trip? For instance, the built-in spec for defn-args doesn’t round-trip in at least one case (more details http://blog.klipse.tech/clojure/2016/10/10/defn-args.html) - is that considered a bug?
2017:01:11 19:54:57 bbrinck The reason I ask is that we’re experimenting with using the conform+modify+unform pattern for modifying data, and just wondering if the core specs will be written to support that
2017:01:11 20:37:40 seancorfield There are currently some known bugs in unform based on discussions I’ve seen here.
2017:01:11 20:57:26 mikerod it seems like having unform be able to roundtrip successfully in all cases is a tricky situation
2017:01:11 20:59:50 bbrinck @mikerod can you talk a little more about that? “tricky” as in there’s going to be a long-tail of bugs for this? or “tricky” as in “it’s not realistic to treat every case as a bug”? or something else?
2017:01:11 21:00:18 mikerod @bbrinck oh, I’m just hypothesizing and considering how I’ve seen the topic come up quite a few times.
2017:01:11 21:00:48 mikerod I actually don’t know. I do hope that it is meant to be able to do the type of roundtripping you are going for in you defn-args article though
2017:01:11 21:00:49 bbrinck @mikerod ah, i see. for now, we just ended up writing our own spec for this case
2017:01:11 21:01:01 mikerod I’m definitely not going to give any definitive say on it though 😛
2017:01:11 21:01:06 bbrinck 🙂
2017:01:11 21:01:10 mikerod Yeah, I saw you mentioned adding a conformer
2017:01:11 21:01:30 bbrinck Oh, to be clear, I didn’t write that post - we just used it as a starting point for writing our macro
2017:01:11 21:01:40 bbrinck but it’s a good explanation of the issues we ran into
2017:01:11 21:02:17 mikerod ah, I misread that part
2017:01:11 21:04:21 alexmiller yes, conform -> unform should roundtrip (with a few caveats)
2017:01:11 21:10:16 alexmiller and then I'd say one area that could benefit from something additional is specifically for specs that are using regexes in a vector - some of the core macro specs suffer from this and unform to a list rather than a vector. that’s a common and important case and I think we need something specific for it (whether that’s creating vcat or something else is TBD)
2017:01:11 21:11:21 alexmiller that’s the case mentioned at the top of this conversation
2017:01:11 21:14:20 bbrinck OK, that’s very helpful. Thanks!
2017:01:11 21:15:03 alexmiller if you find something else, feel free to file a bug
2017:01:11 21:16:56 mikerod thanks!
2017:01:11 21:17:03 mikerod for the details
2017:01:11 23:51:04 rickmoynihan @tjtolton: Another option is to run clojure 1.9alphas in dev/test profile's only and have a policy to avoid including 1.9 features in the production environment.... you can set this up with leiningen profiles pretty easily.
This way you could e.g. write specs in your unit tests, but not put them in the production code itself, keeping it on 1.8 … when spec is finally released you could then move some of the specs out of the tests into the production code. You could also switch to 1.9 whenever you want and know you can switch back if it’s not good. Obviously it requires some discipline, and might mean you can’t test code in the production env; so there’s still some significant risk - and you might also want to separate the 1.9 tests into a separate profile.
Anyway it’s just one possibility I’ve been considering
2017:01:11 23:53:34 rickmoynihan The core team are incredible at keeping clojure stable though - but it’s alpha for a reason, so we should respect that they may want to change their minds and make significant changes to 1.9 code at any point.
2017:01:11 23:59:36 seancorfield I’d add the caution that there are quite a lot of oh-so-convenient new predicate functions in 1.9 that you’ll find hard to resist using in your code and then you’ll be unable to run on 1.8. You’ll also be forced into keeping all the spec stuff in separate namespaces so your non-test code will run on 1.8.
2017:01:12 00:00:21 seancorfield That said, clojure.java.jdbc currently takes that approach (specs in separate namespaces) so it can run on 1.4 thru 1.8, and it optionally uses spec for the tests if they are run on 1.9.
2017:01:12 00:00:59 seancorfield Another option would be to look at the backport of spec to 1.8 that someone in the community maintains (future spec, I think?).
2017:01:12 00:56:01 tjtolton That's a cool idea, thanks!
2017:01:12 16:35:27 actsasgeek I have a philosophical (?) question about how far spec should be pushed in terms of semantic types, type checking and testing. For example, is it a reasonable use of spec to specify that the list argument to a binary search is sorted? Can you do it?
2017:01:12 16:37:55 williewillus it's certainly possible
2017:01:12 16:39:02 williewillus that said I'm also interested in exactly where it should be applied, and if "overuse" of it is possible, and what that would look like
2017:01:12 16:46:22 actsasgeek well, it’s funny you say that because that was my leading question...
2017:01:12 16:46:46 actsasgeek one of the things that has been mentioned is the separation of specifying the keys in a map and the values those keys may take.
2017:01:12 16:49:04 actsasgeek but it is very often the case in complex data structures that those values are interdependent. For example, is it too far to require a ::street to actually be in the ::city in the ::state with the right ::zip-code? Or, perhaps midway between the two, if I’m modeling a “blocks world” using blocks with the keys ::id ::pile ::above ::below, it’s certainly true that a block cannot be above itself, below itself or both above and below the same block. And so I find myself writing block? instead of (s/def ::block (s/map …).
2017:01:12 17:04:42 joshjones It is very true that certain “schema” is expressed via the relationship of one value to the other in the entity map. Spec still allows us to express this relationship via a custom predicate, but the nice part is that it does not force us to define this structure. Instead, its intention is for us to specify reusable pieces where possible.
How far we take it is largely up to us. In the example of city/zip, personally this feels as if it’s overstepping the intended bounds of spec, but I cannot quantify that yet with anything sensical so it’s not worth much.
2017:01:12 17:10:30 actsasgeek yeah, I’m just trying to think out loud a bit. One of the points stressed in spec is the generation of examples…but automatically generated samples of addresses that are just random strings seems…sad to me.
2017:01:12 17:11:31 schmee why?
2017:01:12 17:12:32 actsasgeek because if I want to use this as documentation and to specify what would be acceptable input, a ::street of “V+q.29b7gqao9$” seems strange.
2017:01:12 17:12:42 joshjones that’s what custom generators are for...
2017:01:12 17:13:05 schmee there are a lot of street names out there 😛
2017:01:12 17:14:20 actsasgeek yeah, I watched that…I would like to see more on that topic…it seems like a buried but important part of the overall usefulness of spec.
2017:01:12 17:14:45 actsasgeek after watching that, I’m still not entirely clear what a model is.
2017:01:12 17:15:01 schmee I don’t see what is wrong with random data unless you depend on a certain structure for the string in your code
2017:01:12 17:15:09 joshjones I think it’s only buried because it’s still in alpha and most people don’t “get spec” yet even for basic cases, and need to build a level of understanding that can support that level of detail
2017:01:12 17:15:17 actsasgeek and that’s fine @schmee. I do.
2017:01:12 17:15:46 schmee ahh, then custom generators are the way to go 🙂
2017:01:12 17:16:55 actsasgeek I think that video above and the one before it illustrate the point to some degree. I was watching the testing video, and kept going, “but you aren’t actually testing the function…the odds of generating a random string that is a substring of another random string are very low”.
2017:01:12 17:17:37 actsasgeek but even at the end, if my-index-of was an actual function, there’s absolutely nothing in there that tests if the index returned is the correct one.
2017:01:12 17:19:08 joshjones gotta lunch but would love to discuss after i eat 😉 you have a great topic and is at the heart of spec and how it can be useful to everyone
2017:01:12 17:19:21 actsasgeek if think if I replaced the use of string/index with 1, I think all the tests would pass.
2017:01:12 17:21:33 schmee I don’t see how you could test that in a generative fashion without re-implementing the function itself
2017:01:12 17:21:47 schmee I suppose this is where example-based testing as a complement comes in
2017:01:12 17:22:22 schmee or generative testing as a complement to example-based testing, if you prefer
2017:01:12 17:43:27 actsasgeek @schmee I think I agree. My concern would be that in the video (the 2nd one of 3 in the series on Testing), Stu specifically contrasts the generative approach with the assert approach where you have to do everything yourself.
2017:01:12 17:44:57 actsasgeek Now, one interesting thing is that the custom generator contains the information (implicitly) that you need. The generator takes a prefix, match and suffix to generate items that will definitely have a match. If you could save the length of the prefix, you could verify that my-index-of was working correctly.
2017:01:12 17:46:24 schmee you could return a tuple of [(count prefix), rest of the stuff] and use that in your tests
2017:01:12 17:53:49 actsasgeek Hmm, doesn’t what is generated have to be arguments to the function under test?
2017:01:12 17:55:48 schmee I’m not sure, but either way you can still write tests that use generators without spec and do whatever you want 🙂
2017:01:12 17:55:50 actsasgeek I think a subtle related problem is that the :ret is specified as <= the length of the string but it is < length of the string because of zero indexing.
2017:01:12 17:56:32 actsasgeek well, you can always do what you want 😛
2017:01:12 17:57:08 schmee haha, yeah
2017:01:12 17:58:06 schmee but I mean, it’s cool that spec can auto-generate tests and all, but if you have a very specific (generative) test you want to make it might be better to just write that the old-fashioned way and not try too hard to cram it into a spec-based approach
2017:01:12 18:01:56 joshjones @actsasgeek which video are you talking about, with my-index-of?
2017:01:12 18:02:27 actsasgeek you posted a video on custom generators. It was 3rd in a series of 3.
2017:01:12 18:04:07 joshjones I have not watched it, I’m looking at it now
2017:01:12 18:06:53 actsasgeek you can safely watch it a 1.25 speed 🙂
2017:01:12 18:14:01 actsasgeek sorry, I have to pair for a bit so I’ll be away from this conversation but I am interested in continuing it.
2017:01:12 19:30:33 tjtolton So, the docs for spec.test/instrument seem kind of circular:
> Instruments the vars named by sym-or-syms, a symbol or collection
of symbols, or all instrumentable vars if sym-or-syms is not
specified.
Is there a decent instrument lesson of some sort out there?
2017:01:12 20:08:44 joshjones https://clojurians.slack.com/archives/clojure-spec/p1484243750006346
@actsasgeek Regarding this, the index-of “” within “” is 0, so in this one case the length of the return is equal to the length of the source. edge case though, and you could account for this in the :fn portion of the spec
So to continue the earlier discussion: in order to test something, we have to have something that is known, correct, reliable, etc., to test against. In the case in the videos for index-of, it was the “eye test” — make some sample data, manually count the characters, see if the function spits out what we determine as the “correct” value.
The :fn spec can be made more robust (I’ll post an example later), but imagine we’re just spec’ing substring, determining whether a string is actually a substring of another. Well, this is not so easily spec’d, because that’s essentially what this function does! So, if you had another “source of truth” that could definitively tell you that, well, you’d just use that function. Even when you write your tests manually, you’re still placing your trust in your ability to accurately type strings correctly. There is always room for error in the testing phase, and you can’t get around it.
Perhaps the answer is to hand write your own tests as usual, and it may make sense to do that. But one takeaway is that going through this process is likely to find edge cases that your own manual testing would NOT have found, even if it does not have the ability to determine with 100% certainty that your function is correct. (In the case of the video though, I think it can be made more rigorous by altering the :fn)
2017:01:12 20:33:34 actsasgeek To the specific point, I was a little surprised that (index-of “abcd” “”) even works. It feels like an overly mathematical definition of “index of” because (index-of “abcd” “a”) returns the same result…but that’s neither here nor there.
2017:01:12 20:36:35 actsasgeek To the general point, I think I agree. My point was that in the 2nd video, it seemed to me at any rate, that generative testing was presented as an alternative to manual testing. Just in the way Stu describes how “difficult” that coming up with the manual test was. I completely agree on the point that generative testing is much, much better at finding the edge cases. In a way, I’ve always thought of manual testing as a way of preserving REPL development of the shape of the function and documentation.
2017:01:12 20:37:59 actsasgeek But documentation, examples, etc., are being touted as leverage points of spec and it seems to me that these are going to require more custom generators than the relative abundance of documentation for making customer generators would suggest. I could be wrong, tho.
2017:01:12 20:41:22 actsasgeek but then I think that the opposite of conform should be deform and not unform. 😉
2017:01:13 09:48:38 pyr Hi spec'ers
2017:01:13 09:48:46 pyr I'm up against a plumbing issue it seems
2017:01:13 09:49:35 pyr I wrote a simple function to run generated tests in a test namespace:
2017:01:13 15:48:43 triss ok, think i might swap the fspecs out for fn?for what I’m doing
2017:01:13 15:49:14 triss setting up generators for js objects of specific types seems fiddily
2017:01:13 16:23:28 gfredericks it's not hard if they can be constructed easily from data
2017:01:13 16:23:40 gfredericks e.g., with gen/fmap
2017:01:13 21:11:56 eggsyntax I want to make a map spec, but with a custom generator that generates it with an extra key (so eg the spec might be for a map like {::a 1 ::b 2}, but when I generate one I want to generate it like {::a 1 ::b 2 ::c 3}). Right now I'm mucking around with creating a spec for the fuller map including ::c, pulling the generator from that, and using that generator in a with-gen (and I haven't even quite got that working yet). It just seems excessively ugly -- anyone see a better way?
2017:01:13 21:13:42 eggsyntax I could create the generator essentially manually, generating {::a (gen/generator (s/gen ::a)) ...} but that seems ugly too, especially because I'm creating these specs dynamically on the fly, so it'd result in awfully complicated code.
2017:01:13 21:14:51 alexmiller I’d probably do your first idea with grabbing the gen from the fuller spec - why is it looking ugly?
2017:01:13 21:15:24 alexmiller if it’s repetition, you might try s/merge of the original spec with an additional map spec, then grabbing the gen from that
2017:01:13 21:16:15 eggsyntax Oh, that's exactly the elegant sort of solution I was looking for 🙂. I think this'll be my first time actually using s/merge. Thanks, Alex!
2017:01:13 21:16:35 bbloom merge is kinda awesome
2017:01:13 21:16:44 eggsyntax The ugliness is just yeah, the repetition & all the lines of code, and creating a spec only to throw it away.
2017:01:13 21:16:53 bbloom way nicer than typical inheritance schemes
2017:01:13 21:21:49 alexmiller to get in the spec mindset… maps are sets of attributes. merge is the union.
2017:01:13 21:22:36 bbloom @alexmiller i found union and intersection to be very confusing terms when talking about “semantic subtyping” and open maps - the problem is that there is a subtle distinction between the union of keys and the union of the sets of values that match the specification
2017:01:13 21:22:46 bbloom and they have opposite implications
2017:01:13 21:22:55 bbloom if you union the keys, you shrink the set of possible values
2017:01:13 21:22:57 alexmiller yeah, don’t think about it that hard :)
2017:01:13 21:23:25 bbloom i only share this information with you b/c i’ve seen the same confusion about the words “union” and “intersection” affect people working with TypeScript and Flow
2017:01:13 21:23:40 bbloom that’s why i’m glad the name chosen was “merge"
2017:01:13 21:30:23 eggsyntax Hmm. Now that I try to think it through further, I'm less sure I see how to use s/merge to make it better. Are you envisioning something like this?
That seems like it would require creating three specs in total instead of two...
2017:01:13 21:30:42 eggsyntax But it's very possible I've just gotten muddled trying to think it through 😉
2017:01:13 21:31:09 eggsyntax (where ::extra-key-spec is a spec for a map containing only the extra key)
2017:01:13 21:37:37 eggsyntax Oh, wait, I can use s/spec to make most of that implicit, I think.
2017:01:13 21:37:51 eggsyntax Duh 😉
2017:01:13 21:41:34 rickmoynihan I have a function in some legacy code that I want to write an example based clojure.test test for… it returns a vector of various types and I thought I could write a spec for it. In an is definition the granularity of failure is just a boolean… so I could extend clojure.test to display the explain data (I’ve experimented with this once before). I did think I could instrument it with an fdef but obviously that only works for input values.
Anyway just wondering if people are currently using spec for example based testing
2017:01:13 21:41:45 rickmoynihan (I realise it’s for generative testing)
2017:01:13 22:09:08 eggsyntax Damn, I'm getting close, but I keep tripping over java.lang.IllegalArgumentException: No matching ctor found for class clojure.test.check.generators$such_that$fn__32821 when I try to eval the s/def (which I have to do because this is all getting defined at runtime)
2017:01:13 22:12:02 eggsyntax I can just do the s/def in the REPL and it works properly. And the eval was working until I started trying to add this generator, so I'm screwing something up...
2017:01:13 22:23:35 tjtolton is there some kind of configuration I need to set up to make spec.test/check work?
2017:01:13 22:41:28 eggsyntax Having just gotten some help myself, I wish I could help either of the people who just asked questions, but unfortunately I don't have an answer to either of those 😕
2017:01:13 22:49:01 rickmoynihan tjtolton: did you try with a back-quote?
2017:01:14 00:21:10 richiardiandrea so let's say I am enriching a data structure with chained transformations (`map`-ing or specter-ing)...and I want to spec every substep. For instance the first step has {:family {:name "bla"}} the second step will have {:family {:name "bla" :id :sda}}.
2017:01:14 00:22:18 richiardiandrea how can I spec the second step after having already spec-ed the first as:
2017:01:14 00:23:53 richiardiandrea now it should be ok
2017:01:14 00:24:52 richiardiandrea so basically I'd need to add another spec adding :family/id but I can't see how to do it without overriding the :family/family spec
2017:01:14 00:26:07 richiardiandrea ok I could change the namespace probably
2017:01:14 00:26:16 eggsyntax Can you s/merge the :family/family spec with a new one which only specifies the new element?
2017:01:14 00:26:48 richiardiandrea maybe 😄 let me try that
2017:01:14 00:27:23 richiardiandrea but then I still need to change the namespace in the register
2017:01:14 00:29:48 eggsyntax You could eval the s/def with a modified name…although having been doing a lot of it, I’ll warn you that that’s a route that can get reeeaaally sticky pretty fast. & there seem to be issues with evaling s/merge (that’s where I stopped today, had to call a halt. In fact, crap, what am I doing here? Going AFK immediately 😉 )
2017:01:14 00:35:06 richiardiandrea it works, but now for each transformation I need:
2017:01:15 18:11:51 bbloom @rickmoynihan that’s a decent hint to programmers, but it’s not quite good enough semantics
2017:01:15 18:12:11 bbloom for example, a generator will produce arbitrary values there, which is clearly wrong
2017:01:15 18:12:44 bbloom oh well actually maybe spec isn’t aware of aliasing and you’ll get an error
2017:01:15 18:12:46 bbloom let me see...
2017:01:15 18:14:30 bbloom this works: (def unspeced (fn [x] true))
2017:01:15 18:14:39 bbloom breaks the generator, which is what i’d expect
2017:01:15 21:27:19 dottedmag Is there a way to drill into specs from REPL? Say, an output from (doc ns) gives args: (cat :name simple-symbol? :docstring (? string?) :attr-map (? map?) :clauses :clojure.core.specs/ns-clauses), how do I see the definition of :clojure.core.specs/ns-clauses spec?
2017:01:15 21:28:28 dottedmag Oh, (doc :clojure.core.specs/ns-clauses) works 😄
2017:01:15 21:36:23 joshjones this is effectively the same as (s/describe :clojure.core.specs/ns-clauses) fyi
2017:01:15 22:47:54 bbloom some specs seem to resolve eagerly, some lazily - is there some way to force laziness for the sake of forward references?
2017:01:15 22:48:04 bbloom ie if i do (s/def ::foo ::bar) that fails
2017:01:15 22:48:12 bbloom but (s/def ::foo (s/and ::bar) works
2017:01:15 22:48:21 bbloom which seems like a quirky way to accomplish this....
2017:01:15 22:49:02 bbloom i was hoping s/spec would do it, but sadly it resolves eagerly
2017:01:15 23:35:58 rickmoynihan > that’s a decent hint to programmers, but it’s not quite good enough semantics for example, a generator will produce arbitrary values there,
@bbloom: What else could a generator do though? As far as I can see it can only explicitly fail or generate a likely incorrect value
2017:01:15 23:36:16 bbloom i’d want it to fail
2017:01:15 23:37:06 bbloom i kinda also want to be able to test things w/ incomplete specs - like if i have (s/or :foo string? :bar some-crazy-thing?) i should at least be able to test the :foo path, even if :bar isn’t tested
2017:01:15 23:37:08 bbloom although i’d expect a warning
2017:01:15 23:37:47 rickmoynihan bbloom: then maybe just use the generator (fn [] (throw (ex-info “Unspeced function”))
2017:01:15 23:39:01 bbloom that solves the conform/instrument case (modulo fspec) but not quite the partial generator case - which i guess would require some changes to the generator monad thing? i dunno if it supports filtering
2017:01:15 23:39:20 bbloom generator would be like (fn [] (warn “unspeced function”) (skip))
2017:01:15 23:40:07 rickmoynihan yeah - was just thinking you want them to flow
2017:01:15 23:41:50 rickmoynihan it seems you want something a little like a prolog cut
2017:01:15 23:42:12 bbloom a little 😉
2017:01:15 23:42:40 rickmoynihan yeah might be the opposite of that 🙂
2017:01:15 23:43:09 bbloom i haven’t looked too deeply at writing generators yet, so i don’t know what the monad thing is about - i imagine it’s like mfilter or something
2017:01:15 23:43:32 rickmoynihan its not something I’ve looked into
2017:01:15 23:44:15 bbloom ah yeah, it’s “MonadPlus” - basically you just lift map in to mapcat
2017:01:15 23:44:30 bbloom ie you had map x -> y, now you have map x -> [y] and then you can filter by doing map x -> []
2017:01:15 23:44:37 bbloom i dunno if the generators support that or not 🙂
2017:01:15 23:44:42 bbloom i assume they must
2017:01:15 23:45:29 bbloom since they are generating multiple possible things
2017:01:15 23:45:54 bbloom but they might expect exaclty one item at a time, not zero-or-one item
2017:01:15 23:46:34 bbloom MonadPlus provides the “zero” value, ie [] for the list monad or just an empty stream of generated data
2017:01:15 23:50:37 rickmoynihan I’m still trying to get my head around exactly what being a partial generator means… presumably it’s just something like (s/or :walk gen-me :don-walk unspecced-gen-ignore-me)
2017:01:15 23:51:03 bbloom well, you’d walk it, but emit a warning and then not actually generate anything
2017:01:15 23:51:04 rickmoynihan (semantically - rather than in practice)
2017:01:15 23:51:07 bbloom yeah
2017:01:15 23:51:53 rickmoynihan why walk it? rather than ignore it?
2017:01:15 23:52:15 bbloom i guess that’s the same thing
2017:01:15 23:52:52 gfredericks gen/such-that is the monad-plus part
2017:01:15 23:53:14 bbloom is the warn & skip idea doable?
2017:01:15 23:53:27 gfredericks but it's not really monad-plus since it will just give up if it can't get the pred to pass
2017:01:15 23:54:05 gfredericks @bbloom warn and skip just applies to s/or specs?
2017:01:15 23:54:31 bbloom that’s where it would be most useful, but i don’t see why it wouldn’t apply to any spec… for example:
2017:01:15 23:54:48 bbloom (s/exercise unspeced)
2017:01:15 23:55:09 bbloom that’d just print something like “skipping unspeced at []"
2017:01:15 23:55:25 bbloom something like that at least
2017:01:15 23:55:58 bbloom for s/or specs it would be much more useful b/c i could run partial tests
2017:01:15 23:57:52 rickmoynihan can’t a spec be included anywhere though? And anywhere you include an unspeced spec wouldn’t it imply the specs had to effectively be rewritten to use an s/or over it?
2017:01:15 23:58:48 bbloom yeah - specs are implicitly a big and/or tree
2017:01:15 23:59:00 rickmoynihan precisely
2017:01:15 23:59:27 rickmoynihan but just thinking the s/and with an unspec in implicitly also becomes an unspec
2017:01:15 23:59:44 bbloom hhmm
2017:01:15 23:59:48 rickmoynihan i.e. its contagious like a type
2017:01:16 00:00:06 bbloom nah, i think it would only pause that level of the tree
2017:01:16 00:00:17 bbloom like if you did (s/and a? b? unspeced c? d?)
2017:01:16 00:00:31 bbloom then you would check a and b, then issue a warning, but not even bother with c or d
2017:01:16 00:00:41 bbloom in theory it shouldn’t affect the context at all tho
2017:01:16 00:01:20 rickmoynihan for checking… but for gen?
2017:01:16 00:01:35 bbloom ah, hmm
2017:01:16 00:02:01 bbloom yeah, i think it would annihilate specs until an or higher up in the context
2017:01:16 00:02:10 rickmoynihan yeah
2017:01:16 00:02:10 bbloom which matches my intuition for cut
2017:01:16 00:02:21 bbloom or continuations in general
2017:01:16 00:02:30 bbloom ie skip is “throw” and every or has a catch
2017:01:16 00:03:31 rickmoynihan thats what I was getting at before
2017:01:16 00:03:38 bbloom gotcha
2017:01:16 00:03:51 bbloom anyway - this is a neat idea that i have no time to investigate implementing 😉
2017:01:16 00:03:55 bbloom thanks for the discussion tho!
2017:01:16 00:03:56 rickmoynihan also wonder if it’s kinda like choose
2017:01:16 00:04:06 bbloom choose?
2017:01:16 00:04:36 rickmoynihan pretty sure it was called choose… will need to dig out my ansi common lisp
2017:01:16 00:10:20 rickmoynihan ok just dug it out - it’s in “On Lisp - Advanced Techniques for Common Lisp"
2017:01:16 00:11:33 rickmoynihan it’s basically just an operator in common lisp for doing this kinda thing - choose and fail
2017:01:16 00:12:25 rickmoynihan anyway it’s not really helpful
2017:01:17 09:05:49 lmergen is anyone else struggling with a sudden ‘spec creep’ in their code ? how would one effectively determine what does and what does not belong in a spec ?
e.g. if you’re accepting a file as a parameter, should you spec that this file exists, or should it only spec that it looks like a file ?
2017:01:17 09:06:39 lmergen i’m err’ing towards making spec only define what things should look like
2017:01:17 09:21:23 joost-diepenmaat @lmergen I would assume that checking for file existence is not very useful since you need to check that actually opening the file succeeded anyway (and do something relevant when that failed). a file existing during precondition does not guarentee that it’s still there when you try to access it
2017:01:17 09:21:54 lmergen that makes sense
2017:01:17 09:22:43 joost-diepenmaat depending on your code it may make sense to generate files though...
2017:01:17 09:22:52 joost-diepenmaat for checking
2017:01:17 09:23:39 joost-diepenmaat personally I’d try to avoid messing with the filesystem in tests and write as much as possible in terms of sequences or whatever makes sense to your problem
2017:01:17 09:26:01 joost-diepenmaat The idea with speccing function arguments as I understand it is to describe whatever input data that is supposed to be handled. In this case, handling non-existing files is sort of necessary given the mutable nature of the file system.
2017:01:17 09:26:48 dottedmag @lmergen My rule of thumb: spec predicates should be pure functions.
2017:01:17 09:28:41 dottedmag Another one: behaviour of the system should not change if spec checking is disabled (except for the boundaries where spec is explicitly used for parsing/destructuring external data). So all runtime error handling (file not found etc) ought to be in the function itself, and spec should only prevent programming errors.
2017:01:17 09:29:05 lmergen what about s/conform ?
2017:01:17 09:31:08 joost-diepenmaat if you’re using that explicitly in your function body then you’ve sort of guaranteed the specs can’t be disabled 🙂
2017:01:17 09:31:17 joost-diepenmaat same with macros
2017:01:17 09:32:18 joost-diepenmaat that’s the parsing case that @dottedmag is talking about I think
2017:01:17 09:35:50 dbushenko are there any libs wich already have specs? just wanted to see some good examples of using specs
2017:01:17 09:37:07 dottedmag There are some specs for clojure.core already. Mostly macros.
2017:01:17 09:37:32 dbushenko nice!
2017:01:17 12:38:58 joshjones @lmergen I found myself asking this question recently, and reading the rationale/motivation for spec made things a little clearer. while I like @dottedmag ‘s “only pure functions” rule of thumb, the spec guide itself has a section called “Using spec for validation” that includes several runtime validation examples which are not at the edge of the system at all, but which are deeply integrated. That level of integration makes me a bit nervous, but it’s clear that spec is integrated enough that this is a valid enough use case, at least for some people.
2017:01:17 16:57:18 tetriscodes I’m struggling with how much spec right now
2017:01:17 16:57:58 tetriscodes I’m thinking of putting it only where developers use the system (http endpoints, components apis,…)
2017:01:17 16:59:14 tetriscodes I’ve seen the most benefit in my system from using the generators after specing. Its found more bugs in there than our devs have seen using the functions that are spec’ed.
2017:01:17 18:20:04 angusiguess Has anyone had to spec something that might be used by older versions of clojure?
2017:01:17 18:20:35 angusiguess What's a good way of providing spec that's only read for 1.9+?
2017:01:17 19:53:04 gfredericks seperate namespace for the specs
2017:01:18 02:22:52 bbloom i could imagine such a thing being integrated with spec, but not exactly sure how
2017:01:18 02:27:42 bbloom i’m using it to do something akin to constraint propagation
2017:01:18 02:30:58 bbloom lots of other details to work out with it tho - like what to do if a contradiction occurs
2017:01:18 07:25:35 ikitommi Are there any news on the next steps & roadmap for the spec? Trying to develop the runtime dynamic conforming stuff on top of spec. Would love to see Specs as types/records instead of reified protocols so that one could extend the specs easier. I'm toying now with stripping out extra keys from map specs & optionally giving errors for extra keys. Both at runtime, which is cool at api borders.
2017:01:18 07:28:31 ikitommi e.g. clojure.spec/KeysSpec. could make it implement protocols to extract it’s conforming type & to list it’s keys. and to generate docs etc.
2017:01:18 07:32:35 ikitommi this works now, but requires explicit conforming type hints (to tag specs):
2017:01:18 14:39:44 mpenet it's on my "kick tires" short list
2017:01:18 14:40:52 mpenet but yes, an update would be nice indeed
2017:01:18 15:14:29 ikitommi thanks @mpenet for the link. Spectrum, spec-tools and speculate are all trying to convert specs into something that is easier to mold - with custom records, walkers & ast. The upcoming(?) specs of specs will make them easier to interpret, but records would give a one way to extend them.
2017:01:18 15:19:30 mpenet hard to know how to do something like that until spec stabilizes really
2017:01:18 16:03:13 souenzzo Is someone getting pull-patterns from specs?
2017:01:18 16:21:20 kspear Does spec enable transformation from one 'spec' to another? i.e. could I take a map {:name "Bruce" :email " and transform it to conform to another spec {:name-first "Bruce" :contact {:email "? I feel like that might be possible with conform/unform or some other spec function, but I have yet to figure it out. Any help or direction would be appreciated.
2017:01:18 17:31:06 seancorfield Yes, but… bear in mind that all users of your spec would then get the transformed result, which is why that sort of thing is generally frowned on.
2017:01:18 17:32:12 seancorfield FWIW, we use specs that conform strings to non-strings as part of API input handling, but we try to avoid changing the shape of the data, unless there’s a very specific reason for that (and we want all spec client code to experience that change).
2017:01:18 17:32:30 luxbock @seancorfield I took his question to mean if there might be an automated way of translating data that has the same components but different structure by specying two specs and maybe some mapping between sub-components
2017:01:18 17:35:38 seancorfield Ah, I see what @kspear means now. TL;DR: “no” 🙂 That’s a general data transform issue tho’ — even with specs, you’d have to be able to specify exactly what parts of one spec are transformed to which parts of another.
2017:01:18 17:36:45 luxbock if you have specs-of-specs perhaps you could automatically map certain ident-kw's to each other
2017:01:18 17:47:17 lmergen I think then you're starting to build a type system, where you define rules how types can be casted
2017:01:18 17:47:46 lmergen I think this is something Clojure wants to avoid, and wants to make these transformations explicit
2017:01:18 17:49:58 kspear Yeah-- it seems like the sort of thing I could do 'easily' enough without spec at all; the thought occurred when I was playing around with conform/`unform`. I can, for example, use s/cat to say that each value in a list corresponds to some key and spec and see it in the 'specced' form with (conform ::my-first-spec my-list); then, i can unform it with a different spec to re-arrange the values into a different format (conform ::my-second-spec (conform ::my-first-spec my-list)). I thought if such a thing were possible on a list, there might be a way to do it with a map.
2017:01:18 17:55:24 kspear I figure a bit of context might help. In this specific case I'm taking reports generated (XML and json files, typically) by third party tools (vulnerability scanners) and I want to be able transform each 'issue' in each report into a common internal format within my program. Then, I want to be able to transform issues in my internal generic format to a JSON object that I can pass to our issue tracker API (i.e. JIRA). I'm just wondering if there is a way to specify what all of those formats look like (the data I get from my various third party tools, my internal generic issue format, JIRAs create issue format) and transform between them taking advantage of spec.
2017:01:18 18:04:50 lmergen yeah, I think that just mapping them might be better, you want to make these types of transformations explicit
2017:01:18 18:32:45 alexmiller I’d say the purpose of spec is not a generic data transformation library
2017:01:18 18:34:07 alexmiller and Clojure itself has plenty of tools for doing that (outside of spec)
2017:01:18 18:38:47 alexmiller conform/unform can be used to move back and forth between the original and conformed version of data, but pushing this use into transformation leads to treating spec as a “meat grinder” (Rich’s words) rather than something that states a specification of your data. in the long run, I think that path will cut you off from some uses of spec as you are no longer using it as a specification but as a process.
2017:01:18 18:42:45 mpenet We use spec a lot. Never to coerce data tho. We actually use specter for this
2017:01:18 18:43:19 mpenet Now that I think of it l dont think we use coercers at all
2017:01:18 18:43:27 alexmiller +1 :)
2017:01:18 18:43:37 mpenet I mean directly
2017:01:18 18:44:52 alexmiller @ikitommi I’d say it is still unresolved what the final implementation form will be for specs (protocols, records, or something else) but regardless we would not encourage direct access to those internals (and in fact the likelihood of people abusing that access is considered to be one downside to records as impls). The encouraged path to move from a spec (which is data, but in a form) to more map-like data is to conform the spec’s form.
2017:01:18 18:45:39 mpenet We did go through a phase where we did use coercers for transforms, but imho it gets messy fast
2017:01:18 18:46:08 alexmiller yes
2017:01:18 18:46:52 alexmiller spec works best when you primarily use it to speak truthful, assertive statements about what your data is
2017:01:18 18:47:59 alexmiller and not to transform your data or make statements about what your data is not
2017:01:18 18:48:14 mpenet One of the thing I like is togglable asserts in pre/post
2017:01:18 19:00:24 hiredman once you've created data transformation functions, and spec'ed them, you can examine the spec registry to find transforms with matching inputs and outputs to derive larger transforms
2017:01:18 19:17:18 kspear Thanks everyone so far for the responses. Definitely helps. In my specific case, I'm getting this third party data that, in essence, is the same data, just in proprietary representations. My assumption was that, using spec, I could author specs that define what the common data is in these different representations and leverage those to extract the common data into a generic representation. If spec isn't the answer to this problem, I have to assume there is another idiomatic approach to this type of problem in clojure already?
2017:01:18 19:19:58 joshjones your first example seemed to imply that you’re just talking about different key names for the same basic map structure — is that accurate, or was your example oversimplified?
2017:01:18 19:20:42 alexmiller well I think all that is fine if you’re talking about specifying the data in either the different representations or the common representation and where it is less so is when it’s about transformation
2017:01:18 19:22:32 kspear @alexmiller Aye, ok. So realistically I should be writing transformation functions like (defn external-format-1->common-format [data] ...), and then using spec to validate the input (external format spec) and output (my common format spec).
2017:01:18 19:25:04 kspear @joshjones Simplifying a bit. I need to rename keys, but also apply some transformation to the corresponding values (i.e. one external format might represent a date as a string with a particular ISO format that I then want to transform it to something compatible with the clj-time library). The structure, also, is going to be different between sources.
2017:01:18 19:26:08 alexmiller yeah, that’s not spec’s job (imo)
2017:01:18 19:26:22 alexmiller even though you could coax it do so with conformers etc
2017:01:18 19:27:01 alexmiller but you could for example write a spec for both sides and then write code that read both specs and built the transformation between them for you
2017:01:18 19:29:08 joshjones so in this case @alexmiller , you’re not using spec to do the actual transformation, but to describe the two pieces, and then you write something to bridge the two specs … is that kind of the idea? kind of like keeping spec “clean” and letting something else do the dirty work.
2017:01:18 19:30:28 alexmiller right (although I object to clean/dirty connotations :)
2017:01:18 19:30:35 joshjones lol
2017:01:18 19:30:57 alexmiller in that I think spec is good at specifying and that clojure transformation functions are good at transforming
2017:01:18 19:31:06 alexmiller and you should use the right tool for the job
2017:01:18 19:32:02 gfredericks man I seriously need to go take a year off in the woods and figure out how to do bijections right
2017:01:18 19:32:44 alexmiller the first 80% is easy
2017:01:18 19:32:50 alexmiller the last 10% is impossible :)
2017:01:18 19:33:01 alexmiller in other words, the usual :)
2017:01:18 19:33:46 alexmiller seriously though, bijections over spec would be a fantastic lib
2017:01:18 19:35:55 alexmiller actually even narrowing the problem from json <-> clojure would be interesting in that it’s clearly the most common need for this and might be able to capture a bigger piece of the pie with default behavior
2017:01:18 19:36:37 gfredericks every time I think about that I get grumpy that spec doesn't have a s/string-keys
2017:01:18 19:37:22 gfredericks but maybe that can be done in userland
2017:01:18 19:38:20 gfredericks the two things I hate about that ⇑ library are A) I think surjections/coercions should be first-class, hooking in to generators, and B) the composition API is terrible and I don't know how to make it better
2017:01:18 19:40:38 triss hey all. has any one looked at generating WSDL from specs yet?
2017:01:18 19:44:28 alexmiller I am professionally obligated to question your use of WSDL in the first place :)
moving past that, I have not seen anyone mention it
2017:01:18 19:45:32 triss hah indeed! does seem like a messy bit of tech… not sure it solves what it claims to or not. Seems to me a lot of it could be generated though
2017:01:18 19:45:58 triss I’m just looking at it for academic/comparitive reasons
2017:01:18 19:46:16 alexmiller from my experience, wsdl is almost always generated (which is why it’s so bad)
2017:01:18 19:46:53 alexmiller people have primarily used it to write giant xml docs that describe an rpc call
2017:01:18 19:47:46 triss yup. loose coupling. did that ever come really? I suppose we’ve abstracted over the lower level comms protocols at least
2017:01:18 19:49:56 alexmiller Rich has a lot of ideas about this area, many of which are in harmony with spec
he teased those in the last conj keynote, but they are worthy of a lot more time
2017:01:18 19:52:02 donaldball Many folk are using swagger to do the job WSDL used to pretend to do, though I think it may be called openapi now
2017:01:18 19:54:05 triss thanks @donaldball I’ll take a look.
2017:01:18 21:01:07 ikitommi thanks @alexmiller. looking forward to the new alphas.
2017:01:18 21:33:31 hiredman I've been using spec internally, and generating json schemas from specs for external interfaces (and I think someone else had a github repo or a pastebin doing it too), and I've been thinking about using it to generate graphql types (but I am not sure if I am going to use graphql) and the same for protobufs. Because spec is clojure data it is great for using the same spec to generate lots of other kinds of schemas. You take the same spec data and interpret it different ways.
2017:01:18 23:25:38 qqq I have a three line spec that looks like:
(s/def ::node any?)
(s/def ::children (s/coll-of ::tree :kind vector?))
(s/def ::tree (s/keys :req-un [::node ::children])))
;; then I call generate on it via
(gen/sample (s/gen ::tree))
this is getting me stack overflow, probably due to generating infinite depth tree
how cna I make generating trees work?
2017:01:18 23:30:26 gfredericks I thought spec handled that via some depth limit
2017:01:18 23:30:56 qqq Given how well thought out it is, it probably did, and I, asa a spec noob, simply am not using it correctly.
2017:01:18 23:30:58 qqq How do I use depth limits?
2017:01:18 23:31:02 gfredericks If that doesn't help, you can build a generator manually in test.check via the recursive-gen function
2017:01:18 23:31:34 gfredericks I don't know, I'd search the two namespaces for anything with the word depth in the name
2017:01:18 23:35:47 qqq (def ^:dynamic recursion-limit
"A soft limit on how many times a branching spec (or/alt/*/opt-keys/multi-spec)
^^ I'm sorry, can you please explain how this helps with my ::children / ::tree recursion via s/coll-of ?
2017:01:18 23:35:53 qqq I don't seem to be using any of the functions they 'limit'
2017:01:18 23:36:25 gfredericks That's probably the problem
2017:01:18 23:37:12 qqq How do I rewrite my spec to use those functions? 🙂
2017:01:18 23:37:16 gfredericks So you want node values on internal nodes too?
2017:01:18 23:37:22 gfredericks Not just leaves?
2017:01:18 23:37:33 qqq Yes, both leaves and internal nodes can store values.
2017:01:18 23:37:53 qqq Are you suggesting separate classes and and (s/or ::internal ::leaf) ?
2017:01:18 23:38:21 gfredericks That might fix it, yeah
2017:01:18 23:38:40 gfredericks Only other idea I have is the manual generator idea from earlier
2017:01:18 23:39:06 gfredericks Separate classes is probably easier
2017:01:18 23:40:00 qqq Spec is important enough that. I want to learn both.
2017:01:18 23:40:14 qqq Do you know of a good tutorial for manual generators? seems like I'm going to run into it sooner or later
2017:01:18 23:41:16 gfredericks There are some docs in the test.check repo
2017:01:18 23:45:20 bfabry there's a screencast on manual generators
2017:01:18 23:46:15 qqq @gfredericks : I can't get the two class approach to work; would you mind showing me sample code?
2017:01:18 23:46:22 qqq this is what I have so far:
2017:01:18 23:47:34 qqq but when I try to gen on ::nt, I get some error about nt-leaf nt->children nt-leaf nt->chlidren ...
2017:01:18 23:47:37 bfabry when does threading drop? this week right? I've been missing threading since flowdock died
2017:01:18 23:47:53 seancorfield Refresh now -- it's enabled for this Slack already.
2017:01:18 23:49:02 qqq this is the new ->> ?
2017:01:18 23:49:22 bfabry seancorfield: oh sweet. aaaaand I'm disappointed. this is not low friction. oh well
2017:01:18 23:50:09 seancorfield Yes, so you and Gary can chat in a thread about your specific issue at length and then just report a summary back to the channel -- and folks can read all the details if they want, or be spared them if they don't.
2017:01:18 23:50:47 qqq interesting
2017:01:18 23:51:04 qqq I'd still like an ignore button though 🙂
2017:01:18 23:52:54 bfabry yeah... and no one will notice a conversation they can help out with casually and chip in, like I just did with the screencast link for qqq. meh. feels like if people use it low traffic slack channels will be even less useful than they are now. oh well
2017:01:18 23:55:08 qqq resolved it: I was mis using (s/or .... )
2017:01:19 00:11:34 seancorfield (You could / should have marked that last reply as “Also send to #clojure-spec” so it would appear back in the main channel)
2017:01:18 23:52:51 bbloom i fear this is one of those “good ideas on paper” sort of things...
2017:01:18 23:52:59 bbloom the slack threads, that is
2017:01:18 23:56:07 qqq I don't like threads either; what I want is a way to "mute" people taht seem way too chatty; but the problem is, those ppl are probably not goign to voluntarily start a thread
2017:01:19 00:12:20 seancorfield qqq: We (admins — and the community at large) can educate them to do so… Threads will just take some getting used to, like all changes 🙂
2017:01:18 23:56:22 qqq s/"mute"/ignore
2017:01:18 23:56:55 qqq trees via 2 classes works now
2017:01:18 23:57:08 qqq @bfabry : going to study your video on manual generators now
2017:01:18 23:58:05 bfabry qqq: not mine! stuart holloway's work
2017:01:19 02:54:17 qqq just worked through taht video ; am now convinced that stuart holloway is a genius
2017:01:19 00:00:17 qqq somethign else tht would be nice in spec would be to gnerrate recursive structures breadth first, and when we hit a certain node count, only take non recursive branches
2017:01:19 00:18:39 qqq is there a spec option to limit the size of coll-of ?
2017:01:19 00:28:40 qqq where is the code which shows how coll-of are generated?
2017:01:19 00:29:06 qqq I'm looking at github/clojure/clojure, which sees to show checking of coll-of, but not generating of coll-of
2017:01:19 00:36:04 alexmiller :gen-max is the option to limit coll-of
2017:01:19 02:57:47 qqq does clojure spec support things like parameterized spec? I want to be able ot say things like "this is a ::tree-of ::widget" and "this is a ::tree-of ::gadget" instead of havint to define ::tree-of-widget and ::tree-of-gadget (or one for each 'type')
2017:01:19 03:01:55 seancorfield No.
2017:01:19 03:12:09 alexmiller you can however use s/multi-spec to create an “open” thing in the middle
2017:01:19 03:14:34 seancorfield Ah yes, I need to into multi-spec some day 🙂
2017:01:19 03:29:21 qqq @alexmiller: I have no idea what this multi-spec technique is. Can you point me at a tutorial?
2017:01:19 03:38:21 alexmiller but basically you use a multimethod to choose which spec to use based on the data
2017:01:19 12:26:22 roman01la This question probably was raised already. I’ve played with spec in ClojureScript a little, created a spec for application state for validation purpose. But I found it annoying to use fully-qualified keys with maps, because it requires the namespace whenever I need to get or destruct a map. This is what stopping me from using spec in production code.
Am I missing something? I feel like there should be a reasonable explanation for this.
2017:01:19 12:27:45 gfredericks @roman01la there are some new syntaxes for namespaced keywords
2017:01:19 12:29:46 roman01la @gfredericks do you mean namespace alias? I’m using them.
2017:01:19 12:30:21 gfredericks for constructing maps whose keys all have the same namespace
2017:01:19 12:30:41 gfredericks also {::keys [foo bar baz]} I think
2017:01:19 12:30:48 gfredericks when destructuring
2017:01:19 12:31:19 gfredericks the map syntax is #:foo{:bar 12} and #::baz{:bang 12} I think
2017:01:19 12:31:35 gfredericks you should see it at the repl when it prints a map with namespaced keys
2017:01:19 12:32:16 rickmoynihan @roman01la: Not sure if this is what you mean by "require" you don’t need to :require and load a namespace to use keys that are fully-qualified
2017:01:19 12:33:02 gfredericks but you do to alias
2017:01:19 12:33:06 rickmoynihan yes
2017:01:19 12:35:59 rickmoynihan roman01la: also not sure if you know but s/keys has :req-un and :opt-un to let you bridge between unnamespaced keys in maps and namespaced keys in specs…
2017:01:19 12:36:31 roman01la @gfredericks that’s great, thanks! #::baz{:bang 12} doesn’t work in ClojureScript, probably not implemented yet
2017:01:19 12:37:33 gfredericks @roman01la: if the clojurescript compiler runs in clojure 1.9 does it work?
2017:01:19 12:40:33 roman01la @gfredericks didn’t try it on 1.9 yet
2017:01:19 12:42:13 gfredericks if cljs still uses clojure's reader then that'd be a prereq for sure
2017:01:19 12:48:39 roman01la @gfredericks tried with 1.9.0-alpha14, still doesn’t work. I guess ClojureScript has it’s own reader. Anyway, thanks for sharing the new syntax, this will save me some time!
2017:01:19 12:48:40 pyr @mpenet can you expand on what you mean by "One of the thing I like is togglable asserts in pre/post" ?
2017:01:19 13:20:12 mpenet @pyr using :pre/:post in fn definitions, mostly s/valid? calls. and having the abitily to switch it on/off depending on assert value at compile time
2017:01:19 13:20:24 mpenet *assert*
2017:01:19 13:21:03 mpenet ex with a profile with :global-vars {assert false} for prod
2017:01:19 13:21:28 mpenet you can do the same with s/assert, inline, but I find pre/post kind of nice for this stuff
2017:01:19 13:21:45 mpenet s/assert has the advantage that you can toggle in on/off at runtime
2017:01:19 13:24:17 pyr yup ok
2017:01:19 13:24:18 mpenet all of this is handy for the cases when instrumentation is either too intrusive or when you want to actually run this stuff in prod
2017:01:19 13:56:42 pyr @mpenet thanks, i'll play with that later today to see how I fare
2017:01:19 13:57:23 pyr I didn't know :pre/:post validation asserts could be en/disabled at compile time
2017:01:19 14:04:03 pyr I was reading that yes, still first google link to show up 🙂
2017:01:19 14:05:02 mpenet I guess it could make it relatively easy to write some sugar on top of it to have Schema like defn with annotations.
2017:01:19 14:05:27 mpenet But personally I am good with what we have
2017:01:19 14:06:49 pyr yes, this is also enough for my needs
2017:01:19 15:18:41 owen is there a good way to test that an fdef with a generator will throw an exception somewhere? I'm not having success
2017:01:19 15:59:28 alexmiller can you expand that question? not understanding it
2017:01:19 16:04:55 owen Sure thanks Alex. I'm testing a function with stest/check, an fdef and a generator that is guaranteed to cause an exception. I can't figure out what :ret I can use in the fdef to check for the exception
2017:01:19 16:06:41 alexmiller you can’t
2017:01:19 16:06:53 alexmiller :ret is for non-exceptional returns
2017:01:19 16:07:10 alexmiller there is no way to spec exceptions
2017:01:19 16:08:46 owen Ah ok makes sense then
2017:01:19 16:16:00 ddellacosta what is the rationale behind clojure.spec/or returning map pairs? I was hoping to be able to use it to validate what is essentially a sum type, but seems like I will have to do more processing to handle that. I guess my other question is, is there anything suitable for validating a sum type (i.e, simply has a bunch of predicates, and is valid if one of the predicates returns true)?
2017:01:19 16:16:11 ddellacosta happy to get pointed to docs; apologies if this has been asked many times here
2017:01:19 16:18:37 ddellacosta I do see the doc-string says
> Returns a destructuring spec that returns a map entry containing the key of the first matching pred and the corresponding value. Thus the 'key' and 'val' functions can be used to refer generically to the components of the tagged return.
2017:01:19 16:19:00 ddellacosta I sense the rationale is lurking in there, but I suppose this speaks to my larger ignorance of how to use spec idiomatically
2017:01:19 16:19:24 ddellacosta something something use maps more something something
2017:01:19 16:27:14 joshjones Maybe I’m oversimplifying your question @ddellacosta , but when you conform using s/or, how would you know which predicate matched without some way to identify it?
2017:01:19 16:28:04 alexmiller the idea is that the tag is used to identify how spec parsed the value (and also used to identify problems in explain errors)
2017:01:19 16:28:22 ddellacosta @joshjones why do I care if it’s a valid value of one of the types I’m expecting?
2017:01:19 16:28:42 ddellacosta @alexmiller I guess I don’t see how to easily use that when composing it with another spec
2017:01:19 16:29:00 alexmiller many specs conform to a structure that describes how they were parsed
2017:01:19 16:29:08 alexmiller all of the regex specs, etc
2017:01:19 16:29:28 ddellacosta yeah, there seems to be something basic I’m missing here
2017:01:19 16:29:43 alexmiller two options for not having this issue specifically with s/or...
2017:01:19 16:30:46 alexmiller 1. write a function that accepts all the options, so instead of (s/or :e even? :p pos?), do #(or (even? %) (pos? %))
2017:01:19 16:31:11 alexmiller that makes the choice opaque (which may be good or bad depending on your needs)
2017:01:19 16:32:12 alexmiller 2. use the currently undocumented and possibly to-be-removed spec s/nonconforming: (s/nonconforming (s/or :e even? :p pos?))
2017:01:19 16:33:13 alexmiller if you just want to check whether a value is valid, use s/valid?. if you want to know why it’s valid, use s/conform. If you want to know why it’s invalid, use s/explain.
2017:01:19 16:34:05 ddellacosta @alexmiller thanks for all of that
2017:01:19 16:34:40 ddellacosta I guess part of the problem is that I’m using s/conform and expecting it to more or less simply return the value I’m testing, except in specific cases (changing vector to set, in this case I think)
2017:01:19 16:34:56 ddellacosta and part of that is maybe that I don’t yet understand the use-case for s/conform
2017:01:19 16:35:12 ddellacosta and how one would use something like s/or effectively
2017:01:19 16:35:17 ddellacosta within that context I mean
2017:01:19 16:35:25 alexmiller yeah, s/conform returns a different value than the original value specifically to tell you why/how it’s valid
2017:01:19 16:35:39 alexmiller if you don’t care, don’t use conform
2017:01:19 16:35:51 ddellacosta right, okay…that is a helpful way to think about it
2017:01:19 16:36:20 alexmiller note also that you can use s/unform to retrieve the original value from the conformed value too
2017:01:19 16:37:24 ddellacosta I guess thinking about it a bit more, I see two distinct possible values that could be returned from conform: one is the value coerced to a new value, and the other has added spec metadata —it seems to me right now that s/conform conflates these two somewhat
2017:01:19 16:37:31 ddellacosta for example, converting a vector to a set is the former
2017:01:19 16:37:40 ddellacosta whereas the output of s/or is the latter
2017:01:19 16:38:12 ddellacosta am I off here? I still feel like I don’t get the broader implications of spec, or how we might be able to use it
2017:01:19 17:16:03 seancorfield (if (s/valid? ::my-spec some-value) some-value (handle-error (s/explain-data ::my-spec some-value))) seems to be what you’re looking for @ddellacosta ?
2017:01:19 17:16:19 seancorfield (or some similarly structured setup)
2017:01:19 17:18:50 seancorfield We use both s/valid? and s/conform — we use the latter when we want a result that “conforms” to a spec but may start off slightly differently (which we use for our API inputs, to produce domain model values). And as Alex said, when we don’t care about any potential change in the data, we use the former.
2017:01:19 17:29:09 ddellacosta @seancorfield thanks, will see if that does it. Per your second point, ideally I’d like to use s/conform in the same way—but if s/conform returns data that has been changed to represent how a value conforms to a spec, doesn’t that defeat the purpose?
2017:01:19 17:30:15 seancorfield Well, it will only do so under the rules of the spec itself.
2017:01:19 17:30:37 lmergen @seancorfield: i would be very interested to learn how you use spec in real world projects
2017:01:19 17:30:51 lmergen we're introducing it right now, running alpha 14
2017:01:19 17:31:09 lmergen and it's a bit tough to find the right "pattern" that makes sense to us
2017:01:19 17:31:45 seancorfield For example, we have API input specs that accept either a date, or a string that can be parsed to a date using two allowed formats. s/valid? on such a string will just return true (and we still have a string; or false for an illegal string). s/conform on such a string will return a date (or ::s/invalid for an illegal string).
2017:01:19 17:33:49 joshjones so in this case is it accurate to say that you are using spec to actually transform the data? (string -> date)
2017:01:19 17:34:05 seancorfield Another use case: we allow members to search within specific distances (dating site). If they are a subscriber, they can specify distances of 20, 30, 40, 50. All members can specify distances of 150, 500, 1000, and -1 (”any”). We use a spec with s/or to get back either [:basic distance] or [:platinum distance] so we can validate both the actual value and which category it belongs to.
2017:01:19 17:35:10 seancorfield @joshjones Yes, some coercions. As Alex often cautions, that conforming / coercing means all clients of those specs get the coercion if they use s/conform but in this case that’s what we want.
2017:01:19 17:35:14 ddellacosta to me that seems like a spec is somewhere in between a value and a statement about a value
2017:01:19 17:35:20 ddellacosta I find this pretty confusing, fundamentally
2017:01:19 17:35:39 ddellacosta seems like it makes s/conform of limited use
2017:01:19 17:35:49 ddellacosta or at least, limited in the case of using s/or
2017:01:19 17:36:53 ddellacosta again, I’m talking about s/conform here @joshjones
2017:01:19 17:37:13 ddellacosta and it seems to violate exactly what those comments are saying, in fact
2017:01:19 17:37:17 seancorfield If you need to know which “path” of conformance your value takes, s/or is very useful. Since otherwise you’d need to code up both the validation logic and the parsing logic in order to determine that.
2017:01:19 17:37:21 joshjones I’m not referring to anything you’re talking about @ddellacosta , only to what @seancorfield said about his use of spec
2017:01:19 17:37:54 joshjones no problem, I am also seeking a better understanding of best practice 👍:skin-tone-2:
2017:01:19 17:40:44 ddellacosta yeah, along those lines this is all really helpful—thanks for linking to those comments @joshjones
2017:01:19 17:42:58 seancorfield It’s taken us a while to settle on this approach — given that Alex has repeatedly cautioned against specs that coerce data 🙂 — and we also have the added complication that for a given API, some of our input parameters must be looked up and “conformed” dynamically (i.e., we have a database table listing possible inputs and each row identifies a spec that should be applied). So we can’t just write static specs for input arguments, we have to have somewhat generic specs and conform each input argument based on a lookup.
2017:01:19 17:43:07 seancorfield (and we’re still wading through cleaning that up)
2017:01:19 17:43:50 seancorfield We could probably simplify this by dynamically generating specs directly from our metadata at this point — and we may well do so.
2017:01:19 17:44:05 ddellacosta so question for you, @seancorfield —why not simply write something explicit that does not rely on spec to add the metadata about your distance (platinum vs. basic)? It seems to me that this is a value, not a spec.
2017:01:19 17:44:31 ddellacosta I’m asking selfishly to clarify my understanding of spec, to see how you are thinking about this, as I still don’t fully get it
2017:01:19 17:45:37 seancorfield Because using spec allows us to remove a lot of logic that we had before.
(and the api-spec macro) replaces a bunch of explicit code that attempted to coerce string to number and validate it and categorize it as basic vs platinum. Now we can just have (let [[level distance] (s/conform ::distance input)] …) and we’re done.
2017:01:19 17:48:38 seancorfield Having the declarative specs means we can show those to business and they can “read the code” without having to actually read real code full of conditionals and transformations.
2017:01:19 17:49:30 seancorfield We can also have the business rules in one place (for the most part) and they drive not only our actual business logic but can also be used to generate test data (and test functions extensively).
2017:01:19 18:46:08 ddellacosta okay, I’ll have to think about that a bit. Thanks a lot @seancorfield , this has been very helpful
2017:01:19 20:50:40 scriptor how would one spec varargs?
2017:01:19 20:51:32 scriptor could I just use the vararg's name and use coll-of?
2017:01:19 20:55:18 schmee scriptor I think the idea is to spec the args as a sequence with spec/cat and use spec/? for the optional args
2017:01:19 20:56:11 joshjones @scriptor yes, compose the var args the same way you’d compose any sequence.. only it doesn’t have to be coll-of … can be k/v pairs, any sequential spec. for example:
2017:01:19 20:56:58 joshjones can be a coll-of, use spec regexes, anything … the above just shows a common use case for var args: k/v pairs in a sequence (as opposed to a map)
2017:01:19 20:59:37 scriptor so :rest indicates the following spec is for the optargs parameter?
2017:01:19 20:59:53 scriptor well, not indicates, you're just using the name :rest
2017:01:19 21:00:12 joshjones yes, should probably have called that optargs .. will change it for clarity
2017:01:19 21:00:35 scriptor perfect, thanks @schmee and @joshjones
2017:01:19 21:24:15 joshjones @scriptor I need to clarify something after doing another example — my example above works but I wanted to use another common case because I was wrong about just using coll-of for cases where you have one required and then an optional argument. for example:
2017:01:19 21:24:50 joshjones if you use coll-of for :optargs it will not work because you’re giving it multiple items, not an actual collection
2017:01:19 21:25:41 scriptor ah, interesting, I guess it's because of the regexy nature of s/cat
2017:01:19 21:26:27 joshjones well, s/cat gives the actual argument vector spec. but inside that, you then have several elements, and one of those is not a collection if it’s for example: [”abc” 42 43 44]
2017:01:19 21:26:29 scriptor the s/* greedily consumes the rest of the arglist, which as you said is multiple separate items and not a single collection
2017:01:19 21:26:44 joshjones what you have there is a string, an int, an int, and an int .. no collections
2017:01:19 21:27:08 joshjones yes that’s correct. i feel my answer above was misleading so i wanted to clarify
2017:01:19 21:43:46 joshjones another example that will match (fn [s & opts]):
2017:01:19 21:44:27 joshjones wanted to show that the optional argument portion of the sequence can be more than just a simple homogeneous concat of elements
2017:01:19 21:54:48 alexmiller you should really use s/keys* for this - that is it’s reason for existence
2017:01:19 22:11:03 alexmiller I’m going to go out on a limb and say that’s weird :)
2017:01:19 22:11:15 alexmiller and you shouldn’t write a function signature that way in the first place :)
2017:01:19 22:11:54 joshjones yes it is — my first example showed what you said — i was just showing how the optional arguments portion of a function need not be a homogeneous group of items .. not endorsing it, just demonstrating the versatility 🙂
2017:01:19 22:12:06 alexmiller ok
2017:01:19 22:12:28 joshjones saying “hey, if it can do this, it can definitely do something much simpler” 👍:skin-tone-2:
2017:01:19 22:32:56 gdeer81 is there a way to do multiple specs at once like you have a bunch of things that should be a string so instead of doing (s/def ::foo string?) (s/def ::bar string?) you can just say [::foo ::bar :baz ::fname ::lname ::email] string?
2017:01:19 22:54:51 bbloom has somebody done a richer explain function yet? i find that when you have some ors or alts w/ larger maps involved, the output becomes unwieldy
2017:01:19 23:01:26 alexmiller @gdeer81: no
2017:01:19 23:02:18 alexmiller @bbloom Drogalis started porting some of his onyx stuff to work over spec explain-data (and using fipp) - really more of a poc stage
2017:01:19 23:02:31 bbloom cool
2017:01:19 23:02:33 bbloom 🙂
2017:01:19 23:02:36 alexmiller I actually picked that up and hacked on it a bit but found that explain-printers are missing some crucial data
2017:01:19 23:02:43 alexmiller namely, the original root value and root spec
2017:01:19 23:02:59 alexmiller that’s missing from the explain-data map
2017:01:19 23:03:09 alexmiller I have a pending enhancement + patch to add that
2017:01:19 23:03:14 bbloom ah yes, those things and the immediate parent or some chain of anscestors should be passed down
2017:01:19 23:03:23 bbloom this way explainers can be context sensitive
2017:01:19 23:03:29 alexmiller everything else is discoverable within the explain-data path
2017:01:19 23:03:40 bbloom ah right, can just pop on the path
2017:01:19 23:03:42 bbloom that makes sense
2017:01:19 23:03:48 alexmiller but you don’t have the root value
2017:01:19 23:05:40 alexmiller I’ve shelved it for the moment until the patch above gets through as then it will be easier to demo
2017:01:19 23:05:59 bbloom i’m running in to a situation now where i have a recursive spec that is giving me useless explain output b/c it’s producing LOTS of output
2017:01:19 23:06:05 bbloom b/c the recursive spec fails to match
2017:01:19 23:06:46 bbloom one thing you could do with explain-data is find the CLOSEST match
2017:01:19 23:06:46 michaeldrogalis @alexmiller Curious as to how you attempted it. Took me ages to do it with Schema, not really looking forward to trying it again with Spec.
2017:01:19 23:06:52 alexmiller I seem to recall Colin made something similar in the macro grammar stuff he did
2017:01:19 23:07:03 alexmiller ^^ @bbloom
2017:01:19 23:07:13 bbloom yeah, it’s kinda critical i think
2017:01:19 23:07:37 bbloom otherwise spec falls down on ASTs and other recursive structures after data gets too big
2017:01:19 23:07:39 alexmiller @michaeldrogalis well, I mostly brought it up to more recent spec and extended it just slightly further
2017:01:19 23:07:41 bbloom at least explain does
2017:01:19 23:08:35 michaeldrogalis Ah, got it.
2017:01:19 23:08:36 bbloom unfortunately the literature on error detection and recovery in parsers seems to be kinda ad-hoc
2017:01:19 23:08:39 alexmiller @bbloom I also have a patch that sorts the longest errors first, which generally is a good strategy
2017:01:19 23:08:48 bbloom yeah, that’s a good start
2017:01:19 23:08:50 alexmiller except with recursion when you probably want the reverse :)
2017:01:19 23:08:54 bbloom heh
2017:01:19 23:09:00 bbloom is that longest path to an error?
2017:01:19 23:09:06 alexmiller yeah
2017:01:19 23:09:07 bbloom or just BIGGEST error?
2017:01:19 23:09:10 bbloom ok
2017:01:19 23:09:55 bbloom i asked about this on twitter a while ago in the context of text parsing
2017:01:19 23:10:41 bbloom chapter 10 is about error handling, which may be worth studying
2017:01:19 23:10:54 bbloom there seems to be some attempt at a sensible system for producing good error messages
2017:01:19 23:11:02 alexmiller I think cleaning up fanout is another big area for improvement
2017:01:19 23:11:38 alexmiller For removing repetition
2017:01:19 23:11:52 bbloom you mean like representing the output as a tree?
2017:01:19 23:12:18 alexmiller Nah, just reporting all errors at common path with single context
2017:01:19 23:12:27 bbloom ah ok
2017:01:19 23:12:29 alexmiller An ide might do a tree though
2017:01:19 23:12:39 bbloom another idea is a sort of two column view
2017:01:19 23:12:48 bbloom ie pretty print (or just indent) on the left
2017:01:19 23:12:55 bbloom and messages on the right
2017:01:19 23:13:00 alexmiller I suspect recursion will just suck regardless :)
2017:01:19 23:13:11 bbloom yeah, well recursion always sucks when it comes to printing stuff 🙂
2017:01:19 23:13:22 bbloom as a man who does a lot of recursion and a lot of printing, believe me, i know
2017:01:19 23:13:22 bbloom heh
2017:01:19 23:13:47 bbloom whatever happened to that data structure visualizer? is that still built in? lol
2017:01:19 23:13:53 alexmiller it is :)
2017:01:19 23:14:10 alexmiller it still doesn’t work very well :)
2017:01:19 23:14:23 bbloom yeah, that’s the thing about text based UIs
2017:01:19 23:14:27 bbloom when they are bad, they are kinda bad
2017:01:19 23:14:32 bbloom vs guis
2017:01:19 23:14:35 bbloom when they are bad, THEY ARE VERY BAD
2017:01:19 23:14:49 alexmiller gotta run, later
2017:01:19 23:14:56 bbloom cya
2017:01:20 01:23:41 bbloom just curious: why does s/keys check the map? predicate? seems like ILookup is the minimum requirement, or am i missing something?
2017:01:20 01:26:28 seancorfield I believe there's a JIRA issue open for that...
2017:01:20 03:10:08 alexmiller It has to iterate through the entries so ILookup is not sufficient
2017:01:20 03:31:25 bbloom Makes sense. I forgot about how it handles present keys even if not specified as optional.
2017:01:20 03:33:23 bbloom One solution would be to only provide that functionality if iterable or seqable. Not sure if that's a great idea though.
2017:01:20 09:26:13 joost-diepenmaat maybe this is a FAQ, but is there a way to automatically re-run clojure.spec/instrument after defining new functions or redefining already specced ones? I’d be fine if it would work in CIDER only.
2017:01:20 09:56:25 mpenet instrumentation is not meant to be used in production
2017:01:20 09:56:59 mpenet it's actually spelled out explicitly on that page
2017:01:20 09:57:10 mpenet as for conform, yes you can
2017:01:20 09:59:16 mpenet In production what I do is leverage :pre/:post + s/valid? in functions. and you can always turn it off with a compile time toggle. It works quite well
2017:01:20 14:49:06 timgilbert Hey, I have a kind of polymorphic map where some keys should be present depending on the value of one of the other keys, so it's like either {:x/type :t/foo :foo/val 32} or {:x/type :t/bar :bar/baz "47"}
2017:01:20 14:49:57 timgilbert It seems like this is difficult to describe with spec since the map specs only look at keys, not values. Am I missing something?
2017:01:20 14:51:39 potetm If I understand right, that's what you're looking for.
2017:01:20 14:51:59 timgilbert Aha! Exactly what I was looking for. Thanks @potetm.
2017:01:20 14:52:05 potetm np 🙂
2017:01:20 21:14:42 viesti Posed a question in the general Clojure channel, but realised that I need to think about predicates that reference their environment a little more. This was the snippet that I was pondering:
2017:01:20 21:15:56 bbloom there’s really nothing you can do other than eager substitution
2017:01:20 21:16:00 bbloom ie a macro
2017:01:20 21:16:38 viesti was using s/form for unit tests on spec that I generate from data
2017:01:20 21:17:04 alexmiller I just answered in #clojure
2017:01:20 21:17:12 viesti thinking that the serialization thing is probably still on it’s way to clojure.spec
2017:01:20 21:17:40 alexmiller s/form is there for that purpose (with some known bugs)
2017:01:20 21:18:51 viesti thanks for the answer @alexmiller and sorry for posting on wrong channel 🙂
2017:01:20 21:18:58 tbaldridge serialization with captured environments can get a bit tricky anyways.
2017:01:20 21:19:12 bbloom indeed
2017:01:20 21:19:35 bbloom it’s among the crazy ideas I’ve been studying for a number of years 😉
2017:01:20 21:20:57 viesti 😄
2017:01:20 21:45:16 viesti was actually using s/form for unit testing specs that I generate from sql ddl statements, just toying around for now
2017:01:22 08:32:15 andrewzhurov
2017:01:22 08:32:15 andrewzhurov
2017:01:22 08:32:21 andrewzhurov first
2017:01:22 08:33:42 andrewzhurov I more than sure that those questions appear often here, so, some links ?
2017:01:22 14:45:09 settinghead for those interested in seeing another javascript port of clojure.spec (in additon to js.spec), here is one I've made: https://github.com/clausejs/clausejs. Regex specs and fspecs are currently working. Data generation comes next.
I've also made a spec documentation generator called clausejs-docgen (still WIP), which I have been using to generate Clause.js's own documentation.
Lastly, there is a syntax function which takes in an fspec, enumerates over all of its arguments' possibilities, and outputs a concise list of lambda-ish syntax references (you can find examples in the doc site here: https://clause.js.org)
2017:01:22 14:46:04 Yehonathan Sharvit metalclj
2017:01:22 23:24:21 mrg Does anyone have any examples on how clojure.spec/conformer works? I am trying to coerce a json map (including UUIDs and instants, and seqs of keywords) and I cannot find any good examples anywhere
2017:01:22 23:46:10 olivergeorge I don't think it's really intended to use conform in that way
2017:01:22 23:49:28 olivergeorge as best I understand it, spec is intended to describe static data rather than the coerce data
2017:01:22 23:50:29 olivergeorge Technically that looks a bit like coercion in that the conformed data is a different shape.
2017:01:22 23:50:35 olivergeorge Hope that helps.
2017:01:22 23:55:21 seancorfield @mrg I answered in #clojure but I'll respond to @olivergeorge's comment: yes, you generally want to be very wary of specs that coerce data because then every "client" of that spec has to accept the coercion. So it's only good for a spec where you always want all "clients" to see the same coercion too.
2017:01:23 00:12:15 mrg Thanks @olivergeorge and @seancorfield . I was basing that off a comment of Timothy Baldridge on reddit that was suggesting that conformer was the way to go if one wanted to use a spec to coerce data
2017:01:23 00:12:29 mrg What would be the right way to do something like that then though?
2017:01:23 00:13:03 mrg Say I have a spec (s/def ::myspec #{:foo :bar})
2017:01:23 00:13:37 mrg and then i have a deeply nested json map that at multiple places uses ::myspec
2017:01:23 00:14:05 seancorfield Well, yes, s/conformer is the way to go -- but it's a "concern" to have a spec do data transformation.
2017:01:23 00:14:38 mrg sorry, there would also be (s/def ::myspecs (s/coll-of ::myspec :min-elements 1 :max-elements 5)) or something like that
2017:01:23 00:14:46 mrg I don't really like it either
2017:01:23 00:15:19 mrg But I can't think of a better way that doesn't have me re-implement half of spec for every branch of the tree
2017:01:23 00:16:24 seancorfield I'm not sure what you're asking -- and I think you're misunderstanding what we're saying...?
2017:01:23 00:16:30 mrg Quite likely
2017:01:23 00:17:56 mrg My actual use case is this: I have a service that 1) generates random data that conforms to a spec and spits it out as json 2) is supposed to take such a json map and check its conformance to the spec
2017:01:23 00:19:11 mrg 1) works mostly, using generators and gen/generate
2017:01:23 00:19:29 mrg My problem is in step #2 when I try to read the generated json back in
2017:01:23 00:20:02 mrg the uuids have become strings, the enums (like the ::myspec) above as well, and so have the #inst datetimes
2017:01:23 00:20:20 mrg And I can't figure out how to coerce them back into the format that my spec expects
2017:01:23 00:32:04 seancorfield Here's an example of some specs etc we use that accept strings and coerce them to dates:
(defn coerce->date
"Given a string or date, produce a date, or throw an exception.
Low level utility used by spec predicates to accept either a
date or a string that can be converted to a date."
[s]
(if (instance? java.util.Date s)
s
(-> (tf/formatter t/utc "yyyy/MM/dd" "MM/dd/yyyy"
"EEE MMM dd HH:mm:ss zzz yyyy")
(tf/parse s)
(tc/to-date))))
(defn ->date
"Spec predicate: conform to Date else invalid."
[s]
(try (coerce->date s)
(catch Exception _ ::s/invalid)))
(defmacro api-spec
"Given a coercion function and a predicate / spec, produce a
spec that accepts strings that can be coerced to a value that
satisfies the predicate / spec, and will also generate strings
that conform to the given spec."
[coerce str-or-spec & [spec]]
(let [[to-str spec] (if spec [str-or-spec spec] [str str-or-spec])]
`(s/with-gen (s/and (s/conformer ~coerce) ~spec)
(fn [] (g/fmap ~to-str (s/gen ~spec))))))
(s/def ::dateofbirth (api-spec ->date #(dt/format-date % "MM/dd/yyyy") inst?))
2017:01:23 00:33:11 seancorfield This accepts an instant or a string that will coerce to an instant. It auto-generates as well based on inst? and converts it to a MM/dd/yyyy string.
2017:01:23 00:34:19 mrg hey, thank you @seancorfield , that is super useful!
2017:01:23 00:34:32 mrg And I appreciate you helping me out on a Sunday evening 🙂
2017:01:23 13:31:37 rickmoynihan @brownmoose3q: Interesting… I’ve asked similar questions before, but never had any responses… I’m not sure if it’s because there isn’t an answer for it yet… I’d certainly like to know how to integrate clojure.spec with a clojure.test runner.
2017:01:23 13:32:51 rickmoynihan not sure what others do… but might be nice to have some clojure.test wrappers for tools like exercise etc...
2017:01:23 13:33:07 rickmoynihan obviously you can also use instrument
2017:01:23 13:58:22 andrewzhurov I just find out that lein test is giving that wierdo error and refind-out for myself this issue: https://github.com/technomancy/leiningen/issues/2173
turned off monkeydispatch and whoop - all is good.
2017:01:23 13:59:13 andrewzhurov So... seems to be a way of testing 🙂
2017:01:23 17:20:00 lsnape Can anyone suggest how to simplify the following:
I would like to ditch ::foo*, but if I reference the ::foo generator from within s/def ::foo, then understandably I get a SOE
2017:01:23 20:22:56 souenzzo There is option in s/keys that fails/remove on unwanted keys??
(s/valid? (s/keys :req [:a/b]) {:a/b 0 :c/d 1}) ;=> false
or something like
(s/"filter????" (s/keys :req [:a/b]) {:a/b 0 :c/d 1}) ;=> {:a/b 0}
2017:01:23 20:32:40 seancorfield @souenzzo spec is designed to be open for extension — read the guide for more details.
2017:01:23 20:33:15 seancorfield If you want a spec to fail for unwanted keys, you need to explicitly s/and a predicate that checks the set of keys in the map.
2017:01:23 20:33:26 seancorfield (but you probably shouldn’t do that)
2017:01:24 04:32:27 basti Hi, I’m pretty new to Clojure and I am playing around with Spec. I’ve got following error message
[{:type java.io.FileNotFoundException,
:message "Could not locate clojure/test/check/generators__init.class or clojure/test/check/generators.clj on classpath.",
while evaluating
(ns foo-test
(:require [clojure.spec :as s]
[clojure.spec.test :as stest]
[clojure.spec.gen]
[clojure.pprint :as pprint]
[clojure.test :as t]))
;; Sample function and a function spec
(defn fooo [i] (- i 20))
(s/fdef fooo
:args (s/cat :i integer?)
:ret integer?
:fn #(> (:ret %) (-> % :args :i)))
(stest/check `fooo)
What’s going on here? ^^
2017:01:24 04:33:38 alexmiller generators use the test.check library. this is optional so it’s not a required compile-time dependency
2017:01:24 04:34:01 alexmiller but if you want to use it at test time, you’ll need to include test.check as a test dependency
2017:01:24 04:37:22 alexmiller you’d want that in the :dev profile for it to be test-scoped only (not a dependency that downstream consumers would get)
2017:01:24 04:39:37 basti okay I c, sweet now it works, thank you for explaining it to me so clearly @alexmiller - really appreciated 🙂
2017:01:24 13:29:19 luxbock I'm specced a function which performs a deep-merge via addition on some maps with longs at the leafs, and I wanted to create a generative test for it
2017:01:24 13:30:02 luxbock however my generative test actually fails because I run into integer overflow, which is fair enough, that's good to know in theory
2017:01:24 13:30:21 luxbock but in practice none of the values I use are ever going to get that large as to cause an overflow
2017:01:24 13:31:19 luxbock so in this case, should I be handling the bounding of these values via instrument by using :gen overrides?
2017:01:24 13:32:49 luxbock or should I handle it at the site of the spec definition by adding some arbitrary bound as a predicate to go with pos-int?
2017:01:24 14:44:44 luxbock I tried working around this issue by using stest/instrument and overriding the gen that is causing the overflow via :gen, and then running stest/check on the function again but this did not seem to have any effect
2017:01:24 14:45:21 donaldball Maybe you want something like a ::smallish-long that describes a domain consistent with your expected values in production?
2017:01:24 14:49:49 luxbock @donaldball yeah I tried that approach as well, but it has the following problem: the function I'm testing is essentially taking a collection of values of ::map-with-smallish-int and summing all of the ::smallish-int together, so I specced the function to use the same return value
2017:01:24 14:50:27 luxbock and this of course fails because the largest values of what ::smallish-int generates summed together are going to be larger than the bound I'm using
2017:01:24 14:50:46 luxbock so of course I could use use a new spec for the return value of the function under test, which uses a higher bound
2017:01:24 14:50:54 luxbock but that feels less than ideal
2017:01:24 14:51:31 donaldball You… could merge with +’
2017:01:24 14:52:02 luxbock yeah that's another option but then I'm changing the definition of my function to use a slower function just to appease my tests
2017:01:24 14:52:32 luxbock but I'm guessing that I don't understand how instrument with :gen overrides is supposed to be used
2017:01:24 14:54:19 luxbock it would be nice if during tests I could just override the generators of my specs to match to the domain values I'm expecting during normal use, and then I write the test to perform inside this context
2017:01:24 15:18:58 luxbock using +' doesn't really help either as once the values get high enough to coerce to BigInt they will fail nat-int?
2017:01:24 15:53:19 alexmiller @luxbock the approach of using instrument with a :gen override is what I would recommend and should work
2017:01:24 15:53:57 alexmiller if you can a repro, would be interested
2017:01:24 16:34:21 luposlip Is there any chance that you guys could make the clojure.spec guide (PDF) printable? 🙂https://clojure.org/guides/spec
2017:01:24 16:45:47 qqq Does Chrome -> Print -> Save as PDF work? Or do you need something else?
2017:01:24 17:11:45 luposlip @qqq The result is just a lot of empty pages (I'm using Chromium BTW)
2017:01:24 17:14:15 qqq @luposlip: my bad, I also get only empty pages in chrome / safari
2017:01:24 17:14:31 qqq time to edit some HTML
2017:01:24 17:15:42 qqq hmm, I get something printable
2017:01:24 17:15:46 qqq but it loses all the formatting
2017:01:24 17:16:33 luposlip Sounds just great @qqq, because printable pages would make http://clojure.org so much more useful (for users that want to read the guides etc. on ereaders etc.)
2017:01:24 17:16:58 qqq I don't know how to fix this, sorry -- it seems like the CSS I delete to make it printable also deletes the nice CSS formatting that makes it pretty and has nice code layout.
2017:01:24 17:17:00 luposlip Would be really nice with a epub export button for the long pages 🙂
2017:01:24 17:17:23 qqq is the main clojure site on github ?
2017:01:24 17:17:28 qqq I wonder if you can just print the github rendering of the page
2017:01:24 17:17:40 luposlip I don't think so [UPDATE: I stand corrected]
2017:01:24 22:52:07 alexmiller Making that work is probably not something I’m going to work on, but if someone else wanted to help, that would be cool
2017:01:25 03:31:10 bbloom @alexmiller i think you could get 90% of the way there with a smaller change than a custom stylesheet - Safari etc’s reader mode is broken on the current site b/c div#preamble doesn’t match div.sect1
2017:01:25 03:32:18 bbloom i think that if you just make all the seconds have the same class, then reader modes will treat it all as body text and that’s a de facto print mode
2017:01:25 03:32:53 alexmiller most of that did not actually mean anything to me :)
2017:01:25 03:33:20 bbloom each page in the reference has multiple divs, but only the first div shows up if you use an article reading filter
2017:01:25 03:33:21 alexmiller I’m like a monkey banging rocks together in web programming
2017:01:25 03:33:27 bbloom lol
2017:01:25 03:33:57 bbloom you on a mac? go to any reference page in safari and you’ll see a little paragraph icon in the top left corner of the url bar
2017:01:25 03:34:10 bbloom there’s various plugins for all the other browsers that do that too
2017:01:25 03:34:53 bbloom but since the “preamble” in the generated pages have different structure and style classes applied, only that first section gets treated as body text
2017:01:25 03:35:45 alexmiller so just some tweaks might fix that
2017:01:25 03:36:19 bbloom yup, and then people can print that view just fine
2017:01:25 03:36:34 alexmiller ok, if I get a chance I’ll see if I can muggle my way through it
2017:01:25 03:36:37 bbloom an aria hint might do the trick super quick too
2017:01:25 03:36:54 alexmiller is that a font?
2017:01:25 03:37:12 bbloom 🙂 ARIA = accessible rich internet applications
2017:01:25 03:37:16 bbloom it’s about accessibility
2017:01:25 03:37:19 bbloom screen readers etc
2017:01:25 03:37:55 alexmiller 🐒
2017:01:25 03:38:16 alexmiller sadly, no rock emoji
2017:01:25 03:38:46 bbloom the simplest thing to try:
2017:01:25 03:38:50 bbloom wrap the entire content of the page in a <article> tag
2017:01:25 03:39:44 bbloom specifically, change the div with class clj-content-container from div to article - and see if that does it
2017:01:25 03:39:50 bbloom i bet that it does, no impact to styles at all
2017:01:25 03:40:34 alexmiller cool
2017:01:25 03:41:23 alexmiller thanks
2017:01:25 03:41:32 bbloom no problem. hopefully that helps 🙂
2017:01:25 03:41:58 bbloom i abuse reader mode and use huge fonts for most stuff i read 🙂 i have good eye sight and hope to keep it that way!
2017:01:25 03:46:40 alexmiller I match my font size to my age
2017:01:25 04:38:37 alexmiller I will take a look in the morning
2017:01:25 05:59:14 nick Maybe this is silly but say I have a spec like this (s/def ::dog (s/keys ::req [::name ::type ::sound])) but I also want to include a key that's always something specific like :feet 4 - so if I generate that I'll get like {:name "red" :type "golden retriever" :sound "woofwoof" :feet 4}
2017:01:25 06:01:47 nick I want to do something like this:
But also have in the keys a type key that is dog-type and the actual key.
2017:01:25 06:56:17 teng Is it possible to use spec to validate maps that has keys that are not namespaced, like :name instead of :user/name?
2017:01:25 07:54:49 seancorfield @teng Yes, with :req-un and :opt-un. Although they require namespaced keys for the specs, the actual map keys are unqualified.
2017:01:25 07:55:05 seancorfield (see the guide for more details)
2017:01:25 07:55:14 teng ok, thx.
2017:01:25 07:55:46 seancorfield @nick Do you mean you'd want (s/def ::feet #{4}) as the spec for the :feet key?
2017:01:25 13:23:36 alexmiller So that will work eventually
2017:01:25 13:41:20 odinodin @alexmiller that is awesome 🙂
2017:01:25 15:13:36 alexmiller @luxbock I agree that I’d expect something like this to work. I did just notice this line in the instrument docs though: ":gen overrides are used only for :stub generation.” which honestly surprises me. it seems like they should be used also for instrumented :args gen too.
2017:01:25 15:14:13 alexmiller although maybe you should really be passing those :gen overrides in your call to check
2017:01:25 15:24:41 alexmiller yeah, you should move your gen overrides map from instrument into the check options map. also, in the :gen map, the values should be no-arg functions that return the generator, not the actual generator, so just prefix all of them with # to turn them into anonymous no-arg functions
2017:01:25 15:28:27 alexmiller once I did that, it was running, but I started getting OOMEs
2017:01:25 15:29:12 alexmiller to address that, I narrowed the collection generators which default to max size 20. I added :gen-max 3 to both the fdef :args spec and the ::operations override gen.
2017:01:25 15:29:13 luxbock @alexmiller ah yeah I think I tried :gen with check but I didn't realize that I need to wrap them in thunks
2017:01:25 15:29:19 alexmiller Then it worked
2017:01:25 15:29:42 luxbock thanks, I will play around with it a bit more later today
2017:01:25 15:30:28 alexmiller these days I generally preemptively add :gen-max 3 to all collection specs reachable by fdef :args or used in recursion
2017:01:25 15:30:51 alexmiller I think it would be a good idea to change the default there from 20 to 3
2017:01:25 15:32:16 luxbock yeah that's good to know as well
2017:01:25 15:36:19 alexmiller if you’d like to vote :)
2017:01:25 15:36:59 luxbock it feels that all the spec related testing functionality one needs to write these types of tests is quite spread out, both in terms of namespaces and where the different options for customizing the generators go
2017:01:25 15:37:53 luxbock nothing that a good utility library won't fix, but I have a hard time keeping in my head where everything is supposed to come from and how it ties together
2017:01:25 15:49:35 alexmiller well the stuff in spec vs spec.test is intentionally split to emphasize the parts that are designed specifically for testing. and spec.gen is the same - split out the part that dynamically loads and depends on test.check generators.
2017:01:25 15:49:48 alexmiller those are at this point pretty well factored in my mind
2017:01:25 15:50:22 alexmiller the gen overrides stuff is a bit messier and not consistent enough between s/exercise, stest/instrument, stest/check etc and we may still make some changes in those
2017:01:25 16:02:32 luxbock I'm writing code which I can't fully test in use until it hits the production server so I'm trying to cover as many edge cases as possible with spec and generative tests to avoid the long feedback loop of deploy + test, and it's definitely giving me a lot more confidence in that I got rid of most of the silly bugs before trying it in actual use
2017:01:25 16:05:34 alexmiller cool, that’s good to hear
2017:01:25 20:52:05 ag quick noob question how do I limit the size of a number in spec like: (s/and int? pos?)
2017:01:25 21:12:08 qqq @ag: you can pass in arbitrary function
2017:01:25 21:12:24 qqq if you also need to generate, you need to write you rown generator
2017:01:25 21:37:45 alexmiller but once you start going there, consider integer ranges
2017:01:25 21:37:57 alexmiller (s/int-in 1 100)
2017:01:25 21:38:17 alexmiller the range specs come with a better generator by default
2017:01:25 21:41:50 ag ah, right… thanks @alexmiller
2017:01:25 23:03:23 ag uh… so how do I make a spec for a key like this
:options {:on-click `~(fn [r] ,,,,)}
it contains an fn but in quoted form?
2017:01:25 23:16:02 ag I guess I need s/fdef, how do I make a spec for something like {:on-click (fn [e],,,} - function that takes one argument of anything and returns whatever?
2017:01:26 00:00:37 ag oh, another total noob question.. I need a spec that generates vector of 1 to 5 elements, each element is of another spec (map), something like this:
2017:01:26 02:02:48 alexmiller @ag for your prior question, see s/fspec
2017:01:26 16:51:54 mrcnc is it preferable to use s/assert over s/valid? so you can disable it at runtime?
2017:01:26 16:53:00 mrcnc they seem pretty similar
2017:01:26 17:17:22 seancorfield To me they have very different uses: we use spec for input validation (and some coercion) so we have error handling that says (if (s/valid? ::spec value) (do-good-stuff) (handle-error)) — that’s not the same as (s/assert …)
2017:01:26 17:18:58 seancorfield I personally don’t much like asserts — I’ve never liked them in any language — and part of that is because folks tend to turn them off in production, which is when it’s even more critical that your code doesn’t attempt to operate on invalid data. Seems backwards to me...
2017:01:26 17:44:16 mrcnc good points @seancorfield…thx!
2017:01:26 18:48:53 spieden same — usually check valid? then throw an ex-info with explain-data
2017:01:26 20:37:03 moxaj specs cannot be created at runtime, right? since the api is mostly macros
2017:01:26 20:58:27 schmee I think you can get it done with quoting and eval
2017:01:26 21:01:31 moxaj well, maybe as a last resort .. eval is evil
2017:01:26 21:03:33 bfabry specs being macros is what gives them reasonable reporting on failures, a spec created at runtime would have to either do away with a lot of the error description, or have a very clunky api. there was some talk of some non-macro versions of some of the spec api maybe in the future though
2017:01:26 21:06:18 spieden seems to guide the intended usage as something static, too
2017:01:26 21:06:35 spieden .. in a world of dynamism =)
2017:01:26 21:15:08 tbaldridge @moxaj why is eval evil?
2017:01:26 21:18:48 moxaj @tbaldridge well, most of the time when you use it, there's a better solution (there are exceptions of course)
2017:01:26 21:19:29 tbaldridge That doesn't make it evil though 🙂
2017:01:26 21:19:34 moxaj i'm pretty sure there's a workaround for my issue which does not include eval
2017:01:26 21:19:51 moxaj well yeah, it's perfectly fine, but sometimes abused
2017:01:26 21:20:34 tbaldridge I say this because the "eval is evil" trope is mis-applied in Lisps. Eval is evil in a language like Javascript (where the phrase comes from) where eval means concat-ing strings together. In Clojure eval is akin to something like stored procs in SQL. It makes code injection really hard
2017:01:26 21:21:29 tbaldridge So all that being said, macros + eval works pretty well for dynamic specs, and done correctly you could even get pretty good source code mappings
2017:01:26 21:26:23 moxaj also, i'm targeting cljc, and eval only works in bootstrapped cljs afaik
2017:01:26 21:27:39 tbaldridge fair enough
2017:01:27 03:58:35 wei could someone point me to some real world examples of conform? I have a hunch that it would be very useful for me, especially with specs like s/or, but I haven’t found the output to be especially easy to adapt, for example in a cond statement.
2017:01:27 07:30:01 luxbock is there some existing function that gives me the same value that the predicates/specs in :fn receive?
2017:01:27 07:30:20 luxbock I am assuming no, but it's probalby not too difficult to create
2017:01:27 08:28:46 luxbock this seems to work
2017:01:27 12:35:41 nblumoe Hey, what are uses for the :opts key spec on maps? As additional keys are allowed anyways and all present keys are being validated even if they are not in the explicit spec there does not seem to be any relevance for validation. Is it for generation? Something else I’m missing?
2017:01:27 13:54:00 schmee nblumoe AFAIK it’s for generation and as documentation
2017:01:27 13:57:39 nblumoe The guide says this: “ When conformance is checked on a map, it does two things - checking that the required attributes are included, and checking that every registered key has a conforming value. We’ll see later where optional attributes can be useful. Also note that ALL attributes are checked via keys, not just those listed in the :req and :opt keys. Thus a bare (s/keys) is valid and will check all attributes of a map without checking which keys are required or optional.” I do not know what “We’ll see later where optional attributes can be useful” is referring to
2017:01:27 14:45:17 alexmiller They are used in generators (as well as serving as doc)
2017:01:27 14:51:44 alexmiller @luxbock the :fn spec receives a map {:args …, :ret …} where the values are the conformed values of the args and ret specs. you don’t need that get-fspec-attr - you can call (s/get-spec sym) to get the fspec and that supports ILookup, so you can (:args (s/get-spec sym)) to get (for example) the args spec.
2017:01:27 14:52:57 luxbock @alexmiller: thanks, yeah figured there might have been a better way
2017:01:28 00:12:26 adambros @alexmiller getting weird behavior when trying to conform this:
This also only shows up if there are 2+ numbers in the 2nd or later ::thing
AND the problem goes away if I make :a not optional…
is this at all related to http://dev.clojure.org/jira/browse/CLJ-2003 ?
2017:01:28 00:33:20 ag is it possible to “redef” the spec? if I have a spec that depends on other spec and that one uses a function as a predicate and I want to predicate to be something else in the test
2017:01:28 00:34:35 hiredman @adambros that looks like what phil brown mentions in the comment on that, but alex says it isn't actually related to that issue and he should open a new issue for it
2017:01:28 00:38:40 hiredman a cons somewhere there should be a concat
2017:01:28 00:42:52 alexmiller @adambros not sure if that’s the same or not, feel free to log
The last call does not terminate, just keeps heating my CPU. When removing the :kind vector? constraint from the spec ::bar everything works fine. With :kind list? I also get the erroneous behaviour.
2017:01:28 09:42:34 nblumoe Wondering If I am doing something wrong, if this is a bug and if it was filed on Jira already
2017:01:28 09:44:16 schmee nblumoe try adding :gen-max 3 to the ::bar definition
2017:01:28 09:44:45 schmee it could be that it is generating some crazy big collections
2017:01:28 09:45:40 nblumoe thanks, but still the same issue
2017:01:28 23:22:05 leongrapenthin @nblumoe If you don't provide :into [] to s/coll-of spec will generate a vector every time just to call empty on it to then generate the stuff into it. Apparently these "generated constructor vectors" explode and you can't control their sizing. So you have to provide :into. @alexmiller Is this intended or ticketworthy?
2017:01:28 23:27:27 gfredericks sounds like a ticket to me
2017:01:28 23:29:04 alexmiller Not sure I understand all that without looking at the code but doesn't sound right
2017:01:30 20:32:48 mrg Is there a good way to generate an increasing number as id in a variable-length collection?
2017:01:30 20:32:51 mrg i.e. i have this:
2017:01:30 20:33:44 mrg now if i generate subsegments, how would I get the first one to have the ::id 1, the second one (if there is one) the ::id 2, and the third one (if there is one) the ::id 3 ?
2017:01:30 20:40:22 alexmiller you can’t make that happen automatically, but you could build a generator that did that
2017:01:30 20:41:38 alexmiller I would create a custom generator for ::subsegments using gen/fmap - use the default generator to create, then in the fmap function replace the ::id values to have the index
2017:01:30 21:29:01 mrg I'll give that a shot @alexmiller .
2017:01:30 21:30:24 mrg How would you keep the index state? There doesn't seem to be a fmap-with-index, nor a multi-arity version where I could pass in a range or something and zip them together
2017:01:30 22:03:31 alexmiller Yeah, you can just map over the generated value and range
2017:01:30 22:25:10 mrg Do you have an example or a pointer to somewhere that would explain that?
2017:01:30 22:25:29 mrg I don't quite grok generators yet
2017:01:30 22:45:02 jr Is there a way to spec a mapargs fn? for example:
2017:01:30 23:04:01 jr where the options like :tracks are optional
2017:01:31 02:00:31 rmuslimov Hello, I have newbie question for spec. Let’s suppose I have map {"first-name" “name”} , can you please show the spec which check key (= ”first-name”) and if the is string. Thanks
2017:01:31 02:08:48 rmuslimov Thing I cannot understand here, that if I had keyword instead of “first-name” everything would be trivial:
but in practice, this only makes life difficult. there's not a way i know of to do this cleanly, as spec is not trying to promote this type of behavior
2017:01:31 09:09:15 luxbock it's quite difficult to figure out what other options stest/check accepts in the :clojure.spec.test.check/opts key
2017:01:31 09:09:58 luxbock the doc string says: "::stc/opts opts to flow through test.check/quick-check" but it's not really clear which function this refers to
2017:01:31 09:12:10 luxbock I can go to the source and see it calls stest/check-1, which calls a private function stest/quick-check which calls clojure.spec.gen/quick-check which calls clojure.test.check/quick-check
2017:01:31 09:13:02 luxbock which tells me the keys are :seed and :num-tests, so as far as I can tell :seed is the only other option it accepts
2017:01:31 09:14:34 luxbock I wish the doc-string for stest/check just told me this right away because I might not have clojure.test.check required and so I need to dig quite deep to find the information
2017:01:31 09:15:57 luxbock the same goes for s/coll-of which tells me it accepts the same options as every, rather than just telling what those options are
2017:01:31 11:16:00 thheller so I have a hard time composing specs due to its macro nature
2017:01:31 11:16:28 thheller ie. anything that creates a spec must be a macro
2017:01:31 11:17:12 thheller I can't have a function (validate-a-thing TypeOfThing) that returns a spec
2017:01:31 11:18:08 thheller it must be a macro that generates (s/spec #(instance? TypeOfThing %))
2017:01:31 11:18:42 thheller or am I missing something obvious? using instance? as a basic example my spec is actually a bit more complex than that
2017:01:31 11:29:53 thheller (defn validate-a-thing [type] (s/spec #(instance? type %)) works well enough but the error suffers as it is always fails predicate: (instance? type %)
2017:01:31 11:31:53 thheller can't find an obvious way to provide custom explain logic, don't care about the gen part in this case
2017:01:31 11:34:01 nblumoe Shouldn’t my generator be good enough to be used to check the corresponding function against it’s spec?
2017:01:31 20:05:58 joshjones this exact situation has come up several times here @uwo — unfortunately, there’s not much you can do other than what you’ve already identified, at least, none that I’m aware of. The reality is that you do not have one set of billed-charges; you have two, which means you will need to spec two, whether that’s through a multi-spec or explicit naming
2017:01:31 20:07:08 joshjones do you have more levels of nesting than what you’ve shown?
2017:01:31 21:39:05 uwo @joshjones thanks, and yes, many many levels of nesting
2017:02:01 07:32:58 Yehonathan Sharvit What are the conceptual and practical differences between s/or and s/alt?
2017:02:01 07:40:09 mike_ananev hi! if i need to transfer spec via network, how to get "source" of spec by its name?
2017:02:01 07:40:59 Yehonathan Sharvit something like s/describe?
2017:02:01 07:42:03 mike_ananev @viebel yeah, but any anonymous fn in spec is not readable. i need to get source of spec, like I see in IDE
2017:02:01 08:29:22 dergutemoritz @viebel s/alt is a regex op, s/or is not
2017:02:01 08:30:13 Yehonathan Sharvit @dergutemoritz But if I put s/alt inside an s/cat expression it seems to work as expected.
2017:02:01 08:30:33 dergutemoritz @viebel You mean s/or?
2017:02:01 08:32:37 Yehonathan Sharvit yeah sorry for the typo: But if I put s/or inside an s/cat expression it seems to work as expected.
2017:02:01 08:34:06 dergutemoritz @viebel The difference is that a s/cat within an s/alt will be spliced whereas within an s/or it won't
2017:02:01 08:34:15 dergutemoritz Let me cook up an example
2017:02:01 08:37:26 dergutemoritz @viebel Here you go - hope it's clear, couldn't come up with a more concise example 😞
2017:02:01 08:57:58 Yehonathan Sharvit Thanks a lot @dergutemoritz !
2017:02:01 08:58:10 Yehonathan Sharvit Now it’s 100% clear!
2017:02:01 08:58:23 dergutemoritz You're welcome @viebel, glad it helped!
2017:02:02 21:14:18 villesv Hi, I am wondering if maybe I have stumbled across an issue in clojure.spec. I have perused the issue list but have found nothing that I think describes the issue at hand for me.
2017:02:02 21:17:05 villesv My point being, although I am checking (doesn't matter if I use s/explain-str, s/assert etc) for :some.ns.2/m - clojure.spec ends up also validating another spec.
2017:02:02 21:18:34 villesv That is, although :some.ns.1/m is in there and not conforming to spec - I did not expect it to be checked for. I also noticed that this happens both when I use :req-un and :req for :some.ns.2/m
2017:02:02 21:18:58 villesv I am thinking this is a bug
2017:02:02 21:37:00 villesv Hmm, no seems I am mistaken, I found this in the spec source docs for clojure.spec/keys:
> In addition, the values of all namespace-qualified keys will be validated
> (and possibly destructured) by any registered specs. Note: there is
> no support for inline value specification, by design.
How have I not stumbled across this before?
2017:02:02 23:30:29 ghadi dunno -- it's in the guide too , I'm pretty sure. Best way to understand is that namespaced keys have program-global meaning
2017:02:03 02:38:06 joshjones @fnil, it's always been this way...
2017:02:03 07:32:26 luxbock finally starting to get a hang of how to use instrument effectively while doing interactive development via the REPL, :replace is very handy in isolating parts of a long pipeline of functions that build up the computation step by step
2017:02:03 07:38:58 luxbock having a version of enumerate-namespace that recursively enumerates all the internal functions called by some top-level function would be quite handy
2017:02:03 07:39:41 luxbock not sure if that already exists in some library, if anyone knows let me know
2017:02:03 09:08:49 villesv @ghadi @joshjones You are correct and I am embarrassed 😄 I think the case at hand may have contributed to the confusion. I was applying specs to a deep, previously unspec:ed map where both spec:ed and unspec:ed keys were present.
2017:02:03 09:50:58 chrisblom i'm converting the json schema for AWS cloudformation to clojure.spec
2017:02:03 09:51:16 chrisblom this schema has a few reflexive references
2017:02:03 09:52:46 chrisblom i'm running into problems when converting it to spec
2017:02:03 09:53:51 chrisblom i have a spec of form (s/def :foo/bar ...) where :foo/bar is used somewhere in the ...
2017:02:03 09:54:36 chrisblom this gives me an Unable to resolve spec error, what's the recommended way to do such circular definitions in spec?
2017:02:03 10:38:28 villesv Oh, I would love to have some sanity brought on to cloudformations json. Love the service but I stumble on the JSON details all the time
2017:02:03 11:09:48 villesv How is :foo/bar used recursively? Doesn't something like:
2017:02:03 14:59:18 joshjones but I don't know if you're spec-ing a map, a collection, or what ? you can also do recursive definitions for collections, etc., but if you can be more specific about the structure you're trying to spec..
2017:02:03 15:49:23 chrisblom hmm thanks guys, it's working now, not sure what i was doing wrong
2017:02:03 15:55:03 prepor Hello. I think that fully namespaced keywords without full application's namespace should be considered as very bad practice (:message/state for example) at least in library code. But why the official guide is full of such usages?
2017:02:03 15:59:03 villesv @prepor Why do you think so?
2017:02:03 16:00:56 joost-diepenmaat I think I agree @prepor. since specs are global I would say that libraries should never use keys outside of the namespaces already in the lib
2017:02:03 16:01:11 joost-diepenmaat to prevent clashes with application code and other libs
2017:02:03 16:01:51 frank I think it makes it a tad harder to find the namespace where the s/def lives. Also, if you're referring to the specs in other namespaces, it's harder to figure out which namespace you need to require in order to have those specs available
2017:02:03 16:02:07 prepor @fnil because of @joost-diepenmaat explanation, yes
2017:02:03 16:08:04 villesv All true, although I find the flexibility great.
2017:02:03 16:09:09 joost-diepenmaat I’m using single-keyword namespaces in an application right now and it’s very neat. Writing libraries generally means sacrificing some easyness for interoperability
2017:02:03 16:09:19 villesv I definitely agree that it can be hard to locate where a certain keyword was defined (or sometimes redefined)
2017:02:03 16:21:47 prepor @joost-diepenmaat a boundary between application and library is often not clear. today it's application code, and tomorrow you can decide to split it into libraries to use inside your microservices.
2017:02:03 16:22:35 joost-diepenmaat True
2017:02:03 16:23:55 prepor so, personally I decided to not use "short namespaces" at all. and for me it's strange that official guide forces this usage. maybe I don't understand something...
2017:02:03 16:35:15 seancorfield The guide uses ::stuff in most places tho` which is qualified by the full namespace. I think the other places in the guide are just examples -- certainly not "forcing" any particular usage on you.
2017:02:03 16:36:09 joshjones The spec guide seems to be aimed at learning how spec works, and as such, brevity and simplicity are key -- adding too much info on namespaces makes it more difficult to learn. ::mykey is short, and simple ... at any rate, if you want to do something like this, I think it's a good way to shorten the code while still being namespace-safe:
2017:02:03 16:39:52 prepor When something is used in guide without red warnings — it's "forcing" )
2017:02:03 16:42:31 prepor @joshjones yes, I know about create-ns / alias
2017:02:03 16:42:51 seancorfield It's a guide not a set of rules. You're reading too much into it.
2017:02:03 16:43:53 seancorfield If anything I'd say the core team don't provide enough guidance. They really don't force their views on anyone.
2017:02:03 16:47:36 prepor @seancorfield will you think the same when you catch a bug because two of your dependencies have name conflicts without any warnings because the official guide guided their authors to not always use fully qualified keywords? 🙂
2017:02:03 16:50:30 prepor It's near the only way how you can broke others code in clojure ecosystem in non tricky way, I think
2017:02:03 16:56:44 seancorfield When the core team talk about spec, they're pretty clear about using namespaces to prevent exactly that bug. Again, you're treating a learning guide as a rule book. There doesn't seem much point in this discussion if you're set on blaming the core team for your mistakes 😸
2017:02:03 17:05:22 prepor hm. wrong tools force me to make my mistakes. is there no chance that core team did something wrong? don't think so. I'm sure that its ok, then somebody asks questions about core team decisions.
2017:02:03 17:06:03 prepor I'm trying to find some big real world applications on github which uses core.spec but can't 🙂
2017:02:03 17:50:15 sattvik One option I have found that can work is using the :full.ns.no.alias/kw form can help work around circular dependencies.
2017:02:03 18:18:30 seancorfield @prepor We use spec in production fairly heavily — and I know there are other companies already doing so — but of course you won’t find commercial application code on GitHub 🙂
2017:02:03 18:19:30 seancorfield @prepor And remember the saying: “It’s a poor craftsman that blames his tools.” (not sure whether that saying originates but I remember hearing it a lot growing up)
2017:02:03 18:54:24 fenton can you spec a variadic function?
2017:02:03 18:56:40 sattvik Sure. You use the regex specs to do so.
2017:02:03 19:00:21 sattvik You can do something like:
2017:02:03 19:04:15 fenton ok... I haven't looked into this much but wondering about doing pattern matching with core.match and spec to be like a guard or something...
2017:02:03 19:16:06 alexmiller @prepor the guide uses shorter names and :: kws for readability. it would be reasonable to add a side bar explaining this.
2017:02:03 19:50:23 prepor @alexmiller cool, thank you! the only thing which I imagine to catch (and debug) such errors automatically is "production mode" for clojure.spec which forbids redefinition of specs in explicit way
2017:02:03 19:59:35 spieden @fnil we generate cloudformation templates as EDN and serialize to JSON, works great
2017:02:03 20:00:08 spieden @chrisblom that sounds like a great project. huge surface area, though
2017:02:03 20:08:59 villesv @spieden very cool! I have had such a thought but have not fully grasped cloudformation and its template language yet
2017:02:03 20:09:37 spieden it’s a bit of a curve but has been a great workhorse for us
2017:02:03 20:09:51 spieden can’t imagine maintaining a template by hand, though
2017:02:03 20:10:06 spieden we have lots of functions that build up common structures, etc.
2017:02:03 20:10:45 spieden without DRYing things out like this it’d be crazy town
2017:02:03 20:10:59 villesv Spec should be great for it
2017:02:03 20:13:12 spieden yeah for sure. they have their own (hosted) validation operation, but it misses things and works against the JSON representation
codebreaker=> (s/exercise (:args (s/get-spec `score)))
FileNotFoundException Could not locate clojure/test/check/generators__init.class or clojure/test/check/generators.clj on classpath. clojure.lang.RT.load (RT.java:458)
I am using Clojure 1.9.0-alpha14 in lein repl
2017:02:03 22:41:41 jr add [org.clojure/test.check "0.9.0"] to your (dev) dependencies
2017:02:05 17:32:05 nicola Hello, i need some verification from experts! I’m thinking about refactor my json-schema library to clojure.spec generator from json-schema definitions. The problems i see: this will be generation based on macros (or there is another way?) and some json-schema could come from users - but specs are not garbage collected (some sandbox?) ? Does anybody have spec generation experience?
2017:02:05 17:34:08 nicola Or may be this is a bad idea at all?
2017:02:06 06:22:18 Yehonathan Sharvit I’m trying to understand the difference between s/coll-of and s/every.
> Note that 'every' does not do exhaustive checking, rather it samples coll-check-limit elements. Nor (as a result) does it do any conforming of elements. 'explain' will report at most coll-error-limit problems. Thus 'every' should be suitable for potentially large collections.
2017:02:06 06:23:07 Yehonathan Sharvit What does it mean that every doesn’t do any conforming of elements?
2017:02:06 06:23:41 Yehonathan Sharvit (s/conform (s/every number?) (range 1000)) seems to do the conforming pretty well!
2017:02:06 08:52:07 dergutemoritz @viebel Try with a spec that conforms to a different value
2017:02:06 10:47:07 Yehonathan Sharvit I’m reading (with enthusiasm) "CREATING A SPEC FOR DESTRUCTURING” http://blog.cognitect.com/blog/2017/1/3/spec-destructuring. Everything is pretty clear except one point related to s/merge at the end of the article
2017:02:06 10:48:13 Yehonathan Sharvit I don’t understand why inside ::map-bindings, we have the :msb part
2017:02:06 10:48:45 Yehonathan Sharvit AFAIU, the :msb part relates only to ::map-special-binding
2017:02:06 12:16:50 Yehonathan Sharvit any idea @dergutemoritz or @alexmiller ?
2017:02:06 12:24:00 dergutemoritz @viebel It makes sure that no other keywords than the ones in the set are present
2017:02:06 12:24:31 dergutemoritz ::map-special-bindings is defined in terms of s/keys which is an open key set by design
2017:02:06 12:25:07 dergutemoritz At least that's what I surmise is the motivation. Being the author of that post, @alexmiller might have more insight 🙂
2017:02:06 12:25:59 Yehonathan Sharvit That makes sense. But I think it would be clearer if (s/tuple #{:as :or :keys :syms :strs} any?) was part of ::map-special-binding. What do u think @dergutemoritz ?
2017:02:06 12:31:45 dergutemoritz Not sure I would find it clearer that way
2017:02:06 12:32:42 dergutemoritz The way it is now makes the ::map-special-bindings spec a tad more widely usable I guess
2017:02:06 12:35:54 Yehonathan Sharvit ok. makes sense
2017:02:06 18:23:08 alexmiller making ::make-special-bindings reusable is not really a goal
2017:02:06 18:23:38 alexmiller the tuple check can’t be part of ::map-special-binding b/c that spec is a map spec which is merged with the rest
2017:02:06 18:28:39 alexmiller I think it’s probably a reasonable criticism that this spec is too restrictive (from a future-looking point of view) in narrowing to just this set of options as we know it now.
2017:02:06 18:30:38 alexmiller the intention is really to segment the space of possible tuple forms to bindings (which are simple symbols, vectors, or maps), qualified keywords whose names are “keys” or “syms”, or special options. Here I’ve listed a concrete set of options but lost the open-ness of something like keys. That could be addressed by replacing the #{:as …} with something more generic like simple-keyword?.
2017:02:06 19:11:35 bbloom that makes some sense to me ^^ another option would be to use conformers to rewrite the :as in to a ::as node, so that it’s value can be properly checked to be a simple-symbol, for example
2017:02:06 19:12:32 bbloom that seems like the the way to recover the openness with all the benefits of the global namespaced keys validation
2017:02:06 19:21:12 Yehonathan Sharvit @alexmiller I don’t understand why in this case, the open-ness is desirable. I mean if someone passes :wrong-keys (instead of :keys) to a let statement, it should be invalidated...
2017:02:06 19:22:22 alexmiller @bbloom no desire to use conformers for something like this
2017:02:06 19:22:51 alexmiller @viebel the same reason openness is desirable elsewhere (see Rich’s keynote from the conj)
2017:02:06 19:23:19 alexmiller that is, if you specify a constraint, then removing the constraint later is a breaking change, not an additive change
2017:02:06 19:23:34 bbloom @viebel just to make up some straw man extension, consider if you wanted to add some metadata to the match
2017:02:06 19:24:10 bbloom you don’t want the spec to reject that types
2017:02:06 19:26:35 bbloom @alexmiller sorry, i was thinking about :as in vectors - i think conformers make sense there
2017:02:06 19:26:44 bbloom for maps, it’s already a map 😛
2017:02:06 19:27:50 bbloom the :msb form is interesting tho - why not just use a :req-un for the :as key?
2017:02:06 19:28:12 bbloom i must be missing some subtlety
2017:02:06 19:29:43 bbloom oooh, you also do that, this is about accumulating extra information in the parse - ok, i think that makes sense.... sorry for thinking aloud here
2017:02:06 19:29:51 alexmiller yes
2017:02:06 19:30:11 alexmiller this is a particularly tricky instance of what has been called a hybrid map
2017:02:06 19:30:25 alexmiller which has both a k/v spec + a keys-type spec
2017:02:06 19:31:05 alexmiller the tricky part being the :<ns>/keys and :<ns>/syms which are like keys but only semi-fixed
2017:02:06 19:31:08 bbloom yeah, my experience w/ hybrid maps has been that they are more trouble than they are worth except for very limited syntax use cases
2017:02:06 19:31:19 alexmiller nonetheless, they exist :)
2017:02:06 19:31:22 bbloom indeed
2017:02:06 19:31:27 bbloom i’ve run in to the ::keys bug
2017:02:06 19:31:34 alexmiller yeah
2017:02:06 19:31:53 bbloom it is not obvious what to do about that 🙂 so i just renamed my thing, heh
2017:02:06 19:32:07 alexmiller well, I have some ideas about that but haven’t had a chance to put a patch on there
2017:02:06 19:32:16 alexmiller but basically it needs to not use s/keys
2017:02:06 19:32:39 alexmiller as that has behavior that we don’t want in this particular case
2017:02:06 19:33:01 alexmiller namely, matching all keys as specs
2017:02:06 19:33:10 bbloom yeah, i think the patch i saw there was like a flag to disable the nice open validation of keys - which seems like the wrong way to go about it. easier to factor out the parts you want and then call the underlying part directly
2017:02:06 19:33:24 bbloom ie composition over parameterization
2017:02:06 19:33:28 alexmiller that patch is dead, not doing that
2017:02:06 19:33:48 bbloom i assumed that 🙂
2017:02:06 19:33:57 alexmiller and I think I rewrote the ticket to remove that as an option
2017:02:06 19:34:08 bbloom cool - i have only been following at a distance
2017:02:06 21:47:55 hiredman it seems like you can't use spec to spec a map like {::results ... ::type ...} where the value associated with the key ::type determines what spec is used against the value associated with the ::results key
2017:02:06 21:49:16 hiredman it seems like the best you could do would be something with multi-spec, with a different ::results spec (registered under a different key) for each possible value of ::type
2017:02:06 21:57:16 hiredman I am trying to spec what I have been thinking of as a result set, a map with a structure like {::results [...] ::next ...} where ::next is used for pagination, and the items under ::results will vary with the query that was run. my first thought was to slap a type tag on the result set, and use a multi-spec, but that doesn't work.
2017:02:06 22:15:26 bbloom hiredman: we’ve talked about context sensitivity a bunch of times here in the past - you have a number of options
2017:02:06 22:15:35 bbloom none of them particularly great
2017:02:06 22:16:13 bbloom the simplest one is to just call the spec you want yourself
2017:02:06 22:16:31 hiredman too opaque
2017:02:06 22:16:41 bbloom well it depends on your use case
2017:02:06 22:16:51 hiredman yeah, for mine it is too opaque 🙂
2017:02:06 22:16:56 bbloom heh, ok
2017:02:06 22:17:03 bbloom i mean, it’s just an s/conform call
2017:02:06 22:17:27 hiredman I think I will just have a bunch of slightly different specs, remove any kind of polymorphism
2017:02:06 22:17:38 bbloom yeah, so that’s the next simplest solution
2017:02:06 22:17:43 hiredman for now, or until something better comes up
2017:02:06 22:17:48 bbloom one key for each possible type and then use multi-spec
2017:02:06 22:18:01 bbloom so you have like ::foo-results and ::bar-results
2017:02:06 22:18:03 hiredman I am generating documentation and json schema from these specs
2017:02:06 22:18:07 hiredman yeah
2017:02:06 22:18:41 hiredman well, easiest, since consumers consume json, is to differ based on namespace, which just disappears in the json encoding
2017:02:06 22:18:59 bbloom heh, well i guess that works 😛
2017:02:06 22:32:12 bronsa what about something like
2017:02:06 22:34:59 hiredman once you start building using functions instead of the combinators (like s/keys) explain is less useful, and any kind of treatment of specs as data becomes more of pain (like if you are walking them to generate something else)
2017:02:07 10:01:42 p-himik Is it possible, given some data, a keyword for its spec and a keyword for a simple spec that is a part of the data spec, extract all values conforming to the simple spec?
2017:02:07 10:02:26 p-himik E.g. I have a map id->val and three specs - for id, for val and for the map itself. And I'd like to extract all vals from the map.
2017:02:07 14:42:01 frank so not all of the map entries in this map conform to the map spec, but you want to find the ones that do?
2017:02:07 20:42:58 ddellacosta how to specify a map that may be recursive?
2017:02:07 21:07:51 joshjones actually, any map spec is recursive, since a map spec allows keys not specified in the spec
2017:02:07 21:08:16 joshjones but for clarity's sake, you may want to put the spec name in the :opt portion of the map spec
2017:02:07 21:08:45 ddellacosta it’s recursive perhaps, but not useful if it doesn’t validate anything about the map, is it?
2017:02:07 21:09:03 joshjones it will validate anything given in :req etc
2017:02:07 21:13:50 ddellacosta that works if your keys are namespaced keywords
2017:02:07 21:13:55 ddellacosta but I have string keys here
2017:02:07 21:14:14 joshjones i see
2017:02:07 21:14:46 ddellacosta but, I’m filing all of this away for the time I do need such a construction…thanks @joshjones and @hiredman , very helpful
2017:02:07 23:48:51 ag How do I generate map with keys and values that correspond to a given (s/keys) spec?
let's say I have (s/keys :req-un [::id ::name] :opt-un [::money]) based on that I should generate vector like:
2017:02:07 23:50:28 hiredman having the same key in req and opt doesn't make sense
2017:02:07 23:50:51 bfabry ^ that is true
2017:02:07 23:51:31 ag @hiredman sorry my mistake. fixed it
2017:02:07 23:52:03 hiredman maybe close the paren too while you are at it
2017:02:07 23:53:10 hiredman it depends sort of on what purpose you are generating them for, but if you generally just want to poke at them, exercise is a way to generate data for a give spec, s/keys created or otherwise
2017:02:07 23:55:15 ag oh, I guess I got it all wrong… hold on..
2017:02:07 23:57:03 ag I need to generate map where the keys are “randomly selected” out of the s/keys spec and the corresponding values also generated according to the spec
2017:02:07 23:58:18 bfabry sounds like you want another s/keys spec with all the keys optional
2017:02:07 23:59:16 ag well, can I read the keys out of the previously defined s/keys spec?
2017:02:08 00:00:53 ag or somehow tell test.check.generator/map or hash-map that I want keys to be out of the spec?
2017:02:08 00:02:34 bfabry you should be able to read the keys out using s/form, assuming it's a simple spec
2017:02:08 00:05:35 bfabry (being able to drop into a 1.9 repl in sub-seconds rules, thank you planck)
2017:02:08 00:09:54 ag hmmm, interesting. thanks!
2017:02:08 00:24:19 ag @bfabry I did not know about s/form, just played with it - it’s nice. thanks again!
2017:02:08 00:25:22 qqq what is the standard way of memoizing spec checks? I have a recursive data structure (say a tree), and I don't want it to have to verify the property holds on subtrees every time -- I want to somehow 'cache' "this node passed spec xyz" somehow
2017:02:08 00:26:28 qqq when memoizing, can we somehow do it with "weak references" so that instead of having gc leaks, the object is thrown away when it's no longer needed ?
2017:02:08 00:28:44 qqq I suspect one nice thing about meta data over memoization is that with meta data, with the object is no longer needed, and the gc gcs it, it takes away the metadata too
2017:02:08 00:28:59 qqq with memoization, I fear it'll prevent the gc from doing its work because the spec function will keep th eobject around
2017:02:08 00:32:43 hiredman why are you running spec checks over and over on a large structure? like, just don't do that
2017:02:08 00:36:34 qqq if I have input argument spec validation on, wouldn't it run the spec every time the function is called?
2017:02:08 00:37:00 qqq now, if this structure is recursive (say: this is a tree, where the sum of every subtree is a multiple of 3), then it ends up checking the entire tree every time
2017:02:08 00:38:18 hiredman my first concern with that question is it sounds like you are planning to turn on spec instrumentation outside of your test suite
2017:02:08 00:38:42 qqq that is true; is it bad to have spec instrumentation turned on in production code? if it's a costant time hit, I would not mind
2017:02:08 00:38:53 hiredman it is bad, spec is not designed for that
2017:02:08 00:39:40 hiredman I should say
2017:02:08 00:39:53 hiredman the instrument stuff in spec is not designed for that
2017:02:08 00:40:20 hiredman the intended use, as far as I understand, is to turn instrumentation on when running your test suite
2017:02:08 00:40:26 hiredman and that is it
2017:02:08 00:40:40 qqq I think assertions are good in dev code (not just in test quite). I'm trying to use spec as "ultra powerful assert" in dev code. You're saying this is bad, and I should not use spec as an assert ?
2017:02:08 00:43:39 qqq For some functions, I want to assert that their input satisfies certain pre-conditions. These pre-conditions can be expensive (checking all subtrees); therefore, I'd like a way to cache this.
Now, given that I want to do the above, can I do this with spec, or would spec be a bad match?
I'm hoping that spec can do this, as I really like spec's language.
2017:02:08 00:43:39 bfabry imo many many people will use spec the way you describe, but there are some things to be wary of: it will be slow. if you spec any functions that you pass as arguments those functions will be called multiple times to check that they pass the spec as part of instrumentation. instrumentation does not check return values
2017:02:08 00:44:28 qqq @bfabry: we can assume I don't need to "run specs on functions passed as inputs" -- and my only conern at the moment being simple data structures and recurisve data sturcutres // functions are way too hard
2017:02:08 00:45:23 hiredman if you don't have any automatic checking turned on, checking will only happen when you ask for it, so just don't ask for expensive checks more than once
2017:02:08 00:46:59 bfabry fwiw there's no reason you can't use spec's data description language to validate data without using instrumentation. just call s/valid or s/conform directly. you could even wrap your calls to s/valid and s/conform in something that memoizes based on the object's reference id or whatever if you really want to tweak performance of that validation
2017:02:08 00:48:34 qqq or something of that nature, this is clever
2017:02:08 00:50:00 bfabry yeah exactly, then you can stick that in :pre if that's your preferred mechanism
2017:02:08 00:50:41 joshjones if the function is in clojure.spec.test, don't use in prod
2017:02:08 00:50:49 bfabry you could even still associate the spec with the function, so you get the documentation, generation stubbing when wanted etc, just don't call s/instrument
2017:02:08 00:52:34 joshjones the spec guide under "Using Spec for Validation" gives the :pre and :post example for reference
2017:02:08 00:55:06 qqq @joshjones: found it, thanks!
2017:02:08 04:20:31 onetom @ag ^^^
2017:02:08 04:27:11 onetom Is it expected behaviour to get a clojure.spec/unknown explanation if a non-conforming spec has a custom generator, like this:
2017:02:08 04:46:02 seancorfield What sort of generator is identity?
2017:02:08 04:46:32 seancorfield (hmm, and doesn't with-gen take a nilary function that returns a generator?)
2017:02:08 04:47:43 seancorfield Yup, "Takes a spec and a no-arg, generator-returning fn and returns a version of that spec that uses that generator" -- identity does not satisfy that.
2017:02:08 04:48:29 seancorfield So (s/with-gen anything identity) doesn't make sense... you're not going to get a valid generator from that?
2017:02:08 04:52:03 seancorfield Yeah, if you (s/exercise ::a) you'll get
boot.user=> (s/exercise ::a)
clojure.lang.ArityException: Wrong number of args (0) passed to: core/identity
which is what I'd expect.
2017:02:08 04:52:31 seancorfield So the error from s/explain isn't surprising but it is perhaps a bit misleading @onetom
2017:02:08 04:53:38 onetom ah, sorry, i just threw identity in there because it's an irrelevant detail
2017:02:08 04:54:27 onetom i observed the very same behaviour in our actual app with this real generator:
2017:02:08 04:56:21 seancorfield OK, that does s/exercise...
2017:02:08 04:57:18 seancorfield Yeah, that sounds like it's worth a JIRA issue...
2017:02:08 04:57:27 seancorfield I'd expect a better message, at least.
2017:02:08 04:59:09 onetom and this is the very first time i wrote a custom generator for an actual real-world use case...
thats my generic experience with software...
sometimes im wondering im just unlucky.
what comforts me slightly is that i have a friend who is an order of magnitude "unluckier" 🙂
2017:02:08 04:59:38 onetom ok, i will make a JIRA issue. (this will be my first JIRA issue... im already worried what will happen ;)
2017:02:08 05:03:05 seancorfield Functionally-linked people are very useful in QA'ing software 🙂
2017:02:08 05:49:09 seancorfield (uses {code} around code not three backticks)
2017:02:08 05:50:39 seancorfield and {{}} instead of backticks for inline code.
2017:02:08 07:20:14 hiredman with-gen is a function, so it's arguments are evaluated, so the string? argument to with-gen is a function object, when spec is trying to find the name to report it does some stuff, which for symbols and keywords reports a good name, but for other Objects (including function objects) you get :clojure.spec/unknown
2017:02:08 08:42:37 mpenet I almost never use with-gen because of this now that I think of it
2017:02:08 10:41:57 luxbock I have some data which I can neatly define a spec for using the regexp combinators of spec, which is great because I can generate example data for free, but for the functions I'm testing it's quite important that the returned sequences are vectors rather than list
2017:02:08 10:43:22 luxbock coll-of and every allow you to define the type of the sequence, but they don't know about the structure of the sequence
2017:02:08 10:44:03 luxbock is there any easier way to force the generators to produce vectors than using a custom generator that calls vec on them?
2017:02:08 10:46:45 linuss Hey guys, I'm trying to use the simple-type generator in one of my specs, but I can't seem to get it to work. I currently have (s/def ::any (s/spec (fn[] (true)) :gen #(gen/simple-type))) but it seems that whatever permutation of the :gen value I use it returns an error
2017:02:08 11:53:11 dergutemoritz @linuss Can you paste the error you get? At any rate, the parens around (true) look wrong, that means you are calling true as a function
2017:02:08 11:56:52 linuss @dergutemoritz Ah, thanks! Yeah, that doesn't help. However, I'm still getting the following error: java.lang.IllegalArgumentException: No value supplied for key: (fn* [] (gen/simple-type))
2017:02:08 11:57:06 linuss oh, hold up
2017:02:08 11:57:08 linuss typo on my end
2017:02:08 11:57:53 linuss java.util.concurrent.ExecutionException: clojure.lang.ArityException: Wrong number of args (1) passed to: specs/fn--10638 That's the error I keep getting
2017:02:08 12:03:44 dergutemoritz @linuss Ah, right, your spec predicate function needs to accept a single argument.
2017:02:08 12:04:35 dergutemoritz So (fn [x] true) would do the trick. You can use any? instead, too, which is the core function with the same behavior.
2017:02:08 12:05:13 dergutemoritz Plus that clojure.spec has a default generator for it
2017:02:08 12:05:23 linuss Oh wow, thanks!
2017:02:08 12:06:21 dergutemoritz You're welcome!
2017:02:08 12:07:25 linuss Oh, hm, I can't seem to find any?. Could you point me to the right location?
2017:02:08 12:08:41 dergutemoritz @linuss It's in clojure.core since 1.9
2017:02:08 12:08:55 linuss ah!
2017:02:08 12:33:04 triss hey all, I’m trying to run stest/check against some functions
2017:02:08 12:33:12 triss but i get the following error:
2017:02:08 12:33:51 triss
...
{:result #error {
:cause "Could not locate clojure/test/check/generators__init.class or clojure/test/check/generators.clj on classpath."
:via
[{:type java.io.FileNotFoundException
:message "Could not locate clojure/test/check/generators__init.class or clojure/test/check/generators.clj on classpath."
:at [clojure.lang.RT load "RT.java" 458]}]
:trace ...
2017:02:08 12:35:04 triss do i need a particular dependency?
2017:02:08 14:03:53 pbaille looks like performance is an issue here, and the try catch stuff is ugly...
2017:02:08 14:04:13 pbaille but that illustrate what i am trying to acheive
2017:02:08 14:06:42 pbaille that's probably a terrible idea 🙂
2017:02:08 14:30:25 linuss Would it be possible to write a spec for a function with side-effects, like slurp?
2017:02:08 14:38:12 Tim maybe if it takes input?
2017:02:08 16:08:52 cryptorat I am trying to write a definition for “weeks in a year” and having difficulty checking that the integer is < 53.
(s/def ::valid-weeks-in-year (s/and ::non-negative-integer #(< % 53)))
It is failing with
java.lang.ClassCastException: clojure.lang.MapEntry cannot be cast to java.lang.Number
. What am I missing?
2017:02:08 16:09:56 cryptorat In a general case, how do I check that a number falls in a range?
2017:02:08 16:12:21 linuss I think your error is somewhere else, this works without issue on my end
2017:02:08 16:15:09 cryptorat oh, int-in. That looks like it will work.
2017:02:08 16:15:28 cryptorat I wonder if my non-negative-integer is the problem. Let me try with yours.
2017:02:08 16:16:35 cryptorat Yup, that is it.
2017:02:08 16:16:50 cryptorat Well I learned two things. Thank you.
2017:02:08 16:23:49 cryptorat The problem seems to lay in using
`(s/def ::non-negative (s/or :positive pos?
:zero zero?))`
instead of
#(>= % 0)
2017:02:08 16:28:01 cryptorat And since or returns a map entry....
2017:02:08 16:28:14 cryptorat Well that all makes sense now.
2017:02:08 16:54:45 joshjones @cryptorat a non-negative integer is also known as a natural integer. 1.9 has a predicate for this, so you can just use nat-int? as your predicate, although for your case, as @ghadi said, int-in is probably more appropriate since you need an upper bound
2017:02:08 16:57:13 cryptorat My understanding is that whether or not zero is included in the set of natural numbers can be debated. I figured best to avoid that in case someone changes their mind. Too cautious perhaps.
2017:02:08 16:58:52 joshjones this is a good point — i doubt the definition will change in the clojure universe but good catch nonetheless. but as you have already seen, (s/int-in 0 53) is much better anyway for your case
2017:02:08 17:18:18 akiel If you have two maps where map :a has :a/id and map :b likes to reference a particular map :a. How would you call the key in :b? Would you just use :a/id or would you create a new key called something like :b/a-ref?
2017:02:08 17:33:28 joshjones @akiel an example of the two maps would help clarify
2017:02:08 17:37:43 joshjones what do you mean “references” ?
2017:02:08 17:38:12 akiel a big disadvantage of using :a/id also for such kind of references is, that not every map containing an :a/id can be considered to be an :a.
2017:02:08 17:38:47 akiel by references I mean the same what in relational databases forein keys do
2017:02:08 17:39:33 akiel I can’t embed the referenced map directly, because that would blow up the data
2017:02:08 17:44:31 akiel To make it more spec relevant - I ask because in spec keywords are used as names for something like types and that keywords are also used in maps to name something like attributes.
2017:02:08 17:47:52 dergutemoritz @akiel I'd go with a different name, i.e. the :b/a-ref version. Because a reference to a thing is not the same as the thing itself after all.
2017:02:08 17:51:33 dergutemoritz Then again, embedding the data directly shouldn't really have that much of an impact if you have the thing that is pointed to in memory anyhow
2017:02:08 17:55:25 akiel @dergutemoritz I lean also towards using :b/a-ref. Regarding embedding directly you are right, it won’t cost memory. There I was wrong.
2017:02:08 17:56:27 akiel @dergutemoritz But it would cost on wire. I need to transport that data over wire.
2017:02:08 17:56:59 dergutemoritz I see
2017:02:08 17:59:32 dergutemoritz Another reason to name it :b/something is that you can then use a name that describes the relationship. E.g. if :a is :person and :b is :book then you could have (s/def :book/author :person/id) which I'd say would even justify leaving off the -ref suffix.
2017:02:08 17:59:51 dergutemoritz @akiel ^
2017:02:08 18:00:17 akiel @dergutemoritz Yes you are right - role names.
2017:02:08 18:03:30 akiel @dergutemoritz A related thing: Would you go for all keys in a person to start with the namespace :person or would you also use other common keys in a person? Like :person/name, :book/name vs. just :common/name.
2017:02:08 18:09:57 dergutemoritz @akiel I don't know, I think that's subject to an ontological debate you have to have with your domain experts 🙂
2017:02:08 18:11:01 dergutemoritz Note that you could have both in a way: (s/def :common/name string?) (s/def :person/name :common/name)
2017:02:08 18:11:49 dergutemoritz That way you could attach additional meaning to :common/name and also have it influence :person/name
2017:02:08 18:12:36 akiel @dergutemoritz The domain is given in my case. But I just think about map keys in relation to specs. With 20 different names, you end up with 20 specs for names which are all the same. I’m not sure if thats a bit of an antipattern now regarding to spec.
2017:02:08 18:13:06 dergutemoritz It really depends on your domain
2017:02:08 18:13:50 akiel @dergutemoritz Than you have specs like :common/name which are never used as keys in a map. That doesn’t have to be a problem - just thinking about it.
2017:02:08 18:14:25 dergutemoritz Yeah, that would be an abstract spec probably
2017:02:08 18:15:14 dergutemoritz I guess the question is whether the concept of a name is universal in your domain or specific to each entity. Or maybe a mixture of those.
2017:02:08 18:15:57 dergutemoritz But I don't feel like I've fully figured this out, yet, either
2017:02:08 18:19:10 akiel For example I have a transaction type which can have values like :insert or :update. It has the same meaning and the same values for each entity. Should I have a transaction type for every entity or just one?
2017:02:08 18:26:19 dergutemoritz If it has the same meaning in all contexts, then I'd go with a single one
2017:02:08 18:27:45 akiel @dergutemoritz Thanks, that sounds reasonable. Same name for the same thing. 🙂
2017:02:08 18:27:54 dergutemoritz 👍
2017:02:08 18:28:18 dergutemoritz At least that's my current understanding of things 🙂 If anyone else has another interpretation, I'm all ears
2017:02:08 18:28:53 akiel That's why I liked to discuss a bit on that matter.
2017:02:08 23:00:43 seancorfield Was that meant for a different channel @zane ?
2017:02:08 23:10:14 zane Sure was!
2017:02:09 11:17:11 mike_ananev Hello! If i have regular fn which returns some number, how to wrap it into generator for spec?
2017:02:09 11:37:54 not-raspberry Like, random number? This is pretty much not the point of generators. To my understanding, you're supposed to extend some existing generator by using it as a seed for your generator.
2017:02:09 11:39:11 not-raspberry look at clojure.spec.gen/fmap
2017:02:09 11:44:13 mike_ananev @not-raspberry i know about fmap. Can you provide some example how to use it in my question? I have fn (e.g. block-number-fn) which can produce block numbers. I have a spec for block. How to express block-number-fn as gen for block spec?
2017:02:09 11:45:31 not-raspberry what's a block number?
2017:02:09 11:45:38 not-raspberry an id?
2017:02:09 11:45:40 mike_ananev it is objects
2017:02:09 11:45:53 mike_ananev more complex structure
2017:02:09 11:46:58 not-raspberry and without clojure.spec, how would you generate it? {:www (random) :zzz (random)}?
2017:02:09 11:47:41 mike_ananev i have a Java constructor
2017:02:09 11:51:18 mike_ananev how to wrap fn with (UUID/randomUUID) into gen?
2017:02:09 11:51:20 not-raspberry Let's limit ourselves to textual uuids.
2017:02:09 11:52:01 not-raspberry you write a spec for a character allowed in uuid, e.g. a set of those, you get a gen for free
2017:02:09 11:52:38 mike_ananev i understand it, this sounds like workaround
2017:02:09 11:52:47 not-raspberry then you write a spec for a coll of those chars with rexactly 24 chars (let's say uuids have 24 chars, i dunno)
2017:02:09 11:53:40 mike_ananev how to wrap fn above into gen?
2017:02:09 11:53:40 not-raspberry then you write your own genetrator with fmap that maps the previous generator with a function that inserts hyphens blocks with
2017:02:09 11:54:22 not-raspberry you miss the point - don't use null-ary functions in specs. That makes them impossible to reproduce.
2017:02:09 11:54:46 mike_ananev ok
2017:02:09 11:55:33 mike_ananev be reproducable is mandatory?
where do-something-funny-with-string is not funny itself but the results of it should be. Also it's pure.
2017:02:09 11:59:47 not-raspberry Is reproducible mandatory? Nothing is pretty much. But it's inconvenient to stand out.
2017:02:09 12:00:31 not-raspberry I haven't found an easy way to write "non-pure" generators.
2017:02:09 12:01:00 not-raspberry you can always discard the argument the function passed to fmap gets.
2017:02:09 12:01:23 not-raspberry But the API overall seems to have been designed to discourage that.
2017:02:09 12:09:04 gfredericks reproducibility and shrinking are the two big features you get by constructing generators using the combinators
2017:02:09 12:24:10 mike_ananev @not-raspberry thanks. went to "hammock"
2017:02:09 12:25:57 gfredericks @mike1452 to address your specific example, I would generate a UUID by generating a pair of longs and calling the UUID constructor
2017:02:09 12:26:23 gfredericks which is essentially what the builtin UUID generator does
2017:02:09 12:27:16 mike_ananev @gfredericks can you provide source of builtin gen?
2017:02:09 12:27:24 mike_ananev link i mean
2017:02:09 12:27:57 mike_ananev i saw macros name in sources but can't find particular code for this
2017:02:09 12:29:01 gfredericks as the comments say, the jvm code there is lower level because it's faster; that wouldn't be a normal way to build generators
2017:02:09 12:40:02 mike_ananev @gfredericks thank you, I will try it
2017:02:09 15:04:31 ericnormand Hello!
2017:02:09 15:04:36 ericnormand I have a question about the clojure.spec guide
2017:02:09 15:04:43 ericnormand it says:
> The map spec never specifies the value spec for the attributes, only what attributes are required or optional.
2017:02:09 15:05:00 ericnormand but then it does look like the values are being specified
2017:02:09 15:05:39 ericnormand the next sentence is:
> When conformance is checked on a map, it does two things - checking that the required attributes are included, and checking that every registered key has a conforming value.
2017:02:09 15:06:15 not-raspberry you mean s/keys?
2017:02:09 15:06:31 not-raspberry it specifies keys that specify values
2017:02:09 15:07:56 joshjones @ericnormand so what is the question eric?
2017:02:09 15:08:07 ericnormand they seem to contradict
2017:02:09 15:08:24 ericnormand does the s/keys specify values or not?
2017:02:09 15:09:22 ericnormand it seems that it does
2017:02:09 15:10:15 joshjones no, it does not, only keys. however, when a map itself is validated or checked, the check is done to ensure that the keys specs are present, and that the values that those key specs specify are correct
2017:02:09 15:10:33 ericnormand ok, that's what I'm saying
2017:02:09 15:10:43 ericnormand the first sentence says it doesn't specify the value
2017:02:09 15:10:53 joshjones it doesn't
2017:02:09 15:10:54 ericnormand it's a direct contradiction
2017:02:09 15:11:00 ericnormand it does
2017:02:09 15:11:04 ericnormand you just said it does
2017:02:09 15:11:10 joshjones i’ll give an example, just a min
2017:02:09 15:11:30 ericnormand here's my example:
2017:02:09 15:11:51 ericnormand this fails with this message: In: [:name] val: 3 fails spec: :foo.core/name at: [:name] predicate: string?
2017:02:09 15:12:07 ericnormand the value is clearly expected to be a string
2017:02:09 15:12:56 ericnormand perhaps it's just a problem with the wording of that sentence
2017:02:09 15:13:03 ericnormand > The map spec never specifies the value spec for the attributes, only what attributes are required or optional.
2017:02:09 15:13:25 ericnormand to me, that means that s/keys only deals with existence of keys, not their values
2017:02:09 15:15:05 joshjones you quoted the guide which says: “The map spec never specifies the value spec for the attributes” — is this the part that you say is confusing?
2017:02:09 15:15:52 tbaldridge @ericnormand it means that s/keys never contains specs for the values, aside from the values spec'd by the keys themselves
2017:02:09 15:16:15 ericnormand @joshjones yes
2017:02:09 15:16:36 ericnormand @tbaldridge I see, I totally misunderstood
::mymap does not specify the value spec for the attribute above. It only specifies which key is required.
2017:02:09 15:16:55 tbaldridge My favorite analogy: A "Car" is (s/keys [::wheels ::seats ::engine]), doesn't say what a ::wheel is, it just says "A car contains these things"
2017:02:09 15:16:57 ericnormand but it implicitly says the value has to be a number
2017:02:09 15:16:58 joshjones ::mykey can change all day long, and the spec for ::mymap doesn’t change
2017:02:09 15:17:11 joshjones the spec for the KEY says the value has to be a number, NOT the spec for the map
2017:02:09 15:17:18 ericnormand ok, thanks
2017:02:09 15:17:28 ericnormand but I still think it's unclear enough to warrant a rephrasing
2017:02:09 15:17:35 tbaldridge But if you say "Is this a car?", it has to check to make sure that what you say is a ::wheel is actually a ::wheel
2017:02:09 15:17:44 ericnormand if I had trouble with it, others probably will too
2017:02:09 15:19:35 tbaldridge @ericnormand isn't that all covered in the spec guide? From the section on entity maps:
2017:02:09 15:19:48 tbaldridge "Clojure programs rely heavily on passing around maps of data. A common approach in other libraries is to describe each entity type, combining both the keys it contains and the structure of their values. Rather than define attribute (key+value) specifications in the scope of the entity (the map), specs assign meaning to individual attributes, then collect them into maps using set semantics (on the keys). This approach allows us to start assigning (and sharing) semantics at the attribute level across our libraries and applications."
2017:02:09 15:20:34 ericnormand that combined with the sentence I quoted allows for the interpretation I made
2017:02:09 15:20:44 ericnormand that keys are what matter
2017:02:09 15:20:54 ericnormand and that if you want to conform the value yourself, you can
2017:02:09 15:21:05 ericnormand please take my suggestion for what it is
2017:02:09 15:21:17 ericnormand I think it's unclear and would help others if it were rephrased
2017:02:09 15:21:27 ericnormand I don't want to argue about it 🙂
2017:02:09 15:21:38 joshjones I can understand why it’s confusing — (s/valid ::somemapspec {…}) could only check that the keys are there. then another (s/validextra ::somemapspec {…}) could validate both that the keys are present, and that the keys themselves are valid. i get it … but that’s not how it works
2017:02:09 15:22:08 tbaldridge I don't understand: https://clojure.org/guides/spec already explains this about 5 different ways in the section labeled "Entity Maps". I'm not trying to argue, I'm just not sure how it could be any more explicit.
2017:02:09 15:22:46 ericnormand it explains it five different ways, but the first way it is explained can be read the wrong way
2017:02:09 15:22:58 ericnormand and it can color how the rest are interpreted
2017:02:09 15:23:10 joshjones i’ve not heard anyone ask this question before, but I’m sure some people have wondered. more people are confused about how keys that are in the map are checked, even if they’re not given in :req and :req-un
2017:02:09 15:23:41 ericnormand thanks for your help!
2017:02:09 15:24:19 tbaldridge But I mean....that's the 3rd paragraph in the section. And your initial question about "the map spec never specifies" is tied to a code example as context.
2017:02:09 15:29:31 tbaldridge To be frank, I think "read from top to bottom taking context into account" is assumed as part of the guide. At least that's my opinion.
2017:02:09 15:30:58 mpenet yeah I got surprised by this one once (s/def ::foo string?) (s/valid? (s/keys) {::foo 1}) -> false
2017:02:09 15:31:35 mpenet kinda makes sense after all, but I guess doesn't come to mind at first
2017:02:09 15:38:30 tbaldridge Yeah, that one is a bit surprising
2017:02:09 15:38:43 tbaldridge (didn't know about it till I read the guide just now)
2017:02:09 15:39:21 mpenet it makes namespaced kw very specific to specs
2017:02:09 15:39:44 mpenet maybe too much, but that's a matter of opinion I guess
2017:02:09 15:41:46 mpenet one could argue it goes against the "specify what you want in there and ignore the extras" direction spec took
2017:02:09 15:43:45 not-raspberry specifying against extra keys makes it impossible to write backwards-compatible specs
2017:02:09 15:44:33 mpenet yeah the argument you'll hear is "create new keys"
2017:02:09 15:44:42 mpenet which is again, arguably, a bit odd
2017:02:09 15:45:42 not-raspberry (actually, future-proof, not backwards-compatible - it forces you to synchronize 2 services when adding an extra key)
2017:02:09 15:50:31 mpenet @not-raspberry not sure we're talking about the same thing tho. I was referring to the example I mentioned, not to the fact that s/keys allows extra (unespecified) keys to be valid
2017:02:09 15:51:22 not-raspberry I was just guessing the rationale for s/keys behaviour.
2017:02:09 15:51:27 mpenet that is actually a good thing as you mention. the example I mentioned, is, odd.
2017:02:09 15:52:06 tbaldridge It is an interesting topic. One one hand I do like that any value of my map will be valid after I've run it through conform.
2017:02:09 15:52:31 joshjones @mpenet why is your example surprising?
2017:02:09 15:52:39 tbaldridge On the other hand it is silent behavior (code that is executed not based on any specs I wrote in this context).
2017:02:09 15:53:05 mpenet it ran valid? on a key that's not present in (s/keys) (the spec)
2017:02:09 15:53:09 mpenet and failed
2017:02:09 15:53:16 tbaldridge @joshjones because the overall language around spec has been "specify what you want to check, and allow the rest to be whatever".
2017:02:09 15:53:45 mpenet in my book, this is a bit too surprising. Unless you read the doc very carefully you wont know about this behavior
2017:02:09 15:54:03 joshjones but the spec guide clearly states that all keys in a map will be checked for conformance
2017:02:09 15:54:29 mpenet As I said, it's arguable.
2017:02:09 15:54:54 joshjones it’s definitely a “gotcha” but it’s something that is pretty quickly found, if you’re writing specs, IME
2017:02:09 15:55:15 mpenet well the fact that @tbaldridge didn't know about it might be revealing 🙂
2017:02:09 15:55:21 tbaldridge Funny enough I've been writing specs for months, and although I've encountered this, I've never understood it until now.
2017:02:09 15:55:39 tbaldridge that's really my fault though for never reading the guide 🙂
2017:02:09 15:56:22 joshjones the usual case is, write a key spec, write a map spec that doesn’t include that key, and put an invalid value for that key — that’ll uncover the “gotcha"
2017:02:09 15:56:38 tbaldridge For better or worse, I tend to skim guides. So I like my APIs to be explicit. I can handle (s/keys :req [...] :also-check true), since I can go look up what :also-check does. It's silent behavior that can become a "gotcha"
2017:02:09 15:57:00 mpenet I never encountered it personally, we have probably thousand of lines of spec and it's 99% un-* keys
2017:02:09 15:57:14 joshjones i do agree that the behavior is not the most intuitive — but because i’m dense, i usually read the guides very thoroughly 😉
2017:02:09 15:57:16 mpenet I think I saw it mentioned here
2017:02:09 15:57:27 tbaldridge although all of this is also in the doc string, so double bad on me for not reading that either 🙂
2017:02:09 15:57:59 joshjones well, normally when you write specs for regular situations, you’re not going to put a key in the map that is not spec’d in the map spec. but when you’re learning and “tinkering around”, trying to figure things out, it’s pretty normal thing to do imo
2017:02:09 15:59:05 mpenet it makes kinda useless (other than for doc) :opt for instance
2017:02:09 15:59:16 mpenet I think that's actually what the doc says
2017:02:09 15:59:20 joshjones yes 🙂
2017:02:09 15:59:23 mpenet but odd nonetheless
2017:02:09 15:59:35 mpenet ah, and gen
2017:02:09 15:59:55 mpenet so not 100% useless
2017:02:09 16:03:32 mpenet It has one big drawback tho, if you have a "giant map" and need to only validate 1 key it will still try to do its thing on all the others
2017:02:09 16:03:55 mpenet this won't show on synthetic benchmarks like the one that's on github, but i am not sure the other frameworks do this
2017:02:09 16:04:05 mpenet (pretty sure they don't actually)
2017:02:09 16:19:49 tbaldridge :opt is also useful for gen, as I don't think s/keys gens other keys, only :req and :opt
2017:02:09 16:21:17 mpenet yep
2017:02:09 19:21:43 bbloom Just a random note on the open-maps discussion: I discovered that TypeScript allows open js objects but has an interesting feature called “excess property checks” - essentially, it only disallows extra keys at compile-time for literal values. This way it catches typos etc, but doesn’t prevent future growth. A pretty neat solution, I think.
2017:02:09 21:13:48 onetom We just had an awesome introduction to spec at the Clojure Remote 2017 conference!
Adoption will increase for sure 🙂
2017:02:09 21:54:13 alexmiller @ericnormand just skimming the backchat, is there something that needs to be clarified in the guide? if so, let me know or file an issue at https://github.com/clojure/clojure-site/issues
2017:02:09 22:03:46 alexmiller issue welcome even if not sure
2017:02:09 22:21:46 ericnormand thanks, alex!
2017:02:10 00:13:14 bsima can I use a multispec on a vector of tags, instead of just a single keyword?
2017:02:10 00:13:52 bsima how would I write such a defmulti?
2017:02:10 00:28:10 bsima
(defmulti event-type
(fn [m] (->> m :event/type (into #{}))))
2017:02:10 00:28:39 bsima here's what I came up with
2017:02:10 00:29:37 bsima turns the vector of tags into a set, so the multispec can use the set as a predicate function for the multimethod dispatch
2017:02:10 00:31:14 bsima this way I can combine multiple defmethods. If we take the example of :event/type from the official clojure spec guide, then I could do:
2017:02:10 00:32:45 bsima and so both :event/search and :event/error would be valid specs
2017:02:10 01:28:04 alexmiller Yeah, there are no constraints on multi-spec as long as you follow the pattern
2017:02:10 02:05:05 dragoncube In docs for s/keys there is “Note: there is
no support for inline value specification, by design.”. Does some expanded explanation of this exist somethere?
2017:02:10 02:41:48 dragoncube is it possible to specify spec for value referred by specific key in the map? I.e I want to say that {:id “my-id”} “my-id” should be validated by spec ::resource-id
2017:02:10 02:43:47 dragoncube currently if you use s/keys there is only implicit assignment of the spec in case the keys are same
2017:02:10 02:53:45 alexmiller @dragoncube there is some additional info about that at http://clojure.org/about/spec
2017:02:10 02:54:03 alexmiller re rationale that is
2017:02:10 02:54:20 alexmiller re the second question, the key must match the spec name
2017:02:10 02:54:51 alexmiller the idea here (also in the rationale) is that we are assigning enduring semantics to an attribute that can be used throughout the system
2017:02:10 02:55:17 alexmiller it is admittedly a different philosophy than something like Schema
2017:02:10 02:57:39 dragoncube yeah, but it is quite hard to work with the external data which shape you don’t control
2017:02:10 02:58:59 dragoncube for example, {:id “res-1” :name “My resource 1” :sub {:id “sub-res-1” “Sub-resource 1"}}
2017:02:10 03:00:12 dragoncube should I use synthesized keys for registering specs or I need to mimic data structure with packages structures?
2017:02:10 03:01:20 dragoncube It is so tempting to use :: for specs registration
2017:02:10 03:04:32 dragoncube for example above I would like to register two specs: ::resource-id and ::sub-resource-id in the same namespace
2017:02:10 03:04:54 alexmiller well you can certainly do that along with s/keys and :req-un which will only match the name part, not the namespace part
2017:02:10 03:08:54 dragoncube yes, it is possible but in this case you need either give it synthesized (not existing) namespace part like :rest-resources/id and :rest-resources.subresource/id or literally create ‘subresource’ package and define ::id spec there
2017:02:10 03:10:15 dragoncube both of these are far from ideal
2017:02:10 13:27:06 luxbock having s/vcat would definitely be quite helpful
2017:02:10 13:29:48 tcoupland especially if you could unform back to a vector
2017:02:10 13:47:04 alexmiller Agreed :)
2017:02:10 13:47:41 alexmiller Rich is as well I think but hasn't decided how it should be implemented yet
2017:02:10 13:57:20 tcoupland how do people feel about something like this:
I've been avoiding conform == coerce, after getting bitten, but this has got me thinking of dabbling again.
2017:02:10 15:19:59 luxbock I wish the :gen overrides of stest/instrument behaved the same way as the :gen of stest/check
2017:02:10 15:20:14 luxbock right now you can only use it for functions which have been stubbed
2017:02:10 17:03:45 mss hey all, quick q about how instrument and check work in cljs.
I’m running a figwheel repl in cursive. the repl is choking and timing out when I eval an instrument or check call for an fdef‘ed function . further evaluations – even of simple forms (e.g. (+ 1 1) – then result in an eval timeout.
posted in the figwheel channel already but feel like I might just be misunderstanding a core spec concept and how I’m supposed to use it
2017:02:10 18:19:41 alexmiller that sounds weird
2017:02:10 18:20:03 alexmiller instrument should not take a long time to execute
2017:02:10 18:20:36 alexmiller check certainly can take a while as it runs 1000 tests by default - you can change that by passing it additional options. although most often adjusting your generators is what really needs to happen.
2017:02:10 18:33:12 mss yeah it feels like I must be doing something wrong.
fwiw my spec is just a set of a few dozen string values (i.e. (cljs.spec/def ::my-spec #{”something” “something-else” …}).
my fdef looks something like
simply calling "cljs.spec.test/instrument `my-bool-returning-func”
or “cljs.spec.test/check `my-bool-returning-func” causes the same choking behavior
2017:02:10 18:33:43 mss don’t know if that’s idiomatic/correct or not
2017:02:10 21:42:59 assoc-in I am looking at using spec for validation for a SPA app. I am hoping that I can use the same specs for both the front and backend. I am guessing that I would place them in a cljc file, but I was wondering how the s/def would work with s referring to clojure.spec and cljs.spec. Any ideas?
2017:02:10 21:53:46 bbloom what’s the preferred idiom to “optionalize” some map with :req keys? specifically, something like (s/merge ::foo (s/or ::bar (s/keys)))
2017:02:10 21:54:07 bbloom that is, defining a new spec that is a ::foo and optionally also a ::bar
2017:02:10 22:05:53 alexmiller @assoc-in the new cljs feature for automatic aliasing means that I think you can just use the clojure namespace in the cljc file and it will work for both
2017:02:10 22:06:02 alexmiller but you should check with David in #clojurescript
2017:02:10 22:07:41 alexmiller @bbloom not sure I understand what you’re asking?
2017:02:10 22:09:04 bbloom @alexmiller to make a simplification of what i’m doing concrete: imagine i have some spec (s/def ::syntax (s/keys :req [::line ::column])) and i have some other AST nodes that might or might not have line/column information on it: (s/def ::ast (s/keys :req [::type ::op ::whatever] :opt [::line ::column]))
2017:02:10 22:09:32 bbloom however, i have it’s not just two fields, it’s quite a few and i want to basically say “an ast node is these keys and optionally a ::syntax"
2017:02:10 22:09:34 bbloom is that clearer?
2017:02:10 22:10:19 alexmiller you could define ::syntax with :opt instead and just s/merge?
2017:02:10 22:10:34 bbloom ideally not, since the parser requires everything be a ::syntax with proper line and column info
2017:02:10 22:10:44 bbloom right now i’m doing s/or with an empty (s/keys)
2017:02:10 22:10:54 alexmiller then why did you say optionally above?
2017:02:10 22:11:06 alexmiller in that case, just merge ::syntax and ::ast ?
2017:02:10 22:11:16 bbloom there’s two parts of the system
2017:02:10 22:12:57 alexmiller why do you need to do anything in the analyzer? the s/keys you already have in ::ast will validate those
2017:02:10 22:13:09 bbloom b/c it only analyzes them independently
2017:02:10 22:13:24 bbloom so if i do (assoc foo ::line 1) that should be a problem b/c there’s no column
2017:02:10 22:13:38 bbloom the s/or doesn’t really solve that problem tho
2017:02:10 22:14:33 bbloom the s/or does help generation tho
2017:02:10 22:15:51 alexmiller I don’t think there is a way to do it in keys, so you’d need an extra constraint
2017:02:10 22:16:40 bbloom i figured as much 🙂 was wondering if there was a particular pattern that has been successful for this sort of thing
2017:02:10 22:16:49 alexmiller don’t think I’ve seen it
2017:02:10 22:17:35 alexmiller I think I would leave the :opt in ::ast to get gen and and that with a custom predicate
2017:02:10 22:18:15 alexmiller the pred should filter any generated maps that don’t satisfy the pred
2017:02:10 22:18:24 alexmiller in gen that is
2017:02:10 22:18:29 bbloom hm ok
2017:02:10 22:19:22 bbloom it feels like a symmetric operation to s/?
2017:02:10 22:19:31 bbloom or maybe s/nilable
2017:02:10 22:19:33 bbloom oooh
2017:02:10 22:19:54 bbloom would that work? (s/merge (s/keys …) (s/nilable ::syntax))
2017:02:10 22:21:38 bbloom ah, sadly not
2017:02:10 22:21:52 bbloom well wait… hmmm maybe it sorta does? just doesn’t seem to do what i’d want in gen
2017:02:10 22:23:17 assoc-in @alexmiller Thanks I asked David and he said I can just use clojure.spec everywhere in the cljc file and I'll be good to go. I am really happy to see that spec's will be usable across both clojurescript and clojure so easily
2017:02:11 12:46:14 triss is there a way of referencing the size of the outer vector here?
2017:02:11 13:05:04 dergutemoritz @triss Only by wrapping the check around the outer s/coll-of with s/and AFAICT
2017:02:11 13:05:51 triss @dergutemoritz thanks man. just realized this!
2017:02:11 13:06:59 dergutemoritz YW!
2017:02:11 18:07:25 Yehonathan Sharvit What is the proper way to integrate spec.test/check with clojure.test?
2017:02:12 05:23:15 zane See the pinned items in this channel.
2017:02:12 08:23:27 Yehonathan Sharvit It’s not clear from the guide how to use spec inside deftest
2017:02:12 08:22:09 nblumoe Does anyone have experience with using clojure.spec in a scenario where numerical precision becomes an issue? I would like to have generative testings for a function whose output would match the input (except for some numerical imprecision due to matrix calculations happening). I struggle to come up with a good :fn spec which does not require crippling the generator. A simple :fn #(m/equals (:ret %) (-> % :args :matrix) 1e-6) does not work, because the generator ends up using “large” doubles which break the max. diff of 1e-6. I tried a spec using ratios but this ends up with “Couldn’t satify such-that” issues:
2017:02:12 08:23:11 nblumoe I am pretty sure I could constrain the generator in a way to make the simple check work but of course I would like to avoid doing that.
2017:02:12 08:23:13 nblumoe ideas?
2017:02:12 13:58:24 gfredericks @nblumoe what does the function you're trying to spec do?
2017:02:12 15:25:55 nblumoe @gfredericks a simplified answer would be: Principal component analysis and then reconstruction of the original data from scores and loadings. Some special twists due to the specific domain of the problem though (chemistry).
2017:02:12 15:35:02 gfredericks @nblumoe so your problem is that the algorithm breaks down at extreme values, but it would be arbitrary and artificial to impose constraints on the values in the spec?
2017:02:12 15:36:01 nblumoe no, the algorithm works fine. the only issue is specing it reliably without making the generator too specific (with “specing” I mean including generative testing)
2017:02:12 15:36:50 nblumoe (well, if by “algorithm” you mean the :fn spec predicate then YES 😉 )
2017:02:12 15:37:06 gfredericks okay, so it's not that the algorithm doesn't work at extreme values, just that the arbitrary equality threshold of 1e-6 breaks?
2017:02:12 15:38:02 nblumoe yes exactly. because this criterion is easy to break for example when using doubles in the range of 10e100 or whatever large enough
2017:02:12 15:38:24 gfredericks can you make the equality threshold a function of the input data?
2017:02:12 15:38:42 gfredericks that might have the advantage of giving you a tighter threshold at the other end as well
2017:02:12 15:39:47 nblumoe I was trying that too but ended up with unsatisfied “such-that” issues. would also need to set the threshold element wise (e.g. if a matrix contains 1e100 as well as 1e1 those should have different thresholds then)
2017:02:12 15:40:47 gfredericks what does such-that have to do with it?
2017:02:12 15:40:57 nblumoe using ratios of the inputs and returns was how I tried to circumvent the issues....
2017:02:12 15:41:18 nblumoe quote from above: “I tried a spec using ratios but this ends up with “Couldn’t satify such-that” issues"
2017:02:12 15:41:22 gfredericks asserting the ratio should be close to 1?
2017:02:12 15:41:50 gfredericks are you using such-that directly, or is it generated by spec because of a spec/and?
2017:02:12 15:42:12 nblumoe the such-that issue basically means that the generator was not able to find valid data within 100 tries, which can happen when the spec is quite specific and the search space for data is rather large
2017:02:12 15:42:34 nblumoe (lol pun not intended)
2017:02:12 15:43:28 nblumoe indeed I am using s/and
2017:02:12 15:43:30 gfredericks I'm just not seeing why the "using ratios" idea entails modifying the input spec; I'd imagine it just means modifying the :fn on the spec
2017:02:12 15:43:56 gfredericks ⇑ edited
2017:02:12 15:44:11 nblumoe yeah I was also surprised that changes to :fn could result in those issues
2017:02:12 15:44:44 gfredericks I'm 80% sure changes to :fn should be independent of the probability of such-that errors; what does your s/and look like?
2017:02:12 15:45:08 nblumoe I would have that that only :args could have such effect. is anything generated for the :fn itself maybe?!?
2017:02:12 15:45:58 nblumoe :args is a bit complicated tbh:
2017:02:12 15:47:03 gfredericks the such-that errors might be nondeterministic -- are you sure you don't get them with the non-ratio code?
2017:02:12 15:47:15 nblumoe usually that does not cause any such-that issues on generation, works pretty flawlessly accept for the aforementioned changes to :fn
2017:02:12 15:47:16 gfredericks maybe your test fails before it's likely to encounter them?
2017:02:12 15:47:42 nblumoe can test again… how many tries should I give it? 😉
2017:02:12 15:47:59 gfredericks I'm not sure what you're asking
2017:02:12 15:48:10 nblumoe sry, edited
2017:02:12 15:48:57 gfredericks I think just try s/exercise on the args spec
2017:02:12 15:50:41 gfredericks probably wrap that in (count ...)🙂
2017:02:12 15:50:47 gfredericks to avoid printing big things
2017:02:12 15:56:04 nblumoe indeed! it occasionally fails, so the issue is rather the :args spec and the generator....
2017:02:12 16:02:32 gfredericks it could be either of the s/ands, or both
2017:02:12 16:03:14 gfredericks the way s/and works as a generator is to generate something from the first spec and then filter on the rest of them; so the generator of the first spec needs to be highly likely to generate something that passes all of the predicates, or you get the such-that error
2017:02:12 16:04:05 gfredericks sometimes you can simply restructure the s/and to tailor to that, and it works out; other times the only way to get a good generator is to write a custom one for the s/and
2017:02:12 16:04:23 gfredericks since you have nested s/ands it's not obvious which one is the problem
2017:02:12 16:04:29 gfredericks I'd try exercising the inner one and see what you get
2017:02:12 16:06:09 nblumoe ok thanks, that is valuable information! I already have a custom generator for ::matrix but that is not really tailored in any way to satisfy the following predicates. will do some exercises 🙂
2017:02:12 16:20:58 nblumoe great! the outer s/and was the issue: the second predicate for :num-eigenvectors was apparently failing too often
2017:02:12 16:21:07 nblumoe thanks a lot @gfredericks !
2017:02:12 16:25:19 gfredericks np
2017:02:12 16:25:36 nblumoe that specific predicate also does only cover one case where the number of eigenvector matches either the number of columns or number of rows. that is only one specific case though (in which the input data will be reproduced). I need to think about how to cover the usage with less eigenvectors than that… the invariant for the generative testing seems to be quite tricky though then
2017:02:13 08:28:24 dbushenko hi all!
2017:02:13 08:28:36 dbushenko how to test side-effected functions with clojure.spec?
2017:02:13 08:55:33 olivergeorge That complicates things but there are some features you might like to check out. Pretty sure you can stub out functions. That might let you isolate you're logic for testing.
2017:02:13 08:55:45 olivergeorge I haven't used that feature myself.
I’ve found http://dev.clojure.org/jira/browse/CLJ-2035 which covers that but with a larger scope.
@alexmiller Would it be sensible to open a ticket just for the resolution of the predicate?
2017:02:13 13:33:13 alexmiller No, please add to 2035 if it's not already covered by the patch there
2017:02:13 13:34:02 alexmiller I think there may actually be another ticket for this already though
2017:02:13 13:36:07 alexmiller Nah, I was thinking of 2059 which is in explain-data
2017:02:13 13:38:14 cgrand ok thanks
2017:02:13 14:50:03 mss are js objects spec-able without converting to cljs data structures?
2017:02:13 15:24:02 gfredericks you can always write arbitrary predicates that check whatever you want
2017:02:13 15:30:57 sveri Hi, I have a list of maps and each map has an :idx key which is an int. Now what I want is that given one list every :idx is a unique int. Is there a way to enforce that?
2017:02:13 15:37:59 sveri Cool, even exercise respects that function, thank you @frank
2017:02:13 15:39:17 frank np!
2017:02:13 16:46:25 alexmiller @sveri the collection spec have a :distinct option
2017:02:13 16:46:52 alexmiller Although I guess that goes farther than you want
2017:02:13 16:56:45 piotr-yuxuan Hi here! Let’s say I’ve got a map of user. The key ::password must be present if and only if a user is new (the key ::new? is bound to a truthy value). Is it possible to express this with clojure.spec?
2017:02:13 17:00:47 sveri @alexmiller I have seen that, but I suspect it means distinct for the whole map and not only for the :idx key
2017:02:13 17:04:05 alexmiller Yep
2017:02:13 17:04:32 alexmiller @piotr2b: sure, use a custom predicate
2017:02:13 19:41:21 olivergeorge @mss if you can convert to clojure data structures first then you'll get access to more spec features. In particular s/keys & all the standard generators
2017:02:13 23:42:02 gfredericks @sveri test.check has distinct-by generators for that sort of thing; I don't know if spec exposes it in the collection specs or not
2017:02:14 09:42:14 dbushenko @joshjones: thanks for your answer!
2017:02:14 10:06:21 kirill.salykin Hi guys,
I am new to clojure spec, It looks like it can be used for data validation and coercion.
eg checking if if username and password provided, or if phone number in correct format.
But I don’t know how I can show proper error message for failed spec? I was not able to attach any meta data to specs.
Any advice on this? Maybe someone got similar ideas?
Thanks!
2017:02:14 10:16:21 tcoupland @kirill.salykin try explain, explain-str or explain-data
2017:02:14 10:50:48 kirill.salykin @tcoupland contains failed predicate,
but what I want is to have a custom message in explain-data
something like, (s/def ::year integer?, :message “wrong format”)
I tried to attach it as meta-data - but no luck
2017:02:14 10:57:54 olivergeorge I don't think that's possible.
2017:02:14 10:58:31 olivergeorge Would a lib focused on user input validation be more appropriate?
2017:02:14 10:58:52 olivergeorge (Vs data structure validation)
2017:02:14 11:00:03 kirill.salykin But clojure.spec looked so good to describe form objects…
I guess I’ll find some validation library, thanks
2017:02:14 11:02:22 olivergeorge I agree it's great for that.
2017:02:14 11:04:57 sveri @kirill.salykin I was at a local clojure meetup lately and @akiel presented this library: https://github.com/alexanderkiel/material-comp
I totally liked the approach and think this is the direction to go.
It basically normalizes a spec and returns a generic error message for it. Everything is based on explain-data.
2017:02:14 11:05:10 sveri There is not much in it right now, but it should get you started.
2017:02:14 14:32:14 witek What is the right way to provide a predicate which is always true? I want a spec which is a tuple of a keyword and anything. How to spec the anything?
2017:02:14 14:33:45 thheller any? was added for this purpose
2017:02:14 14:58:57 witek And how to spec a tuple with an optional element?
2017:02:14 14:59:41 witek [:a] or [:b :c] should be valid. Is this possible with a simple tuple? Or do I need an or?
2017:02:14 15:09:35 joshjones depending on whether it's nested, you can also use cat instead of tuple
2017:02:14 17:54:50 pesterhazy a wiki of "how do I spec X?" snippets would be useful
2017:02:14 18:40:12 tbaldridge oooh...something like that talk at the last Clojure/Conj where you write example data and the computer starts generating specs until it finds one that matches all your example input?
and my from-fn, which is :name runs fine:
(:name {:name "item-definition-components", :description "The list of item definition components.", :uri "{item-definition}/components", :list-of "item-definition-component", :family-id :itemdefinitions, :id :itemdefinitions/item-definition-components})
Why is that if I instrument I get:
Call to #'rest-resources-viz.extractor/resource->relationship did not conform to spec:
In: [0] val: "" fails at: [:args :from-fn :ret] predicate: qualified-keyword?
:clojure.spec/args (:name {:name "item-definition-components", :description "The list of item definition components.", :uri "{item-definition}/components", :list-of "item-definition-component", :family-id :itemdefinitions, :id :itemdefinitions/item-definition-components})
:clojure.spec/failure :instrument
2017:02:14 19:08:10 richiardiandrea it looks like it runs it in a different way when instrumented
2017:02:14 19:10:55 richiardiandrea I am now doubting, is instrument good for use at the repl?
2017:02:14 19:38:06 alexmiller instrumentation of an fdef with an fspec arg will check that arg by invoking the function you pass it with args generated from the fspec :args
2017:02:14 19:39:31 alexmiller here it’s invoking the function you passed to resource->relationship with a generated :resource/entity and getting back empty string rather than a qualified keyword
2017:02:14 19:40:48 alexmiller instrument is primarily designed for use at the repl (or during tests), so it should be good for that
2017:02:14 19:42:29 alexmiller if from-fn is a keyword, then it will return a string based on the example you have above, not a keyword
2017:02:14 19:42:44 alexmiller looks like a bug in either your code or your spec to me
2017:02:14 19:45:05 richiardiandrea no well, I forgot to include the code of the function, but it "keywordizes" the result
2017:02:14 19:45:38 richiardiandrea in fact, when I launch it with instrument the result is the above, but without it it runs fine, same example...of course I need a repro case 😄
2017:02:14 19:45:54 richiardiandrea thanks for confirming though
2017:02:14 19:46:35 hiredman well, (qualified-keyword? (keyword "")) is going to be false
2017:02:14 19:50:08 hiredman the input you pass in doesn't matter
2017:02:14 19:50:20 hiredman that error is from the generated data being used to test the spec
2017:02:14 19:50:29 richiardiandrea oh really
2017:02:14 19:50:40 richiardiandrea ok so I was completely missing that point
2017:02:14 19:50:47 hiredman yep
2017:02:14 19:50:58 richiardiandrea oh wow that explains everything
2017:02:14 19:51:04 hiredman yep
2017:02:14 19:51:41 richiardiandrea lol thanks...weirdly enough, printlns in resource->relationship where not printing as well
2017:02:14 19:52:37 hiredman I would have to look at the source for instrument, because if a spec for a function argument fails, it never actually runs the function
2017:02:14 19:52:57 richiardiandrea yeah it exactly looks like it
2017:02:14 19:53:42 richiardiandrea thanks (hired) man! You saved my chickens 😸
2017:02:15 01:34:35 seantempesta What’s the best practice for validating data submitted to a server with spec (for an API)? Should I use just check with s/valid? and use code to return an error? Or maybe use an assert and throw an exception?
2017:02:15 01:52:54 olivergeorge Your first suggestion sounds sensible to me.
2017:02:15 01:54:12 olivergeorge (In general people try to avoid relying on exceptions in pure functional code since it breaks the input->output contract. )
2017:02:15 02:12:26 seantempesta Right. That makes sense. Thanks.
2017:02:15 02:15:43 ag I need to have either bool or string, why this generates single value wrapped in a vector? (gen/generate (s/gen (s/alt :s string? :b boolean?)))
2017:02:15 02:16:33 ag eh, nevermind. I forgot that there’s s/or
2017:02:15 11:09:21 smogg Howdy.
Assuming I'm writing a spec looking like this:
how would I go about validating that both :labels and :values are of equal length in :chart?
2017:02:15 11:11:57 dergutemoritz @smogg You could do it like this (s/def ::chart (s/and (s/keys :req-un [::labels ::values]) #(= (count (::labels %)) (count (::values %)))))
2017:02:15 11:14:37 smogg ok, so a custom fn
2017:02:15 11:14:44 smogg thank you @dergutemoritz
2017:02:15 14:10:37 cryptorat So today I am trying to generate a list.
(s/def ::team (list? (s/+ ::team-member)))
I have verified that ::team-members can be generated. What obvious thing am I missing this morning?
2017:02:15 14:17:00 cryptorat hmm.. Looking at what smogg wrote (s/def ::team (s/coll-of ::team-member))
2017:02:15 14:17:06 cryptorat Seems to work for generating.
2017:02:15 14:17:42 smogg @cryptorat yeah, if you want a coll of something you'd use coll-of
2017:02:15 14:19:22 cryptorat Does that mean there is no way to determine if it is a list vs a map?
2017:02:15 14:20:54 cryptorat Or is a map not considered a collection? hmm…let me read up on coll-of
2017:02:15 14:26:09 smogg if you want a map you can use s/keys to specify the keys
2017:02:15 14:26:45 smogg if you want to specify what collection you want to get with s/coll-of you'd pass a :kind
2017:02:15 14:27:02 smogg for example: (s/coll-of integer? :kind vector?)
2017:02:15 14:27:37 cryptorat :kind yeah. I just found that.
2017:02:15 14:28:08 cryptorat Fantastic. It is all coming together nicely. Thank you for the help.
2017:02:15 14:28:13 smogg yw
2017:02:15 16:01:01 lsnape Interesting semantic differences between s/merge and clojure core merge:
I would expect :baz/bar to override :foo/bar, but AFAICT it ands the specs together for that key.
2017:02:15 16:01:54 Yehonathan Sharvit s/merge behaves like s/and
2017:02:15 16:02:04 Yehonathan Sharvit The term merge might be misleading
2017:02:15 16:02:34 Yehonathan Sharvit
merge
macro
Usage: (merge & pred-forms)
Takes map-validating specs (e.g. 'keys' specs) and
returns a spec that returns a conformed map satisfying all of the
specs. Unlike 'and', merge can generate maps satisfying the
union of the predicates.
2017:02:15 16:04:05 lsnape Thanks! What exactly is meant by union of the predicates here?
2017:02:15 16:04:33 Yehonathan Sharvit It means: union of required keys, union of optional keys
2017:02:15 16:05:00 Yehonathan Sharvit And I guess that there is some rule for handling cases where a key appears in one spec as required and in the other as optional
2017:02:15 16:05:03 lsnape Gotcha
2017:02:15 16:05:12 Yehonathan Sharvit Could you test it?
2017:02:15 16:10:49 Yehonathan Sharvit No matter what’s the order inside the merge?
2017:02:15 16:11:40 lsnape Order doesn’t matter. I guess this behaviour is consistent with the anding of the value specs.
2017:02:15 16:11:50 Yehonathan Sharvit makes sense
2017:02:15 16:12:18 lsnape Yeah, good to know 🙂
2017:02:15 16:14:58 alexmiller Merge means: the value must pass each merged spec
2017:02:15 16:15:15 alexmiller The specs are not combined or unions though
2017:02:15 16:18:30 lsnape What I was hoping for was someway to relax an existing spec that I’ve defined. For example, a new map that has blank strings initially ready for a user to fill out.
2017:02:15 16:20:11 lsnape The spec on the server requiring that these strings be filled out in the map.
2017:02:15 17:43:20 vikeri Is there a spec that specifies a sorted-map?
2017:02:15 18:37:22 spieden @vikeri not built in i don’t think, but you could just have a type checking predicate i suppose
2017:02:15 18:39:56 vikeri @spieden Alright thanks, in the end I decided to use a vector instead, better when serializing
2017:02:15 18:47:24 spieden yeah that may make your life easier overall
2017:02:15 19:56:55 alexmiller @vikeri you can do something like (s/map-of int? string? :kind sorted?)
2017:02:15 19:59:56 vikeri @alexmiller That sure looks like it, didn’t think of that you could use :kind for broader things than vector?, list? etc.
2017:02:15 20:06:08 alexmiller it’s an arbitrary function
2017:02:15 20:06:30 alexmiller however, it does affect gen - if you use, either it should gen or you should also use :into
2017:02:15 20:09:07 alexmiller I’m not sure there is some way to get the sorted map constraint in there and make map-of gen (without supplying a custom gen for the overall coll)
2017:02:15 20:12:37 spieden i seem to be hitting an infinite recursion trying to gen for a part of my spec
2017:02:15 20:12:45 spieden it’s not the only recursive definition i have — other one works fine
2017:02:15 20:15:11 viesti hum, trying to figure out this:
2017:02:15 20:16:19 hiredman somewhere in the specs for id and name or in a spec they rely on you are using s/and
2017:02:15 20:16:50 hiredman and somewhere in that s/and is a predicate that is very picky
2017:02:15 20:17:18 viesti where :lol/id is int? and :lol/name is
(s/and string? #(<= (.length %) column_size))
2017:02:15 20:17:27 hiredman the way generates work for s/and is it uses the generator for the first predicate and then filters the output using such-that
2017:02:15 20:17:31 hiredman generators
2017:02:15 20:17:59 hiredman right, so you should hook up a custom generator
2017:02:15 20:18:36 viesti hum, s/exercise works individually for both specs but not for the key spec
2017:02:15 20:18:59 hiredman sure
2017:02:15 20:19:08 viesti should I write a custom generator for the key spec
2017:02:15 20:19:14 hiredman that is going in to the weeds a little bit on what happens as you compose test.check generators
2017:02:15 20:19:49 hiredman for :lol/name
2017:02:15 20:20:29 viesti first thought that the problem was strings limited by length, but it’s probably combination of :req and :opt, since having both key specs as :opt passes s/exercise
2017:02:15 20:20:29 hiredman because you can generate random strings for a long time before you get one that is less than or equal to column_size
2017:02:15 20:21:48 viesti since this seems to work:
2017:02:15 20:22:31 hiredman this is the nature of randomness, sometimes things work that don't always work
2017:02:15 20:22:35 viesti 😄
2017:02:15 20:22:51 viesti have to take a shower now, introducing some randomness 🙂
2017:02:15 20:23:42 hiredman I don't recall the details of how the generator for s/keys works, but it may ignore :opt most of the time, or some large enough amount of the time for it to mostly work when you move the keys to opt
2017:02:15 20:30:33 spieden any thoughts on infinite recursion during spec generated gen? only difference i can see is the “call depth” where the recursion happens — it’s deeper in the case that’s going into the infinite loop
2017:02:15 21:26:35 tbaldridge @spieden you have to always give the generator a "way out"
2017:02:15 21:26:56 spieden ah like a terminal node in the recursion?
2017:02:15 21:27:09 tbaldridge so one of the s/or or s/keys needs to have a nil, or fully optional params, etc.
2017:02:15 21:27:10 tbaldridge yeah
2017:02:15 21:27:11 spieden i thought maybe there’d be a way to control depth
2017:02:15 21:27:51 tbaldridge there is, but the generator needs something to bottom out at, if you say "go 2 levels deep", what's it going to gen at the bottom level? Has to be a empty map, or a nil, or something.
2017:02:15 21:28:03 tbaldridge or basically a scalar value.
2017:02:15 21:28:24 spieden ok thanks, i’ll take a look at my specs again
2017:02:15 21:28:49 spieden seems like there ought to be, actually
2017:02:15 21:30:15 spieden yeah there’s an s/or on the path that has both a recursive and non recursive case
2017:02:15 21:31:38 tbaldridge any hint from the stacktrace where the problem is?
2017:02:15 21:32:46 spieden there’s no stack trace, it just spins endlessly and eats CPU when i try to sample one from it
2017:02:15 21:37:33 tbaldridge the problem here is that spec creates the generators very eagerly.
2017:02:15 21:37:50 spieden ah ok, SO
2017:02:15 21:38:28 tbaldridge and so I've seen really long times to create generators.
2017:02:15 21:39:05 spieden hmm i’ll try giving it longer to see if it ever completes
2017:02:15 21:44:59 spieden nope just cooks my cpus
2017:02:15 21:46:03 alexmiller one thing worth trying is to add :gen-max 3 to all of your coll-of generators
2017:02:15 21:46:35 alexmiller that’s frequently a problem for me in composite gens as the default is 20 and it doesn’t take much nesting for that to be big
2017:02:15 21:49:21 spieden ok thanks, i’ll try that!
2017:02:15 21:50:44 spieden boom, results
2017:02:15 21:51:06 spieden yeah i guess the branching was just going out of control
2017:02:15 21:51:31 alexmiller I have a pending enhancement to change the default to 3 :)
2017:02:15 21:54:55 spieden sounds like a sensible default =)
2017:02:15 21:58:48 qqq I'm wondering if it's possible to statically analyze, for each non-trminal, the smallest structure it can generate -- then, after gen hits a certain size, it tells everything to use it's smallest choice
2017:02:15 21:59:22 qqq thus, instead of controlling depth, gen merrily does its thing until it hits say 1000 nodes, then it goes "oh -%&#$&" and it tries to finish as quickly as possible
2017:02:15 22:06:17 ag hey guys, I forgot how do you select random multiple elements out of a vector? something like gen/elements but for a random number elements instead of just one
2017:02:15 22:07:34 ag ehm, nevermind, I guess I could combine (gen/sample) and (gen/elements)
2017:02:15 22:22:56 ag ❓ what a spec would look like for a map where values depend on each other? e.g. date range with keys :from and :to where :to always should be bigger that :from?
2017:02:15 22:29:21 ag is it possible at all?
2017:02:15 22:31:50 spieden @ag using a predicate anything is possible!
2017:02:15 22:53:03 ag wow… Spec truly is amazing!
2017:02:15 22:57:32 spieden so satisfying when spec finds allthebugs =)
2017:02:15 22:57:50 spieden from hundreds of lines of test code to three
2017:02:15 23:06:01 gfredericks @ag don't use sample to build new generators
2017:02:15 23:06:51 gfredericks (gen/vector (gen/elements ...)) should do what you were asking
2017:02:16 00:45:34 ag @gfredericks I’m using test.chuck, but can’t get my head around this. How do I generate two dates - one always smaller than the second?
2017:02:16 00:46:26 ag I’m using test.chuck.generators/datetime
2017:02:16 07:40:44 viesti the s/keys generator also obeys :req and :opt, so generates maps that have only :req keys, which is neat 🙂
2017:02:16 07:49:57 viesti another thing, I’m tinkering with generating specs from a source outside code (database schema), and doing something like this:
user> (let [k :lol/id]
(s/def k int?))
user/k
2017:02:16 07:50:39 viesti which is not actually what I wanted, so I’m working around it by doing this:
2017:02:16 07:53:13 viesti Which seems a bit odd. Am I missing something or is the idea that specs are defined firstly in Clojure code and not derived from outside sources?
2017:02:16 11:29:23 olivergeorge I faced similar problem when I wanted to generate specs based on an sql db schema by inspection. In that case I generated clojure code.
2017:02:16 11:29:49 olivergeorge The macro based API complicates things.
2017:02:16 11:31:03 olivergeorge It must be possible to do. I'd love to see examples.
2017:02:16 15:34:26 alexmiller you can do that, or use eval
2017:02:16 15:34:51 alexmiller in the future there may be a lower-level function-based api, however it will not be able to automatically capture forms for explain reporting
2017:02:16 15:58:47 mpenet that'd make the life of all the spec to <whatever-format> projects a lot easier
2017:02:16 16:05:47 ikitommi thumbed that one up, thanks @mpenet. Hacked around that just yesterday to get dynamic conformers “drop extra keys” & “fail on extra keys” working for map specs.
2017:02:16 16:20:26 alexmiller @mpenet has been in the plan from the beginning, but has been blocked behind the various form errors
2017:02:16 17:22:23 eraserhd It would be cool to use s/exercise to generate a sample structure for documentation, but I would need to tweak a few things: Make all keys required, ensure all collections have >= 1 element.
2017:02:16 17:22:40 eraserhd I realize this would probably be hacky, but any thoughts on how to do this?
2017:02:16 19:28:30 gfredericks I'd probably write a function doc-sample that has the special cases you described, calls itself recursively, and falls back on gen/generate when it doesn't know what to do
2017:02:16 23:01:58 calvis I’m writing a macro that is essentially a wrapper around defn, so I thought it would be cute to use defn‘s spec to conform for easy access to the args & body. it works on some, but not when the body of the macro is a map, because..
the body conforms as :prepost rather than :body. is there any way to make it backtrack and choose the path where :prepost is empty and :body is correct?
2017:02:16 23:08:20 gfredericks oh man; does that mean defn is essentially unspecable?
2017:02:16 23:08:38 gfredericks I mean I guess you could rewrite it with an or
2017:02:16 23:09:37 calvis well I don’t think it’s un-specable, just that the conform result does not align with the semantics
2017:02:16 23:10:14 gfredericks that's presumably because the spec is ambiguous
2017:02:16 23:10:31 gfredericks and I don't think it should be
2017:02:16 23:11:05 calvis well yeah, I think the spec as written is a bug / oversight
2017:02:16 23:11:12 bronsa yeah
2017:02:16 23:11:17 bronsa i'd open a ticket about that
2017:02:16 23:11:26 calvis just hoping for a way around it
2017:02:16 23:11:33 gfredericks change the spec yourself 🙂
2017:02:16 23:11:57 calvis more work than I was hoping to do
2017:02:16 23:14:00 gfredericks you'd wrap the :prepost and :body in an or or alt or whatever, where the body uses + instead * in the branch with the prepost
2017:02:16 23:38:20 bronsa hmm, maybe prepost should be stricter than just map?
2017:02:16 23:39:35 bronsa the actual grammar for defn only accepts a map with :pre [fn*] :post fn
2017:02:16 23:40:59 bronsa shrug
2017:02:16 23:41:30 calvis still an improvement ¯\(ツ)/¯
2017:02:16 23:43:50 gfredericks it does?
2017:02:16 23:44:01 bronsa well
2017:02:16 23:44:06 bronsa it ignores any other key
2017:02:16 23:44:26 gfredericks it looks to me like the only requirement is :pre's value is a collection
2017:02:16 23:44:45 bronsa an throw at runtime if you don't provide [fn*] for :pre or fn for :post
2017:02:16 23:45:31 gfredericks I thought they were arbitrary expressions
2017:02:16 23:45:41 bronsa arbitray predicates
2017:02:16 23:45:47 bronsa i'm using fn to mean IFn instance
2017:02:16 23:46:14 bronsa wait
2017:02:16 23:46:34 bronsa no you're right I'm stupid
2017:02:16 23:46:42 gfredericks have you been using pre-post wrong for years and didn't notice because the functions are always truthy 😄
2017:02:16 23:47:06 bronsa yes, absolutely
2017:02:16 23:47:34 gfredericks dynamic languages are a magic surprise
2017:02:16 23:52:08 bronsa @gfredericks the few slack users that use the slack->irc bridge will know what we've been up to
2017:02:16 23:52:14 bronsa we need to find them and silence them
2017:02:16 23:52:46 gfredericks glad I stopped using that when the threads feature was released
2017:02:16 23:52:58 bronsa oh god I don't want to know how that looked
2017:02:16 23:53:57 gfredericks I think it just looked like regular messages
2017:02:16 23:54:18 gfredericks so I would've ended up unknowingly replying to thread messages in the main channel
2017:02:16 23:54:25 bronsa lol
2017:02:16 23:55:00 gfredericks I can put up with all sorts of nonsense for the sake of technological idealism, but unknowingly looking like a crazy person is not one of them
2017:02:16 23:55:37 bronsa one thing I really miss about the #clojure irc: slack makes me feel so much more guilty when I derail a conversation/channel into offtopic banter
2017:02:16 23:56:01 gfredericks only because threads exist? or because topics are more specific?
2017:02:16 23:56:08 bronsa the latter I think
2017:02:17 00:04:47 gfredericks I miss the jovial bots
2017:02:17 00:05:05 bronsa and the internet points
2017:02:17 00:05:21 gfredericks oh yeah I had a lot of those I think
2017:02:17 00:05:43 gfredericks should've sold'em before the market crashed
2017:02:17 00:06:12 bronsa no point in crying over what could've been now
2017:02:17 00:06:24 bronsa that was a terrible pun i apologize
2017:02:17 00:07:05 gfredericks can we start threads based on thread messages
2017:02:17 00:07:33 bronsa I've frequently complained to my coworkers about the lack of sub threads
2017:02:17 00:07:33 gfredericks seems not
2017:02:17 00:08:16 gfredericks I miss trying to get the bots to talk to each other
2017:02:17 00:09:40 bronsa was it you who came up with that brilliant double quoting thing to bypass their "security" measures?
2017:02:17 00:10:02 bronsa i remember being very amused
2017:02:17 00:10:44 gfredericks I don't think I ever achieved an infinite loop personally
2017:02:17 00:12:44 bronsa I might spend the next 3 hours going over old irc logs to find the one I'm thinking about
2017:02:17 00:13:59 gfredericks that would be fun to see; I don't recall anything like that
2017:02:17 00:14:32 gfredericks this feels deceptively like a private conversation
2017:02:17 00:16:07 hiredman deceptive indeed
2017:02:17 00:16:16 bronsa somebody's gonna chime in real soo-
2017:02:17 00:17:02 gfredericks it lists names at the top, but I doubt that includes lurkers
2017:02:17 11:27:43 borkdude Clojure IRC still exists, right? Or is it effectively dead now
2017:02:17 11:28:46 bronsa it exists but i don't think it's nowhere as active as it used to be
2017:02:17 11:29:00 bronsa also i don't join it anymore so from my perspective it's dead and I'm mourning
2017:02:17 13:56:44 shem it's there and still useful and enjoyable
2017:02:17 15:27:42 alexmiller still in regular use
2017:02:17 16:59:20 ggaillard Hi everybody !
I'm having a hard time with cljs.spec (1.9.473). I want to instrument my spec-ed functions, but I can't find the
instrument and instrument-all macros it seems they have been removed… do they still exist somewhere ? I can't find any blogpost about it …
2017:02:17 17:03:16 gfredericks did you check under the .test namespace @ggaillard
2017:02:17 17:05:07 ggaillard I'm checking right now
2017:02:17 17:08:30 ggaillard Great ! I had a caching problem with figwheel preventing me to :require cljs.spec.test. Cider was showing me an other file for some reason …
Thank you @gfredericks !
2017:02:17 23:03:30 bnoguchi Anyone here figure out how to constrain a multi-spec generator to generate maps constrained to always generate maps with only one of the tagged values (i.e., to always follow only one multimethod branch during generation)?
2017:02:17 23:04:46 bnoguchi e.g., in the guide example, only generating events of type :event/search
2017:02:17 23:27:12 alexmiller You can just invoke the generator for the result of invoking one of the method specs
2017:02:17 23:31:41 bnoguchi @alexmiller You mean if mm is the multimethod, then (s/gen (mm {:the/tag tag-val})?
2017:02:17 23:32:19 alexmiller Yeah, or just invoke the spec directly if you van
2017:02:17 23:32:29 bnoguchi Won't that not constrain the tag to the given value?
2017:02:17 23:33:02 bnoguchi i.e., :the/tag is open ended as keyword?, so will generate infinite sample of possibly non-matching keywords
2017:02:17 23:35:09 bnoguchi e.g., for :event/type example in the guide, this would effectively be for :event/search:
2017:02:18 01:12:56 wei what happened to spectrum? is it still being developed?
2017:02:19 08:08:31 Yehonathan Sharvit Is there a way to instrument by default while in dev environment?
2017:02:19 14:17:49 gfredericks @viebel I think you could pull that off with leiningen profiles; not sure how that would interact with code reloading though
2017:02:19 15:02:43 nblumoe Is there an established standard to run spec tests (`stest/check`) from a clojure.test suite (e.g. on lein test)?
2017:02:19 15:21:25 tbaldridge Call that and use (s/gen ::my-spec) to get the gen to us in prop/for-all
2017:02:19 15:25:22 gfredericks If it's just the builtin spec tests, then just using deftest and wrapping the stest/check call in an assertion might make more sense
2017:02:19 15:25:53 gfredericks defspec would be more useful for custom assertions/tests
2017:02:19 15:32:29 nblumoe @gfredericks something like (deftest specs (is (stest/check))) would not be sufficient as it seems, what would you check for?
2017:02:19 15:33:29 gfredericks Examine the return value. Is there a result key?
2017:02:19 15:34:45 Yehonathan Sharvit This might help
2017:02:19 15:35:01 Yehonathan Sharvit It’s for cljs but not too hard to adapt to clojure
2017:02:19 15:38:06 nblumoe thx. If I understand correctly all three solutions require looking at the :failure key manually / with a helper. Okay then I will do that too. Was hoping for a generic integration into clojure.test
2017:02:19 16:03:39 nblumoe ok works from the REPL and via Cider, but not with lein test. Any ideas? Here is the error (I don’t get it so far tbh):
ERROR in (foo) (FutureTask.java:122)
foo
expected: (nil? (-> spec-check first :failure))
actual: java.util.concurrent.ExecutionException: java.lang.ClassCastException: clojure.lang.AFunction$1 cannot be cast to clojure.lang.MultiFn, compiling:(clojure/test/check/clojure_test.cljc:95:1)
at java.util.concurrent.FutureTask.report (FutureTask.java:122)
java.util.concurrent.FutureTask.get (FutureTask.java:192)
clojure.core$deref_future.invokeStatic (core.clj:2290)
clojure.core$future_call$reify__9377.deref (core.clj:6849)
clojure.core$deref.invokeStatic (core.clj:2310)
clojure.core$deref.invoke (core.clj:2296)
clojure.core$map$fn__6881.invoke (core.clj:2728)
clojure.lang.LazySeq.sval (LazySeq.java:40)
clojure.lang.LazySeq.seq (LazySeq.java:56)
clojure.lang.LazySeq.first (LazySeq.java:71)
clojure.lang.RT.first (RT.java:682)
clojure.core$first__6404.invokeStatic (core.clj:55)
clojure.core/first (core.clj:55)
geomix.calc.pva_test$check_SINGLEQUOTE_.invokeStatic (pva_test.clj:244)
2017:02:19 16:14:01 nblumoe this happens whenever I call some function with (stest/check my-fn)` as an argument within a deftest body....
2017:02:19 17:48:25 nblumoe had to require [clojure.test.check.clojure-test]
2017:02:19 18:50:21 alexmiller You're running into the lein test monkey patch problem
2017:02:19 18:51:24 alexmiller There are tickets in both lein and test.check about it, but you can work around it by turning it off in project.clj
2017:02:19 18:52:07 alexmiller @nblumoe ^^
2017:02:19 18:54:06 Yehonathan Sharvit What is the flag that needs to be set in order to instrument all the namespaces (while in dev environment)?
2017:02:19 18:54:31 alexmiller There is no such flag
2017:02:20 10:06:07 akiel I have a question regarding required keysets in combination with om.next or pull queries, where required attributes might be skipped. I have formulated my questions in the following gist: https://gist.github.com/alexanderkiel/81daedb133f7939d0c88ff28dd457442
2017:02:20 14:33:55 mike_ananev Hi! I have ns with spec's. how to load all spec names from another ns?
2017:02:20 14:34:32 mike_ananev I want to get list of def's from particular ns
2017:02:20 14:35:19 akiel @mike1452 you can just require the ns with specs
2017:02:20 14:35:23 mike_ananev I want to print on screen available specs
2017:02:20 14:35:27 mike_ananev names
2017:02:20 14:35:57 mike_ananev in order to do it I need to get list of spec names
2017:02:20 15:22:57 dacopare I found that some of my spec check were running slowly and causing timeouts. If you find the same thing happening, you can use the options to specify the number of tests you want to run.
2017:02:20 15:26:58 gfredericks probably had more to do with the volume of data generated than the number of tests
2017:02:20 15:44:16 akiel @dacopare As @gfredericks said already: keep an eye on your generators. You can also overwrite them in tests or control them globally with something like :max-gen in coll-of.
2017:02:20 15:51:02 dacopare @gfredericks: yes, the datastructures that are input and output are very large. Limiting the number of tests is the only way I can see to mitigate this.
2017:02:20 15:56:40 gfredericks @dacopare there are better ways. The :max-gen option mentioned above might do it, I'm not sure. Otherwise there is gen/scale to manipulate the size parameter that a generator sees
2017:02:20 16:48:35 lwhorton heya guys … i’m curious if there’s an idiom for nil-able keys in (s/keys …) wrt other namespaces? what I want to do is something like this:
1) seems redundant to have to define :req [::b] , where ::b is defined locally but just points to another ns
2) ::b should be nilable only from a’s point of view … so I don’t want to put nilable down inside the spec of entity-b
…
so should this really just be
2017:02:20 16:52:46 gfredericks @lwhorton did you consider making the key optional instead of making the value nilable?
2017:02:20 16:54:32 lwhorton i did, but in this particular case the domain is really shoddy. i don’t have time to go refactoring (yet), so entity-a can have the key, but the value might be nil. I suppose that leads to another question - is it more common to just ignore such cases with an opt key, or to capture all possible nil states?
2017:02:20 16:56:00 gfredericks I don't know if there's any sense of what's more common with spec yet
2017:02:20 16:56:26 gfredericks one of spec's core ideas is that the meaning of a key is the same everywhere, which is why you're having to define a different key
2017:02:20 16:57:04 gfredericks which I imagine implies that somewhere you'll have code that translates something to that new key. and if that's the case, then it shouldn't be much more difficult to instead remove the key when the value is nil
2017:02:20 16:58:09 lwhorton good point. or we could just have a proper domain model thought out without monkey patching and quick fixes 😉. thanks @gfredericks
2017:02:20 20:09:31 lvh If I have a fn with 2 arguments that are related for correct operation (it’s a fn-var and an argument name), can I use clojure.spec.test/check to check it, or should I write a more traditional clojure.test.check prop?
2017:02:20 20:10:40 gfredericks depends on if you want to make that relationship part of the arg spec or not
2017:02:20 20:10:48 gfredericks via s/and probably
2017:02:20 20:14:29 lvh I’m not sure I understand the suggestion; if I do something like (s/and ... is-a-valid-arg-name?) I still need to know the value of the method (and random strings are extremely unlikely to be arg names) — is there something else I’m missing?
2017:02:20 20:19:08 gfredericks in that case you'd need to add a custom generator too
2017:02:20 20:21:35 zane There's no way to use clojure.spec/or or clojure.spec/multi-spec to do what lvh wants here?
2017:02:20 20:22:29 lvh @gfredericks just one that applies to the entire arg spec. Makes sense. Thanks 🙂
2017:02:21 10:47:46 mike_ananev Hi! If I need (gen/sample ::somespec) about 1M or more, how to parallelize this operation? is there build-in tools? or just pmap or something?
2017:02:21 13:14:08 gfredericks @mike1452 what do you need them for?
2017:02:21 13:43:38 mike_ananev @gfredericks for data sample generation. I want to give samples to other team, which needs shape of our data for dev process.
2017:02:21 13:56:39 mpenet @alexmiller what's the extra form parameter in s/valid? ?
2017:02:21 13:56:45 mpenet doesn't seem to be documented
2017:02:21 13:57:57 mpenet well, it's not used actually
2017:02:21 14:06:44 alexmiller It's an alternate value to return if not valid
2017:02:21 14:07:46 alexmiller Or maybe I'm reading that wrong
2017:02:21 14:07:57 alexmiller Can't say I've ever used it
2017:02:21 14:08:01 mpenet doesn't seem used at all but in the exception
2017:02:21 14:13:04 alexmiller that’s not the current version of that code
2017:02:21 14:13:49 gfredericks @mike1452 you can parallelize it however you like, that's kind of independent
2017:02:21 14:14:01 mpenet @alexmiller it's the one linked from the api docs I think
2017:02:21 14:14:02 alexmiller @mpenet looks like its used internally via pvalid?
2017:02:21 14:14:21 mpenet I ll have a look
2017:02:21 14:14:22 alexmiller yeah, the autodoc updating was broken by an issue in spec, so they are not the latest
2017:02:21 14:14:35 alexmiller I have a patch pending to fix the code and get that working again
2017:02:21 14:15:43 alexmiller that valid? form is used when you have an alternate form to supply for the error reporting
2017:02:21 14:16:36 mpenet that's exactly what I wanted it to be 😄
2017:02:21 14:16:43 mpenet I ll try it out
2017:02:21 14:17:42 alexmiller I think maybe the only place it’s used is from within keys
2017:02:21 14:17:52 alexmiller used with that extra form that is
2017:02:21 14:18:19 mpenet so it's not supposed to be "public" ?
2017:02:21 14:18:30 mpenet I mean subject to changes/removal
2017:02:21 14:26:08 mpenet not sure it helps actually
2017:02:21 14:26:47 mpenet I am wishing for a explain-info I think. Sometimes I need to add some context to explain data
2017:02:21 14:27:10 mpenet I know I can wrap the whole spec and add this at a higher level, but it's a bit gross
2017:02:21 14:59:55 alexmiller the valid? function should be considered public and I don’t think there are any expectation that api will change
2017:02:21 15:00:37 alexmiller my impression is that Rich doesn’t want to provide extensions to modify explain (other than by writing your own spec and taking care of it there)
2017:02:21 15:02:02 mpenet 😞 care about more detail of my use case for a potential suggestion?
2017:02:21 15:06:11 mpenet in short: that spec predicate is quite resource heavy, and upon failure the function wrapped by that predicate provides a lot of metadata about the failure (think a query parser output, with line/col num, explain traces etc). Right now not only I loose all this data but I need to re-trigger a parse to get back the metadata (or throw upstream, bypassing spec, which is ugly). In theory, one could just wrap and report the error manually, but this values are "deep" in a (spec validated) map.
2017:02:21 15:07:10 mpenet open to suggestions on how to better handle this
2017:02:21 15:11:46 alexmiller sounds like a good use case for writing a custom spec
2017:02:21 15:12:27 mpenet you mean implementing the internal protocol?
2017:02:21 15:12:34 alexmiller yeah
2017:02:21 15:12:41 mpenet is that "safe"?
2017:02:21 15:12:49 mpenet I mean allowed
2017:02:21 15:13:00 alexmiller it’s available to do but is subject to change while in alpha
2017:02:21 15:14:00 mpenet ok, I guess I might just do this. That's how it used to work with Schema btw, seems quite an elegant way to do it
2017:02:21 15:14:10 alexmiller it’s something I would avoid doing as much as possible
2017:02:21 16:23:04 mpenet no explain-data just returns the data behind a failing spec, doesn't allow you to "customize" said data
2017:02:21 16:23:18 mpenet same diff as ex-info vs ex-data (kinda)
2017:02:21 16:27:56 carocad but isnt the data behind a failing spec the one that your parser already returned? I mean if your parser would be a speced fn then the data that you get already contains that custom information that you want. Or am I missing something?
2017:02:21 16:28:33 mpenet no it's the spec form : something like #(try (parse q) (catch ExceptionInfo e ...))
2017:02:21 16:29:30 alexmiller well, you’ll also get the failing value
2017:02:21 16:29:38 alexmiller but not all the work that went into evaluating the failing value
2017:02:21 16:29:55 mpenet hmm I don't think so
2017:02:21 16:30:05 mpenet valid will return false, conform ::invalid
2017:02:21 16:30:16 mpenet ex-data the form
2017:02:21 16:30:17 alexmiller each problem in explain-data has the value that failed the spec
2017:02:21 16:30:53 mpenet yes, the initial value true, but not the "detail" of the failed predicate (in my case an ex-info instance)
2017:02:21 16:31:26 alexmiller right
2017:02:21 16:31:55 alexmiller maybe the exception case in particular is interesting
2017:02:21 16:32:05 mpenet even then, that wouldn't make it as easy as it is not with the custom spec, it just feeds that extra data to the explain map
2017:02:21 16:32:35 mpenet yes, imho it's something that might make sense in spec itself.
2017:02:21 16:32:37 alexmiller well, what if when conforms fails with an exception, the ex-info was conveyed
2017:02:21 16:32:57 alexmiller normally it just returns a truthy/falsey value (and nil/false can’t convey additional info)
2017:02:21 16:33:00 alexmiller but exceptions can
2017:02:21 16:33:14 mpenet that would be nice, that's what I meant with explain-info , it could return that
2017:02:21 16:33:27 mpenet either accept a explain-info return, or the kw
2017:02:21 16:34:44 mpenet but the reify Spec option is actually nicer in my case (for other reasons)
2017:02:21 16:34:48 carocad how about (or (s/explain-data spec x) ::ok )
2017:02:21 16:35:49 carocad returns ::ok when everything when well or the ex-info from your spec if it failed
2017:02:21 16:41:16 mpenet @alexmiller using explain-info you would basically write your own conformer( try/catch potential exception in my case) and return the (explain-info {...}) on failure. I am not sure doing by default for any exception (or ex-info instance) is desirable
2017:02:21 16:42:14 alexmiller perhaps not
2017:02:21 19:22:48 tjtolton Is core.spec able to use something else as keyword qualifiers besides the current ns? As I say that out loud, I realize how silly it would be if that were not the case, but I see a lot of the example code using the double colon keywords, which auto expand to current ns
2017:02:21 19:23:48 alexmiller any ns is fine
2017:02:21 19:23:56 tjtolton I don't suppose there's some kind of dynamic binding that would change the way the double colon autoexpands?
2017:02:21 20:38:30 ag guys, I’m kinda stuck again. I’ve done it before (I think) now I can’t remember the right way of doing that. How do I properly generate things based on vector? So If I have generated vector of maps, and then for each item I need to generate something else
2017:02:21 20:40:36 ag passing vector generator with fmap into a function with for kinda works, but I think this isn’t right approach, because I need to call gen/generate inside for
2017:02:21 20:41:41 ag since it can’t return generator
2017:02:21 20:43:09 zane > For example, say we wanted to generate a vector of keywords, and then choose a random element from it, and return both the vector and the random element.
2017:02:21 20:43:58 ag @zane here in fn in just grabs random element from generated vector and then…
2017:02:21 20:44:05 ag actually that may work for me
2017:02:21 21:38:38 gfredericks Random element?
2017:02:21 21:39:19 gfredericks Why did you generate a whole vector of them just to pick one out?
2017:02:21 22:19:55 alexmiller well one reason you might do that is if you need a vector AND an element from that vector
2017:02:21 23:27:29 ag actually @gfredericks comment in my case make sense.
2017:02:21 23:48:26 ag ok, there’s still something I can’t figure out for the second case, where I have a predefined vector of maps and for each item I need to generate (something based of that map) and attach in a key to that map and finally return that modified vector
2017:02:21 23:50:18 ag my solution requires to call gen/generate within for, which I don’t like.
2017:02:21 23:50:47 ag I though test.chuck’s for would work for me, but I can’t get it right
2017:02:21 23:54:59 gfredericks Calling gen/generate in general is not a proper way to build a generator
2017:02:21 23:55:23 gfredericks And it shouldn't ever be necessary
2017:02:21 23:56:10 gfredericks If you want to describe what you're doing I'm happy to help work it out
2017:02:21 23:59:11 ag so I have list-of-accounts (predefined list) which is a vector of maps, e.g. [{:id 1 :type :customer} {:id 2 :type :investor},,,]. Now, for each item in that vector I need to generate a balance-update data and add it in :bal key and return that modified vector
2017:02:22 00:01:19 ag let’s say balance-update-generator for the sake of exercise generates vector of ints
2017:02:22 00:06:55 ag hold on. gen-balance-update generates data based on the information of each individual item in the original vector i.e.: (gen-balance-update m)
2017:02:22 00:07:43 gfredericks k, modifying...
2017:02:22 00:07:43 ag it returns a vector, that vector should go to m under the :bal key
2017:02:22 00:09:52 ag aha!
2017:02:22 00:10:10 ag I think this is what I need. I’m not sure yet
2017:02:22 00:10:13 ag let me try
2017:02:22 00:10:31 gfredericks an equivalent approach would be to generate the whole map from gen-balance-update, and then avoid doing the map and assoc on line 15
2017:02:22 00:10:38 ag Gary, you are my hero!
2017:02:22 00:25:17 ag btw, has anyone encountered that - very often when I run (gen/generate) in clojurescript it just kills the websocket repl
2017:02:22 00:26:14 gfredericks is it possible it's generating something Very Big?
2017:02:22 00:31:09 ag not too big though…
2017:02:22 00:31:40 ag still must be doing something wrong
2017:02:22 00:34:01 gfredericks there are certain combinations of generators that have unfortunate distributions where they are Not Too Big most of the time but occasionally Very Big
2017:02:22 00:34:18 gfredericks but I don't know anything about the cljs repl so I don't know if that's even a reasonable guess
2017:02:22 00:40:38 ag it’s ok, not critical, at least right now for me. I have specs in .cljc files and that’s pretty nice
2017:02:22 00:40:59 ag I think it should work fine with tests running in headless browser
2017:02:22 00:41:31 ag but in general I try to avoid generating large items, clojurescript struggles
2017:02:22 00:47:37 gfredericks I would like it to be difficult to accidentally generate Very Large things, but unfortunately I don't think there's a good general solution for the existing pitfalls
2017:02:22 07:56:23 mpenet @alexmiller Should I bring up my case earlier in a jira ticket? or is it something that should be discussed with Rich first. Happy to provide a patch if there's interest.
2017:02:22 13:33:51 ikitommi Starting to be quite happy with my custom 3-arity conformer: easy to selectively conform on runtime (json, string, fail-on-extra-keys, drop-extra-keys etc). Is this something that could go into clojure.spec itself? ping @alexmiller. If so, should I just write an issue/proposal of this into Jira? And the idea is allow separating “what” (the spec) from “how” (the conformer). e.g.
Requires some spec inspection, but the http://dev.clojure.org/jira/browse/CLJ-2112 should make it hack-free. Spec as Records would help even more here (could just add a custom new Protocol for this).
2017:02:22 13:36:02 mpenet clj-2112 could exist as a library until it's finalized
2017:02:22 13:36:41 mpenet as long as the final thing doesn't end up too far from what's in the ticket it'd be a decent solution
2017:02:22 13:36:55 mpenet parsing clj forms is a bit gross otherwise imho
2017:02:22 13:37:31 mpenet (there's also https://github.com/uswitch/speculate doing that, monkey patching spec along the way to reach its goals)
2017:02:22 13:37:53 mpenet a bit too convoluted to my taste
2017:02:22 13:38:08 ikitommi we have similar patching around.
2017:02:22 13:38:29 ikitommi while waiting for things to resolve in the clojure.spec side.
2017:02:22 13:41:13 ikitommi and 2112 could be a separate lib, but I would still need the 3-arity confrom for the specs. currently need to wrap all specs to make it work… https://github.com/metosin/spec-tools
2017:02:22 13:42:19 ikitommi (and the dynamic binding hack)
2017:02:22 13:43:23 mpenet that's the one thing that I don't like so far with spec-tools
2017:02:22 13:44:45 ikitommi well that was my question about getting help from the spec, would like to dissolve the hacks on our side.
2017:02:22 13:44:49 mpenet I'd rather have the conformers as an arg at another level (spec maybe).
2017:02:22 13:45:21 ikitommi me too, the 15:33 comment 🙂
2017:02:22 13:49:30 mpenet that said I am dying for swagger/spec in compojure-api
2017:02:22 13:49:59 alexmiller @mpenet ticket would be fine - important thing is a good problem statement
2017:02:22 13:50:12 mpenet @alexmiller I'll try
2017:02:22 13:50:15 alexmiller @ikitommi I don't understand what you're proposing
2017:02:22 13:54:12 ikitommi a 3-arity conform that would a third argument, a callback which the Spec’s conform would call with the sepc and the value as arguments, the callback would return either a new conformer function of nil. If it returned, it would be used instead of the installed normal conformer.
2017:02:22 13:54:50 alexmiller It seems unlikely Rich would want to add that
2017:02:22 13:55:36 alexmiller But starting with a good problem is the way to win his heart :)
2017:02:22 13:55:48 alexmiller This is a solution, not a problem
2017:02:22 13:56:34 ikitommi If I just write the problem statement into Jira?
2017:02:22 13:58:01 alexmiller Yes, that's ok, although I will close it if I don't think it's a problem we are trying to solve
2017:02:22 14:41:22 alexmiller I’ll take a look, thx
2017:02:22 14:46:53 mpenet the proposal "only" covers the need to have more info for failures the rest is probably too specific to our stuff (even tho the example mentions it)
2017:02:22 14:47:52 tbaldridge @mpenet it's unclear from the ticket, are you parsing strings with spec?
2017:02:22 14:48:34 mpenet I want conform/valid to tell me if a spec value is a valid string representation of a "Query"
2017:02:22 14:49:00 mpenet so it does parse under the hood, and right now (with our patch) conform returns the parsed AST
2017:02:22 14:49:10 mpenet but this is a detail of our use case I think
2017:02:22 14:51:19 mpenet the important bit is about the need to preserve the metadata from the failure to extend what's returned by explain* later
2017:02:22 14:51:57 tbaldridge Yeah, that'd be nice, imo. But it also seems that passing a parse tree into spec would get your around the problem.
2017:02:22 14:52:17 tbaldridge treating Spec like a parser, is a good way to get yourself into trouble, in my experience.
2017:02:22 14:52:25 mpenet I am not using spec as a parser at all
2017:02:22 14:53:02 mpenet the "parse" function returns either a valid ast or an ex-info, the ast is not consumed by spec in any way ( it's basically a (conformer #(try (parse s) (catch ExceptionInfo :clojure.spec/invalid)))
2017:02:22 14:53:46 mpenet the goal ultimately is to have our schemas (that contains these query string representations + tons of other stuff), validated by spec
2017:02:22 14:54:23 mpenet and have error reporting/handling at a single level
2017:02:22 14:54:34 mpenet otherwise yes, we could do another pass just for query validation
2017:02:22 14:54:49 mpenet (outside of spec)
2017:02:22 16:12:10 tjtolton Okay, finally managing to get my company to use spec! great stuff! Here's one thing we're stuggling to internalize - our public API uses json that isn't namespaced, and our internal spec's, obviously, use namespaced keys.
Is there some way to say "this incoming data is of the same structure as the spec, but the keys are unqualified, so handle all the keys as though they are qualified to this namespace"?
2017:02:22 20:08:34 zane Did you ever get an answer to this question, @U37NPE2H0?
2017:02:22 20:10:40 tjtolton :req-un is actually the answer, yeah
2017:02:22 20:10:51 tjtolton thanks for following up
2017:02:22 16:12:44 tjtolton or does one have to generate a spec-specific view of the data before passing it into a spec'd function --
essentially
2017:02:22 16:16:26 tjtolton furthermore, I'm struggling to get my company to accept namespaced keys. They essentially view namespaced keys as an implementation detail of spec that should not be complected with our data model or public API.
I think the big idea with spec is that namespacing keys is actually just a different way to represent data which spec has chosen to support and encourage. But without having enough experience using namespaces and qualified data, I'm pretty sympathetic to the "don't mix implementation details with data" argument.
2017:02:22 16:21:46 not-raspberry :req-un?
2017:02:22 16:23:11 not-raspberry If I understand correctly. I only tried to answer the first part of the question.
2017:02:22 16:23:30 tjtolton yeah, I think that's outstanding.
2017:02:22 18:29:48 frerom Hi, I have a small problem with spec that I would like to get some thoughts on.
So the problem is spec name conflicts. Since s/keys couples key and spec names together conflicts can arise:
2017:02:22 18:41:09 joshjones Well, you can either use two different files, each having its own namespace, in which case ::name will resolve to the namespace it's in, or, you can skip the auto-resolve :: and just name them differently, in the same file:
2017:02:22 18:56:41 frerom Huh. So this is exactly what I want, but it doesn't seem to work in ClojureScript? All I get is
Assert failed: all keys must be namespace-qualified keywords
2017:02:22 18:57:10 tbaldridge can you copy your code?
2017:02:22 18:57:26 tbaldridge :person/name is a qualified keyword, I think you might have a typeo
2017:02:22 19:00:33 frerom Sorry about that, it was a typo. It works perfectly fine. Thanks a lot for the help 🙂
2017:02:22 19:05:17 frerom I guess the risk with this solution is that because the spec doesn't have the namespace in the qualified keyword you might actually get spec name conflicts for different reasons? If :person/name is defined in difference files. So I should probably keep that in mind and not be too generic with the naming.
2017:02:22 19:07:54 not-raspberry @joshjones Doing (s/def :person/name string?) is a good idea until someone does the same.
2017:02:22 19:12:04 frerom Yes, what I would want to do (If I can make up syntax) is something like ::person/name which expands to :my.namespace.person/name
2017:02:22 19:22:10 joshjones @not-raspberry I agree, although nothing prevents anyone from naming any spec anything they want, including any namespace, so it's just making a potential problem less likely, not eliminating it, and assumes good behavior
2017:02:22 19:24:22 jasonjckn is there a way to create an s/keys spec from data? e.g. (s/keys :req-un value)
2017:02:22 19:25:00 tbaldridge @frerom right, the names of specs should be bounded to the problem at hand. So if I'm writing a compiler I may use :expr.if/condition while if I was writing a public facing api for a company I may use :com.mycomp.department.person/name
2017:02:22 19:25:51 tbaldridge @jasonjckn macros or eval are your options, in that order normally.
2017:02:22 19:26:25 jasonjckn alright, i guess macros it is
2017:02:22 19:26:41 jasonjckn i have never used eval in 7 years of clojure
2017:02:22 19:31:17 frerom @tbaldridge I agree
2017:02:22 23:12:27 ddellacosta what’s the idiomatic way to write something that is essentially an enum of keywords?
2017:02:22 23:13:14 ddellacosta I was looking for something like (s/enum :foo :bar) but nothing like that exists; should I simply use (s/or :foo #(= :foo %) :bar #(= :bar %)) ?
2017:02:22 23:13:21 ddellacosta seems overly verbose, to say the least
2017:02:22 23:15:00 ddellacosta @zane 😄 I feel dumb
2017:02:22 23:15:04 ddellacosta thanks!
2017:02:22 23:15:04 zane Don't!
2017:02:22 23:15:08 zane No worries!
2017:02:23 04:36:22 not-raspberry @ddellacosta watch out - set-based specs dont work for false and nil
2017:02:23 04:38:52 olivergeorge Is anyone using caching/memoize type techniques to avoid instrumentation cost? Say keeping a cache of hash values for data which has been checked so that future checks are cheap.
2017:02:23 04:51:05 seancorfield @olivergeorge Given that instrument should be a dev/test-only activity, does performance matter?
2017:02:23 04:53:49 olivergeorge @seancorfield in this case it causes odd performance
2017:02:23 04:54:54 olivergeorge Laggy data input etc. In my case: checking the full definition of a complex form at each function/api call.
2017:02:23 04:55:22 seancorfield Well, yes, instrument has an overhead. Of course.
2017:02:23 04:55:22 olivergeorge Something you keep an eye out for when developing. Nice not to have false positives.
2017:02:23 04:55:34 not-raspberry @seancorfield not if performance doesnt matter.
2017:02:23 04:56:05 olivergeorge Yeah. I appreciate that. Love spec. Caching would work. Just wondering if anyone has looked into using it.
2017:02:23 04:57:59 seancorfield If you have an implementation function and then a wrapper -- and the wrapper is memoized -- then instrumenting the implementation should only apply the overhead for each new set of arguments (and this seems the right way to deal with it).
2017:02:23 04:58:59 seancorfield I'd consider very slow/erratic behavior of an instrumented function to be a good indicator that I might need to cache/memoize the function anyway...?
2017:02:23 04:59:16 seancorfield (happy to be proved wrong but that's my initial intuition)
2017:02:23 05:00:01 olivergeorge Thanks. I see what you mean about wrapping a specific fn. That would do the trick for specific cases.
2017:02:23 05:08:54 olivergeorge @seancorfield any chance you know of a fifo memoize implementation for cljs?
2017:02:23 05:09:17 olivergeorge (worried that a long running SPA would gobble memory with memoize in the wrong place)
2017:02:23 05:10:23 seancorfield We can hope that core.cache and core.memoize get converted to .cljc perhaps?
2017:02:23 05:11:02 olivergeorge That does sound ideal.
2017:02:23 05:11:40 seancorfield (and we just got that functionality in the CI support for contrib projects so...)
2017:02:23 05:12:14 seancorfield As to your original Q, no, sorry, I've no idea about memoization for cls otherwise, sorry 😞
Biggest risk I see is a spec changing causing the cached results to be invalid.
Seems like the spec arg may vary in nature (keywords vs implementations). Perhaps it's better to memoize after calling (specize spec) ?
2017:02:23 22:47:21 alexmiller same problems as usual with memoize:
2017:02:23 22:47:41 alexmiller 1) it’s an unbounded cache, which only fills up at 3 am in production
2017:02:23 22:48:20 bbloom aw c’mon now, sometimes it fills up at 4am 😛
2017:02:23 22:48:21 alexmiller 2) you won’t see effects of any stateful changes (if in your repl, probably from changing specs). but if you made calls to anything stateful, you won’t see changes.
2017:02:23 22:48:31 alexmiller @bbloom only on daylight savings change
2017:02:23 22:48:35 bbloom good point.
2017:02:23 22:49:09 alexmiller but specs are built with a delay anyways, which is forced on first use
2017:02:23 22:49:32 alexmiller so they effectively already cache the spec conform implementation and won’t see changes if you change specs after use anyways
2017:02:23 22:49:52 alexmiller so that’s maybe only a small downside
2017:02:23 23:28:49 ag so I have a s/keys spec with :opt fields. When I use (gen/generate (s/gen ::my-spec) it generates them with :req fields and skips :opt fields. Is there a way to force :opt fields to be added?
2017:02:23 23:29:25 schmee if you generate a bunch of them it will add in optionals
2017:02:23 23:29:42 schmee generate starts out with the simplest possible values and gets more complicated as it goes
2017:02:23 23:32:35 ag oh, wait, it actually does not skip :opt fields all the time, but now if I have (gen/vector (s/gen ::my-spec)) some items will have some of :opt fields and some would not. Is there a quick and easy way to fix that?
2017:02:23 23:32:49 schmee ag that’s what I meant above
2017:02:23 23:33:07 schmee if you want the field all the time, why are they optional?
2017:02:23 23:34:04 ag because, in the runtime they are.
2017:02:23 23:34:29 ag meh, I guess it’s actually fine.
2017:02:23 23:34:47 ag I thought for a moment it’s just not generating :opt fields at all
2017:02:23 23:34:50 ag I was wrong
2017:02:23 23:37:48 schmee you can use a custom generator if you want to add them all the time
2017:02:24 00:12:58 olivergeorge @alexmiller Thanks Alex, that's helpful. Sounds like some of those issues could be addressed with a LRU/TTL version of memoize. Shame CLJS doesn't have one packed up at this point.
2017:02:24 00:13:20 olivergeorge (or flushing the memo cache when the spec registry changes)(
2017:02:24 00:13:42 alexmiller Yeah, core.cache and core.memoize have those, but not cljs right now
2017:02:24 00:14:22 alexmiller @ag there is a pending patch to fix keys gen for opts
2017:02:24 01:09:19 ag if my vector contains maps with a key that’s value is a function, how do I check for it? is there a predicate like function?. Actually what I have there is a clojurescript function. (type) returns something like this:
#object[Function "function Function() { [native code] }”]
how can I get a predicate that checks for that?
2017:02:24 01:16:07 bnoguchi If we don't want specs' conformed values to thread in an (s/and spec-1 spec-2 from spec-1 to spec-2 to etc , can we substitute s/merge for s/and? And can spec-* be a non-map spec or just a predicate when called with merge?
2017:02:24 01:28:17 ag @danielcompton something tells me that this is right, but for some reason not really working when placed into .cljc file
(s/map-of keyword? ifn?)
2017:02:24 01:29:41 ag oh, I think my problem not that, spec works, but needs a generator. hold on
2017:02:24 01:31:08 ag
Unable to construct gen at: [:options 1] for: ifn?]
2017:02:24 01:40:23 danielcompton Oh, maybe cljs.core/fn? would be more appropriate, as maps, keywords, e.t.c. are also ifn?
2017:02:24 02:49:28 ag an hour ago I didn’t know that either of them existed. ¯\(ツ)/¯
2017:02:24 02:51:22 ag yup, you’re right! Thank you @danielcompton
2017:02:24 21:15:48 j-po Should (require '[my-ns :as n]) allow you to then address :my-ns/kw as :n/kw? I'd been functioning under the impression that yes, but now I'm trying to address a spec defined in another file in the repl and I'm getting "unable to resolve spec".
2017:02:24 21:16:15 alexmiller No, would be ::n/kw
2017:02:24 21:16:29 j-po Aha!
2017:02:24 21:16:33 j-po Thanks
2017:02:24 21:17:10 alexmiller The :: means "auto-resolve" which will use aliases
2017:02:25 02:44:16 thedavidmeister is there a way in spec to define relationships between attributes, like #(> ::foo :bar) for example?
2017:02:25 04:46:28 thedavidmeister @joshjones oh ok
2017:02:25 04:46:34 thedavidmeister looks a bit weird but i guess it does the job
2017:02:25 04:46:39 thedavidmeister cheers 🙂
2017:02:25 05:23:15 alexmiller Call explain-data to get a list of all problems
2017:02:25 05:40:38 joshjones @thedavidmeister hmm, "weird" is about all I got late Friday after evening activities 🙂 but here you go, same as above but rewritten in a function. cheers to you too
2017:02:25 05:41:41 joshjones it relies on things you possibly shouldn't rely on, like contains? implying that a key is missing, works for now but it's not "public" ... but it's enough to get you going I think 😉
2017:02:25 07:04:16 thedavidmeister @joshjones yup, looks good enough for me to start swapping out my hand-rolled validation code
2017:02:25 18:45:01 bbloom i forget - was there a collection of attempted/in-progress specs for all the core functions somewhere?
2017:02:25 19:34:00 alexmiller from core, no (beyond the one ticket out there that slightly extends what’s committed)
2017:02:25 19:34:38 alexmiller there is another project someone started, but I’ve intentionally not looked at it
2017:02:26 01:01:32 ag Hey guys, so if I have s/keys spec, that enforces a predicate on one of the keys (so it actually requires a generator, since it cannot satisfy predicate after so many tries), how the generator part would look like for the spec like that?
trivial example:
(s/with-gen
(s/and
(s/keys :req [:foo :bar :baz :qux :zap :dap])
(fn [m]) (= (:foo m) 42))
(fn []
;; how the generator would look like?
))
Do I have to generate a hashmap with (s/gen) for each key and then some logic that satisfies the constraint? Is there a better way?
2017:02:26 01:05:11 ag I guess I would create a spec without the field(s) that I want to constraint and then merge it with another spec (with the constraint(s) and generator)
2017:02:26 01:09:36 seancorfield Yeah, splitting it in two would be my suggestion.
2017:02:26 01:10:19 seancorfield Since then you can (g/fmap #(assoc % :foo 42) (s/gen ::base-spec)) or something like that...
2017:02:26 22:24:11 olivergeorge It seems like I can't unform collections. What am I missing?
2017:02:27 01:38:29 bbloom but only b/c the int generator is producing lots of values and then the s/and generator is filtering them
2017:02:27 01:39:08 bbloom i’m wondering if there’s any experimentation done where the filter can feed information back to the generator
2017:02:27 01:39:14 bbloom rather than being a strictly forward pipeline
2017:02:27 01:41:14 gfredericks Ohright. I feel like rich might have wanted something like that. But I have no idea what direction you would go for a general solution to that. Did you have something specific in mind when you said concurrent?
2017:02:27 01:41:44 bbloom something earlier had reminded me of the art of the propagators stuff
2017:02:27 01:42:00 bbloom and i re-read the section in that report about the truth maintenance systems and the search problem
2017:02:27 01:42:21 bbloom once you add predicates, generation changes from an enumeration problem in to a search problem
2017:02:27 01:42:49 bbloom numbers with inequalities are a pretty easy search space - but can get rough if you did something like “is prime” or whatever
2017:02:27 01:43:01 bbloom so you wind up having to override the generator for the whole thing
2017:02:27 01:43:14 bbloom and the more interesting the predicates, the more generator work you need to do
2017:02:27 01:43:59 gfredericks Yeah, it smells like declarative programming, where you can do much more than you might think, but way less than you need in real life
2017:02:27 01:44:09 bbloom heh
2017:02:27 01:45:47 bbloom i was just wondering if you guys had thought about this sort of thing much at all - since i keep trying to use generative testing, and it keeps being a lot of work. so far i’ve only really been truly successful with it in the form of “simulation testing” on a service
2017:02:27 01:46:29 bbloom but i had to do a lot of work to build a good model / generator / validators / etc - made sense for a whole system, but s/exercise etc is fun and rapidly loses utility as i enhance my specs
2017:02:27 01:46:55 gfredericks Yeah I agree about all of that
2017:02:27 01:48:00 gfredericks I haven't thought about it too much because I haven't had any reason to be hopeful that it would be fruitful. I'm definitely open to being persuaded otherwise.
2017:02:27 01:49:07 bbloom i guess that’s why i specifically mentioned “applicative” - as it seems like there might be some way to improve generators in a compositional way using binary combinators
2017:02:27 01:49:31 bbloom the two interesting ones being intersect and union
2017:02:27 01:49:42 bbloom so like maybe rather than defining a generator override for a spec by name
2017:02:27 01:49:59 bbloom you can specify an override for a tuple of union/intersect and a pair of generators
2017:02:27 01:50:24 bbloom which is already sorta what s/and and s/or do, but the do it left to right
2017:02:27 01:51:21 bbloom although it’s not quite clear to me how to go about this without immediately winding up in the land of SAT/SMT 🙂
2017:02:27 01:54:47 gfredericks yeah; and inevitably it would only work for a few lucky combinations of things and otherwise you'd be on your own
2017:02:27 01:55:40 gfredericks if there are common combinations of specs that are easy to generate automatically, that can, I assume, be automated by writing special spec macros for them
2017:02:27 01:57:21 bbloom ok - well i may noodle on this some more at some point, but for now i’ll just resign myself to abandoning generative testing as soon as it gets tough and then re-adopting it as soon as my system gets big enough to justify the simulant sort of thing 🙂
2017:02:27 01:57:24 bbloom thanks for the discussion
2017:02:27 01:58:59 gfredericks haha
2017:02:27 01:59:04 gfredericks np; good luck
2017:02:27 11:21:42 jrychter While writing specs for some existing data structures I encountered a case where a structure has two fields with the same unqualified keyword (nested on different levels, obviously). One field is a collection, the other is a string. Is there a way to make this work with spec? I tried searching online, but couldn't find anything and it seems like generally the answer would be "your data structure is wrong, fix it". Other than splitting the structure into two namespaces (so that each keyword and its associated specs is in a different ns), is there anyway to address this?
2017:02:27 11:24:34 mpenet either you need 2 different ns'ed keywords as spec ids, or use a predicate instead of s/keys
2017:02:27 11:50:51 Yehonathan Sharvit The interesting question is: why s/keys has been designed this way?
2017:02:27 11:51:36 Yehonathan Sharvit I mean: probably the case raised by @jrychter is not idiomatic
2017:02:27 11:51:56 Yehonathan Sharvit Maybe, it is not idiomatic to have different keys with the same name?
2017:02:27 11:52:14 Yehonathan Sharvit I’m asking because I have also encountered the same issue
2017:02:27 11:52:31 jrychter Well, I can see why "your data structure is wrong, fix it" makes sense — I was just hoping for a reasonable workaround. In the meantime, I am writing db migration code to rename one of the keys.
2017:02:27 11:53:21 Yehonathan Sharvit Actually, I don’t understand why it is wrong - but I’ve got the feeling that it’s not the clojure.spec way to shape a map
2017:02:27 11:53:56 Yehonathan Sharvit But I don’t understand why s/keys doesn’t support passing pairs to it
2017:02:27 11:54:51 Yehonathan Sharvit Then the name of the key would be decoupled from its meaning
2017:02:27 11:59:13 jrychter Well, spec by being tied to namespaces assumes that a spec key is "global" within the data structure, as long as everything is within a single ns. I can see (and like) the advantages of that, but there are cases where this becomes problematic.
2017:02:27 12:04:08 gfredericks the workaround is two keywords with different namespaces
2017:02:27 12:10:37 gfredericks yep ☝️ that
2017:02:27 15:00:19 alexmiller @viebel the whole point of spec is that the key of a map is a semantically enduring thing with meaning and that maps are nothing but sets of those attributes
2017:02:27 15:33:26 jrychter This makes perfect sense, except sometimes you run into maps where there is a :name and a :seller :name, and you don't necessarily want to introduce a whole new seller namespace. Spec pretty much forces you to. Note, I'm not saying it's a bad thing, just noting that this is one of the few places where Clojure steps in and says: this is the one right way to do it. I got used to Clojure being very lenient (want objects? fine. want pure functions? fine. want state? fine. want to mutate it? fine.)
2017:02:27 15:34:39 gfredericks a lot of stuff about spec would have to change to accommodate that, I think
2017:02:27 15:34:56 alexmiller spec has aspirational and opinionated goals
2017:02:27 15:35:36 jrychter @alexmiller: Yes, that's what I'm noticing 🙂 I've already learned to appreciate some of those. In particular, spec encourages reuse, and I'm already making use of it.
2017:02:27 15:35:39 gfredericks I guess it could just add a new kind of map spec; that'd be pretty independent
2017:02:27 15:36:00 alexmiller we’re not going to do that
2017:02:27 15:36:05 gfredericks yes I do realize that 🙂
2017:02:27 15:36:29 gfredericks but I think a library could provide that without undermining anything
2017:02:27 15:39:48 mpenet I am tempted to rewrite a bunch of macros for generating specs+json-schemas on that base, and would like to know if it's viable
2017:02:27 15:40:07 alexmiller @mpenet I would not base anything off of that patch - everything there has changed multiple times and is likely to change again
2017:02:27 15:40:31 alexmiller it has been through one round of review from both Rich and Stu but there are still a number of open questions about it
2017:02:27 15:41:18 mpenet Alright, thanks for the info. Any sort of ETA for news on this proposal?
2017:02:27 15:41:24 alexmiller no
2017:02:27 22:23:33 piotr-yuxuan Hello here 🙂 Just a question for my information: has anyone ever tried (or plan to try) to use clojure.spec to validate form answers or models?
2017:02:27 22:32:16 seancorfield We use spec to validate REST API arguments — which is pretty similar to form answers (in that they are both user-provided over HTTP).
2017:02:28 03:35:55 luxbock when writing variations of an existing spec, such as "order" -> "open order", "closed order", should there be a convention for how to name the namespace part of a spec
2017:02:28 03:37:11 luxbock right now I tend to define order as ::order and then for the variations I add the variation name after a dot to the end of the namespace, so :my.ns.closed/order
2017:02:28 03:38:08 luxbock I do this without creating a new namespace, i.e. I just write the full keyword out for the variations
2017:02:28 09:07:14 dergutemoritz @luxbock Another option would be ::order.closed. That way you can still use the real namespace and thus the auto resolving shortcut syntax.
2017:02:28 09:08:18 luxbock @dergutemoritz yeah but then I can't use use it as an unqualified key in a map
2017:02:28 09:09:04 luxbock my use case is basically where there are a series of transformations on this value in a map, and I want to capture some properties about the transformations having done what they were supposed to
2017:02:28 09:09:08 dergutemoritz Oh okay, if you are specing legacy maps, I wouldn't know what's the best way to go about that
2017:02:28 09:10:21 luxbock I think using unqualified keywords is the only way to do this type of thing, where I want to capture changes in the value between functions
2017:02:28 09:10:47 luxbock if I were to use a qualified keyword for the map then the spec has to be static
2017:02:28 09:11:19 dergutemoritz Note that you can also create and alias a namespace from within another namespace like this: (create-ns 'my.ns.closed) (alias 'closed 'my.ns.closed) which then allows you to require the my.ns.closed namespace from other namespaces and use it in the current namespace via ::closed/
2017:02:28 09:11:45 luxbock hmm yeah that could be another good option
2017:02:28 09:14:16 dergutemoritz Re capturing changes in the value between functions: One central idea of spec is to not use the same key for different kinds of values but to attach a global meaning to a key - so different things should use different keys
2017:02:28 09:14:31 dergutemoritz Maybe you can do that, too
2017:02:28 09:15:04 dergutemoritz In other words: what's keeping you from using different keys?
2017:02:28 09:16:02 luxbock yeah I suppose that might be a better a better practice
2017:02:28 09:16:14 dergutemoritz Worth a try at least 🙂
2017:02:28 09:17:58 luxbock I was thinking that since all of the values that are changed only live during one pass from a function to another, the use of unqualified keys would capture this transient nature of theirs
2017:02:28 09:20:28 dergutemoritz Sounds like you have a deep chain of calls which pass a map through each other, each modifying that map? From a maintenance perspective, I think using different keys makes sense here, too. Imagine reading that code for the first time, you'd see the same keys being used in a dozen functions but in each they may have very different values, depending on from where they're called.
2017:02:28 09:21:10 luxbock yeah I think you are right
2017:02:28 09:21:39 dergutemoritz Well, it might be easier with un specs but still unnecessarily context sensitive I'd think
2017:02:28 09:46:40 ikitommi @viebel have also looked for easy way to define more local key specs. Did a simple helper on top of spec to define deeply nested map/vector/set specs. But currently pollutes the spec registry with generated keys: https://github.com/metosin/spec-tools#simple-collection-specs. There could be a custom keys in spec-tools supporting local keys..
2017:02:28 15:12:01 alexmiller @dergutemoritz re the earlier suggestion of ::order.closed - per https://clojure.org/reference/reader#_literals, keywords should not contain . in the name. This is not enforced anywhere, but I think you should steer away from it in general.
2017:02:28 15:20:05 dergutemoritz @alexmiller Ah, thanks for the hint! Yeah it's not enforced anywhere in my experience, I'd rather not rely too heavily on unspecified behavior so the headsup is definitely welcome
2017:02:28 17:49:09 vikeri I know spec does not check “unspecced" keywords for a number of reasons. But sometimes it would be very handy to check big nested map (eg. re-frame app-db) and get the keys that are not specced so that I can add specs for them. Is there any way of enumerating the registered specs?
2017:02:28 17:51:51 bbloom @vikeri instrument etc does this - so you can probably look at how those work
2017:02:28 17:52:33 vikeri @bbloom Ok thanks!
2017:02:28 19:15:53 alexmiller @vikeri you can call s/get-spec to check whether a spec is in the registry
2017:02:28 19:16:18 mikerod Many Clojure features are based on some past research/books/papers. Is there a list of these for Clojure spec?
2017:02:28 19:17:13 mikerod quoted:
> clojure.spec takes "Matt Might et. al. Parsing with Derivatives" and really, really runs with it.
2017:02:28 19:17:38 mikerod RDF and Racket contracts
2017:02:28 19:17:56 mikerod I’ve seen those too, but I wasn’t sure on which aspects of those and if how it related to the parsing stuff
2017:02:28 19:18:08 alexmiller the parsing stuff is specific to the regex op specs
2017:02:28 19:18:11 mikerod but thanks for reminding me that those were at the bottom of the about spec page - I forgot
2017:02:28 19:18:39 alexmiller contracts are at a high level similar to what spec is doing
2017:02:28 19:19:30 alexmiller the rdf reference is really about building growable information systems (Datomic steals liberally from this as well)
2017:02:28 19:20:42 alexmiller Rich is being somewhat modest imo in that he’s been thinking about some of the parts for as long as he’s been working on Clojure (like maps as a set of attributes)
2017:02:28 19:21:46 mikerod Thanks for that added insight Alex.
2017:02:28 19:22:09 mikerod > the parsing stuff is specific to the regex op specs
Does this mean that "Matt Might et. al. Parsing with Derivatives” is not really relevant to an understanding?
2017:02:28 19:22:31 alexmiller only about understanding that very specific part of the code
2017:02:28 19:22:38 mikerod I do intend to dig into the parsing of spec some out of interest in what it is doing. I was just wondering if reading that paper would have any relationship. I’ve read a bit of it so far.
2017:02:28 19:23:09 mikerod I know spec has more to it than parsing of course. I can understand how the RDF is about the growable systems and Racket contracts as a similar lang feature.
2017:02:28 19:23:09 alexmiller there’s very little “parsing” - the only part doing anything you’d call parsing is the regex stuff
2017:02:28 19:23:26 alexmiller everything in spec works on Clojure data, not strings
2017:02:28 19:23:39 mikerod Yeah, I have seen that much.
2017:02:28 19:24:35 alexmiller we looked at a lot of other existing libs in the research for this (I built a couple versions prior to Rich’s work that were thrown away)
2017:02:28 19:25:01 alexmiller of those, I would point most at Christophe’s seqexp lib, which is really quite lovely
2017:02:28 19:27:06 alexmiller spec does not take either of those approaches, but they were helpful as points in the design space
2017:02:28 19:27:51 alexmiller I’d say the ideas of a) starting from predicates and b) automatically generative from the start are also things that Rich insisted on from the very first conversation
2017:02:28 19:29:26 mikerod Thanks for all the details. That’s good background info. It’s always good to get to hear some of the thought process behind these sorts of things to understand the design process some.
2017:02:28 19:32:01 ghadi If I were to rewrite pex, I'd globally register match patterns like spec does.
2017:02:28 19:32:27 bbloom @ghadi still happy with the bytecode interpreter approach?
2017:02:28 19:32:41 bbloom i’ve tried to do some parsing with derivatives and have had pretty poor luck
2017:02:28 19:32:57 bbloom i find the derivative of a grammar to make sense, but the derivative of a parser makes my brain hurt
2017:02:28 19:35:53 ghadi bytecode interpreter is super fast and I love it
2017:02:28 19:36:05 ghadi but -- I didn't implement a stack overflow guard
2017:02:28 19:36:12 bbloom ¯\(ツ)/¯
2017:02:28 19:36:35 ghadi I am not 100% sold on registers vs locals though. @cgrand did registers in segexp
2017:02:28 19:39:04 bbloom am i reading that right? only two registers?
2017:02:28 19:40:48 cgrand No N registers of two slots
2017:02:28 19:40:56 bbloom what are the slots?
2017:02:28 19:41:30 cgrand Each end of a sub match.
2017:02:28 19:41:44 bbloom ah - gotcha
2017:02:28 19:42:17 bbloom parsing is fun/terrible 🙂
2017:02:28 19:43:35 cgrand The goal was to avoid head retention
2017:02:28 19:44:17 bbloom like for streaming parsing? assuming you don’t have unbounded backtracking in your grammar?
2017:02:28 19:44:58 cgrand Yup
2017:02:28 19:47:50 cgrand I use it to query live healthcare data streams.
2017:03:01 09:35:16 cgrand @ghadi I believe it’s not a choice of registers vs locals. With a PEG parser you have ordered choice so I suppose than in (/ A B) you first try to parse A then backtrack and try B. In Seqexp | is non-deterministic so I have to represent concurrent “threads” (and deduplicate them). It’s in the deduplication part that determinism is brought back by keeping submatches of the “leftmost" (according to the choice tree) thread.
2017:03:01 09:39:25 cgrand So because threads are data, It wasn’t possible to use locals.
2017:03:01 15:03:56 devth not possible to spec lambdas? i suppose you could validate it the body of the lambda via assert
2017:03:01 16:22:19 alexmiller depending what you mean, you can spec a function with fspec
2017:03:01 16:23:01 devth right, I mean specing something like f in (let [f (fn [x] ...)])
2017:03:01 17:20:15 dhruv1 how can i generate a json schema from a clojure spec
2017:03:01 18:48:07 bbloom after spec takes over the world for describing data - i hope rich, alex, et al take on session types / protocol specs 🙂
2017:03:01 18:58:15 tbaldridge I'd love to see a spec-take on "isa?", "subset of?", etc.
2017:03:02 10:15:52 thheller so is anyone else constantly doing this: (s/keys ::req-un [::key-a ::key-b])? with all the namespaced keywords I usually don't even notice what is wrong
2017:03:02 10:16:13 thheller and keep wondering why my spec doesn't work, any chance this could throw a warning?
2017:03:02 17:32:39 gfredericks it seemed like a good idea in the 90's
2017:03:02 17:50:06 alexmiller No, it really didnt
2017:03:02 20:45:35 zerocooljs hi please can you help me?? how I can make a spec case insensitive:
(s/def :vtex/channel #{"KUSHKIPAGOS"})
2017:03:02 21:00:49 alexmiller you could replace the spec with #(= (clojure.string/upper-case %) "KUSHKIPAGOS")
2017:03:02 21:04:12 bbloom on the topic of accidental double colon on ::req-un - that’s a situtation where the parameters are plainly provided statically, so MAAAAYBE warning on unrecognized keys wouldn’t be totally objectionable - but that would require trusting ppl to not abuse whatever flag enables that check
2017:03:02 21:06:08 bbloom might be possible to accomplish with metadata or something - if the object has file/line/column info, then it might be a candidate for excess property checking
2017:03:02 21:06:14 bbloom just a thought
2017:03:03 15:07:43 madstap Is this a known bug in spec? I'd expect both to return the same thing, but using s/and with s/keys* doesn't seem to work.
2017:03:03 15:31:48 alexmiller correct - use s/& instead to add a predicate but keep it as a regex op
2017:03:03 15:33:49 alexmiller the and spec above would be looking for data like [3 [:x 3]]
2017:03:03 19:30:22 bbloom This is on HN right now. I’ve read the intro & it’s pretty awesome: https://www.cl.cam.ac.uk/~sd601/thesis.pdf — of interest to anybody doing static analysis with spec, or just generally interested in both open/extensible and types
2017:03:03 23:52:48 jrheard hiya - beginning to add spec to a preexisting project, and i’m running into this situation: i’m trying to spec some “event” maps, which have required keys including :type (eg one of :foo, :bar, :baz) and :data.
and the thing is that when :type is :foo, :data is a map that looks like {:a 1 :b 2};
and when :type is :bar, :data is a map that looks like {:c 3 :b 4};
and when :type is :baz, :data is a number like 1234,
etc.
what do i want in this situation - is this the exact situation where i should reach for a multi-spec?
2017:03:03 23:56:39 jrheard i’m beginning to think that having :data be the name for several different categories of thing will be a problem there
2017:03:03 23:57:23 jrheard and this codebase is small enough, and there are few enough event types, that i can have :foo events have an :event/thing key, and :bar events have an :event/user-id key, etc
2017:03:03 23:57:32 jrheard so i think doing that, and using a multi-spec, is probably the right approach here. rad.
2017:03:04 01:43:17 joshjones @jrheard yes, you want a multi-spec
2017:03:05 17:52:01 michelrandahl how do you 're-use' a random seed for re-checking or re-generating test sample?
2017:03:05 18:16:56 joshjones let's say you ran:
(stest/check `my-func)
which will return a map which has a :clojure.spec.test.check/ret {:result true :num-tests 100 :seed 123456789}
To reuse this seed, run:
this opts map also takes :num-tests which controls how many test are run, and :max-size, which constrains the size of the generated vals
2017:03:05 18:19:38 joshjones @michelrandahl
2017:03:06 11:03:42 jrychter I used s/merge to extend a spec with additional keys, but when I s/conform, it seems that only the newly-added keys get conformed, not the original spec. In general, it seems that while s/valid validates the entire merged spec, s/conform only uses the last spec passed to s/merge. Is this something to be expected?
2017:03:06 11:09:17 jrychter Oh, just found CLJ-2033 which describes this exact problem.
2017:03:06 11:11:36 jrychter After reading the discussion several times, I don't understand what I'm supposed to do. Is the point here that unqualified keys are second-class citizens and should be phased out? Or is there another solution to merging specs and having conform work?
2017:03:06 11:11:55 mpenet s/and
2017:03:06 11:12:12 jrychter Instead of s/merge, right?
2017:03:06 11:12:19 mpenet well it's not really merging anymore, but that can fix it depending on your use case
2017:03:06 11:12:51 jrychter And more generally? Because try as I might, I find this behavior unexpected.
2017:03:06 11:14:26 jrychter Indeed, s/and works just fine. I'm making notes to never use s/merge, but I don't understand why "it does not flow conformed results". Why would one want it to behave that way?
2017:03:06 11:17:39 mpenet not sure really, but yeah I do just like you said and just use s/and
2017:03:06 11:22:17 jrychter I'm sure there was a reason, but to explain my thinking: having read both the s/merge docstring and the clojure.spec guide, I did not expect s/merge to be a lossy operation which will drop some of my conformers. Especially since it is called just like a map merge operation and looks like one. If this is really intended, I think it deserves a warning in both places.
2017:03:06 11:35:25 mpenet with and you also loose :gen
2017:03:06 11:36:15 mpenet so yeah, it's one of the oddities in spec. I also find this quite counter-intuitive (no matter the reason/motivation behind this)
2017:03:06 16:43:03 peeja Is there a way to ask if one spec conforms to another?
2017:03:06 16:43:37 peeja That is, "Is every value which is valid for spec A valid for spec B?"
2017:03:06 16:54:09 not-raspberry I think in general case this is impossible, not only in spec. User-defined functions can be passed to s/def.
But in a limited way, you can generate values for spec A and validate them against spec B.
2017:03:06 16:57:30 peeja I guess that makes sense. One of the major visions for spec is to confirm compatibility between, say, your code and a new version of a dependency (in a world where they both are well spec'd). Is the expected way to do that to use test.check to confirm it works for lots of random values?
2017:03:06 16:58:09 gfredericks I don't think so
2017:03:06 16:58:41 gfredericks I think the main idea was static analysis of keysets and regexes
2017:03:06 16:59:48 peeja So you are expected to be able to do some degree of static analysis, then
2017:03:06 17:10:21 bronsa I think specs can theoretically be comparable in finite time, since AFAICT they describe regular languages
2017:03:06 17:10:46 bronsa but it might take a long time to compare them
2017:03:06 17:12:12 bronsa I might be wrong tho. if spec accepts CFGs then comparing specs is definitely not possible
2017:03:06 17:13:11 not-raspberry @bronsa What about (s/def ::a (fn some-fn [v] ...)) and (s/def ::b (fn different-fn [v] ...)) - how do you compare Clojure functions?
2017:03:06 17:13:28 bronsa you don't, you're right
2017:03:06 17:14:12 bronsa so it's a theoretical no too :)
2017:03:06 17:17:29 gfredericks unless you can get away with saying "different functions are always completely different"
2017:03:06 17:17:44 gfredericks @bronsa the sequence regexes aren't technically regular as they can self-reference
2017:03:06 17:17:50 gfredericks I'm not sure if that's an intentional feature though
2017:03:06 17:18:11 bronsa ¯\(ツ)/¯
2017:03:06 19:09:09 souenzzo How can I express: "One of there keys are required"???
2017:03:06 19:11:10 souenzzo (s/def ::car (s/keys :req [:document/number :license/plate]))
But I just need one of these keys
2017:03:06 19:12:50 alexmiller You can use or in :req - check the doc string or the guide
2017:03:06 20:59:05 seancorfield (s/def ::car (s/keys :req [(or :document/number :license/plate)])) — note this allows for cars with both document number and license plate so if you specifically want exclusive-or, you need to s/and another check onto that.
2017:03:07 08:16:12 hkjels Ohh. I misread the doc. It returns a map-entry, not a map
2017:03:07 10:40:51 andrewboltachev Hi. Does spec support recursive types?
2017:03:07 10:57:12 mpenet yes
2017:03:07 11:05:27 andrewboltachev cool
2017:03:08 18:33:58 nwjsmith I’ve really enjoyed working with spec the past few months, but there is a problem that’s crept up again and again: specs are objects, and can only be manipulated w/ the closed set of operations provided by the clojure.spec ns
2017:03:08 18:34:45 nwjsmith Time and time again, I’ve found myself wanting to manipulate the structure of a spec.
2017:03:08 18:35:22 nwjsmith But the only way to do that is to parse the spec’s form.
2017:03:08 18:40:03 nwjsmith If specs were represented as data structures rather than as opaque objects, I think it would open up a lot of possibilities. I’m sure there are reasons (performance and the alpha API status) that make it more desirable to have specs represented as they currently are. Hopefully somewhere down the line this kind of thing will be made, ahem, easier.
2017:03:08 18:46:51 tbaldridge I agree, metaprogramming of specs is something I'd like to see explored. Pretty much the only missing part IMO, is the specs-for-specs thing you linked above.
2017:03:08 19:08:14 ghadi Unrelatedly, one thing that has been mentioned before was this scenario:
You have a bunch of functions that are processing some map input. If you knew at the outset the specs of all the function, you'd be able to check everything before processing, Will this not repeatedly check specs for all qualified keys in the input map? Or is there some way to avoid that
2017:03:08 19:19:14 alexmiller why do you care?
2017:03:08 19:20:12 alexmiller if you’re doing this at dev or test time, then maybe don’t worry about it.
2017:03:08 19:20:29 alexmiller if you’re validating in production, then validate once at the entry point, and don’t validate in each step
2017:03:08 19:20:36 alexmiller or memoize s/valid?
2017:03:08 19:22:31 alexmiller @nwjsmith specs (as macros) capture their form for error reporting. if you are somehow manipulating the spec, the form is not going to track those changes. this seems bad?
2017:03:08 19:24:39 alexmiller is the stuff in CLJ-2112 (as a Rush fan, I can’t really say how happy I am this happened to be ticket 2112, plus it is trivial for me to remember it) sufficient? If not, what’s missing? Note that you can start with a spec form, conform it to data, modify the data, then unform it back to a spec form
2017:03:08 19:26:00 alexmiller for that matter, you don’t even need the initial form - just build the conformed data, then unform
2017:03:08 19:37:26 nwjsmith Yes, that would be sufficient. I hadn’t thought about the form capturing, which would make manipulating specs-as-data ridiculous.
2017:03:08 19:38:27 nwjsmith What if there was no form capturing though? And spec just provided terrible error messages?
2017:03:08 19:38:48 nwjsmith troll
2017:03:08 19:40:28 nwjsmith I suppose I'll sit here waiting for CLJ-2112 to drop, soothed by the sounds of Geddy Lee and Neil Peart.
2017:03:08 19:44:07 ghadi @alexmiller that makes sense, thx
2017:03:08 20:20:26 alexmiller @nwjsmith well a still possible option is to provide function-level entry points for specs that did not have the ability to use forms in explain (or rather you’d have to supply those if you wanted them)
2017:03:08 20:20:58 alexmiller I guess in some ways the functions under the existing macros are that, although they are not designed to be public
2017:03:08 20:23:54 nwjsmith I think being able to conform, manipulate, and unform the spec forms should be enough. It would be nice to maintain the behaviour of explain through such a manipulation.
2017:03:08 21:20:40 wei does anyone use arohner/spectrum and recommend it?
2017:03:08 21:50:25 alexmiller @nwjsmith I think you would retain that as you’re going back through to the form
2017:03:08 21:51:13 alexmiller @wei I saw his talk at the conj on it and was pretty impressed. it was unclear whether development on it was going to actively continue though
2017:03:08 22:25:40 arohner It is being developed, but I’m very busy
2017:03:08 22:26:10 arohner I’m working through filter, and I have a few hundred lines that haven’t been pushed yet
2017:03:08 22:26:28 arohner but yeah, it’s not ready for real use yet
2017:03:08 22:57:22 wei @arohner thanks for working on it, seems like a lot of people are excited about the prospect of static analysis at compile time
2017:03:08 22:57:29 wei if it works, it would be amazing!
2017:03:09 10:23:39 ilevd Can I use fdef for validation in runtime? It seems it works only for :args not :ret if use instrument
2017:03:09 15:12:09 jjttjj Is there an easy way to disable clojure.spec assertions thoughout a lein project? Right now I have
2017:03:09 15:29:31 rauh @jjttjj I believe it's :jvm-opts ["-Dclojure.spec.compile-asserts=false"]
2017:03:09 16:15:59 bbrinck @ilevd Instrumentation only checks :args, not :ret or :fn. It’s a common point of confusion. :ret and :fn specs are only with generative tests
2017:03:09 16:19:30 bbrinck I’d personally prefer if instrument checked ret and fn (at least optionally), but the authors of spec would say that the point of instrument is to check that you are calling the function correctly, whereas the point of generative testing is to ensure it is implemented correctly.
2017:03:09 16:21:13 bbrinck In my experience, this means that I often don’t write ret or fn specs for functions that can’t be tested with generative testing, which is a shame, because I miss out on the documentation benefits of spec for these functions.
2017:03:09 17:46:54 peeja Ah, so there is! Thanks.
2017:03:09 21:38:09 bbloom for those folks who are attempting to meta-program specs (@nwjsmith, @tbaldridge, @ghadi), i’m very curious to hear about your use cases
2017:03:09 21:40:18 bbloom (i love me some good metaprogramming!)
2017:03:09 21:41:35 tbaldridge In my case, generating JSON schema (for use with swagger) from pre-defined specs.
2017:03:09 21:41:57 tbaldridge Also, generating Datomic schemas from spec.
2017:03:09 21:42:14 bbloom ok, so both to AND from spec use cases
2017:03:09 21:42:31 tbaldridge Also decomposing specs into logic/datalog, so I can say "find me all ::people in this DB"
2017:03:09 21:45:12 nwjsmith @bbloom here’s an example that @peeja came up with at work yesterday: I want to have a spec for an entity map (e.g. (s/def ::widget (s/keys :req [::id ::name]))). Let’s say I only have a function that deals with entities’ ::names. It’d be nice to take the ::widget spec and make it’s ::id into an :opt key. So that the name-handling fn’s spec could be (s/fdef munge-name :args (req-only ::widget [::name]) …
2017:03:09 22:36:52 tbaldridge @waffletower use s/form
2017:03:09 22:37:33 waffletower many thanks!
2017:03:09 22:38:53 bbloom so the reflection use case makes tons of sense to me - especially for doing stuff like generating docs w/ cross reference links or whatever
2017:03:09 22:41:32 schmee can someone enlighten me: why are specs macros instead of data?
2017:03:09 22:42:09 bbloom it’s mostly about error messages
2017:03:09 22:42:28 schmee cause now the accepted thing to do seems to be “use form, transform that and then conform and unform” which seems backwards
2017:03:09 22:42:29 bbloom functions don’t have enough reflective capability to provide good messages with predicate names, etc
2017:03:09 22:42:55 waffletower I am interested in programmatically generating specs, and my simple example caused the s/keys macro to throw:
(def as-req [::museum])
#'user/as-req
user> (def as (s/keys :req as-req))
CompilerException java.lang.IllegalArgumentException: Don't know how to create ISeq from: clojure.lang.Symbol, compiling:(*cider-repl fulfill*:253:15)
user> (macroexpand '(s/keys :req as-req))
IllegalArgumentException Don't know how to create ISeq from: clojure.lang.Symbol clojure.lang.RT.seqFrom (RT.java:547)
2017:03:09 22:48:28 waffletower Could use another macro I guess
2017:03:09 22:48:44 waffletower unless I am making a silly mistake I can’t see
2017:03:09 22:48:54 bbloom i hesitate to suggest this, but....
2017:03:09 22:49:08 bbloom eval exists on the jvm side 😛
2017:03:09 22:50:15 alexmiller @waffletower this won't work right now
2017:03:09 22:50:49 alexmiller You can eval a form or use a macro
2017:03:09 22:51:02 waffletower Thanks Alex
2017:03:09 22:51:47 alexmiller The discussion in the back channel would let you start from a map and then unform with spec form specs back to a spec
2017:03:10 08:25:11 ikitommi @tbaldridge you are doing json schema? here too. I think there are ~4 people doing the same now. Should we do join efforts somehow?
2017:03:10 11:14:35 Yehonathan Sharvit Is it a good idea to use spec for validating data before inserting it into a mongodb document?
2017:03:10 11:15:17 Yehonathan Sharvit The problem I have is that spec is “open” and we have been told by Rich that it is a bad idea to have “closed” specs
2017:03:10 11:15:55 Yehonathan Sharvit But I need to make sure I catch typo errors like fristname instead of firstname
2017:03:10 11:19:37 bronsa if your firstname key is required spec will still catch it as a missing required key
2017:03:10 11:20:06 Yehonathan Sharvit Yes but in my case it’s an optional key
2017:03:10 11:59:06 yonatanel @viebel What's the behavior you're looking for?
2017:03:10 12:00:31 Yehonathan Sharvit I’d like s/valid? ::my-spec to fails for {:fristname “Jo” :age 10} but to succeed for {:firstname “Jo” :age 10}
2017:03:10 12:01:27 Yehonathan Sharvit (s/def ::my-spec (s/keys :opt-un [::firstname ::age])) will not catch this error
2017:03:10 12:03:17 yonatanel Maybe roll your own using clojure.core/keys
2017:03:10 12:04:37 Yehonathan Sharvit My question is not technical but philosophical
2017:03:10 12:05:06 Yehonathan Sharvit I mean: is it good practice to forbid keys?
2017:03:10 12:05:33 Yehonathan Sharvit In a lot of occasions Rich and @alexmiller recalls us that specs should be "open"
2017:03:10 12:08:02 yonatanel Forbidding keys will inhibit forward compatibility, mixing keys e.g for metadata etc.
2017:03:10 12:12:48 yonatanel In your case, I would at least select-keys and dissoc nils before persisting data.
2017:03:10 12:38:51 alexmiller You can catch invalid keys without writing the exclusion into the spec
2017:03:10 12:52:21 Yehonathan Sharvit how?
2017:03:10 12:52:30 Yehonathan Sharvit (In my case it’s a deeply nested map)
2017:03:10 13:46:06 alexmiller You can write code, admittedly more challenging with nesting
2017:03:10 15:00:52 alexmiller @devth indeed, although that should have just been an even number
2017:03:10 15:00:55 alexmiller will fix
2017:03:10 15:03:38 alexmiller fixed!
2017:03:10 16:15:39 yonatanel @viebel If you write a closed keys predicate for each parent key and combine with s/keys, it will drill down the nested map. (s/and (s/keys ...) your-closed-keys-pred?)
2017:03:10 17:41:43 joshkh bit of a newbie question about spec... i'm currently using gen/generator to generate some maps with UUIDs. i'd then like to generate another set of maps that will contain vectors of those UUIDs. i can generate both sets of maps and then assoc the UUIDs from one set to the other but i'm wondering if there's a better workflow for this?
2017:03:10 17:49:19 not-raspberry Like {:a #uuid "sth", :b #uuid "sth-else"} and {:sths [#uuid "sth", #uuid "sth-else"]}?
2017:03:10 17:56:20 joshkh yes, i think so. my example is a collection of "items" with UUIDs {:type :item :uuid #someuuid} and then another spec for a "package" containing random items {:type :package :contents [#someuuid #someotheruuid]}
2017:03:10 17:58:14 not-raspberry i'm no expert but I'd just write a function for that transformation
2017:03:10 17:58:47 not-raspberry and have specs for both data structures
2017:03:10 17:59:42 joshkh yes, i have specs for both data structures. after generating samples of both i can just reassoc contents from one collection to the other
2017:03:10 18:06:42 gfredericks gen/fmap lets you transform the value that a generator generates
2017:03:10 18:07:54 gfredericks so e.g., if you have a generator integers g, then (gen/fmap str g) is a generator of stringified integers
2017:03:10 18:10:03 joshkh oh, nifty
2017:03:10 19:46:59 alexmiller I have been remiss in not mentioning that I will be doing clojure.spec training before the Conj in Portland March 29th - you can register independently from Clojure/west (although why would you want to?)
http://2017.clojurewest.org/training/
2017:03:10 22:10:54 peeja Is there something like s/multi-spec that doesn't use the extra machinery of a multimethod that would be more appropriate for a situations where the cases are a closed set? I don't need to be able to extend the list of cases easily.
2017:03:10 22:11:46 peeja I could use s/or, but that doesn't dispatch by looking at the value being conformed
2017:03:10 22:11:51 peeja which maybe is fine?
2017:03:10 22:12:46 peeja It seems like dispatching on a tag would be faster in general when it's an option, but since it's a small, close set of variants, maybe it just doesn't matter much
2017:03:10 22:23:49 bbloom just use multi-spec
2017:03:10 22:24:10 bbloom you can put ^:private on your multimethod if you want
2017:03:10 22:24:15 bbloom that’ll keep it closed
2017:03:10 22:26:36 yonatanel @peeja It depends on what's bothering you with the multi-spec solution. You can't do much about the style. The set of options is kinda closed and you'll get a "no method" exception on unknown dispatch values. In addition you could define a spec for the dispatch key using a set #{...} to get a nicer explanation for invalid values.
2017:03:10 22:28:43 peeja It's not so much that I need to enforce that it's closed (I do have a spec on the key) as much as a multi-method is grammatically pretty heavy and I don't need the advantages it offers. Having written it out, though, the s/or actually works pretty well for me here.
2017:03:10 22:30:27 bbloom i do think a syntactic shortcut for the common case of single-keyword dispatch may be warranted
2017:03:10 22:31:29 bbloom it’s really not too bad tho
2017:03:11 16:12:26 eggsyntax Anyone know offhand whether there's a way to set a random seed for gen/sample? I saw that you can do so in clojure.test.check/quick-check, but couldn't find mention of it re: sample and generate. I'd like to reproducibly generate some seed data.
2017:03:11 16:19:19 eggsyntax Yeah, that should do me quite nicely.
2017:03:11 16:20:59 gfredericks the test.check API was designed for reproducibility when running tests, not for ad-hoc generator use like spec seems to encourage
2017:03:11 16:21:15 gfredericks adding an arity to gen/generate would presumably fix that
2017:03:11 16:21:36 eggsyntax Yeah, that totally makes sense -- I just happen to be generating some seed data from a spec.
2017:03:11 16:23:07 eggsyntax It's essentially a one-off, so I'm perfectly happy to abuse non-public API calls 😉
2017:03:12 18:36:42 dottedmag I am trying to use spec to validate/destructure external data (JSON). This data reuses key names throughout the various entities: there are keys named N in various parts of data meaning "name", "route number" and "allowed number of things", and they should have different specifications. How do I specify that in spec?
2017:03:12 18:39:17 dottedmag I can do it via :req-un in separate namespaces, but I'm not thrilled by the necessity to create ~15 namespaces, one for every entity kind.
2017:03:12 18:47:29 yonatanel @dottedmag You don't have to actually create the namespace, just fully qualify the specs (unless you want to alias namespaces, but you don't have to)
2017:03:12 18:48:01 dottedmag d-oh!
2017:03:12 18:48:05 dottedmag @yonatanel Thanks!
2017:03:13 00:14:25 nickik Why can I not say (spec/and (spec/keys :req-un [::id]) (spec/keys :req-un [::old-id]))?
2017:03:13 00:15:21 nickik How do I combine two map specs?
2017:03:13 00:15:45 seancorfield spec/merge?
2017:03:13 00:16:38 nickik That sounds correct.
2017:03:13 00:16:45 nickik I did not know this existed
2017:03:13 00:17:33 seancorfield There are some examples about a third of the way thru the Guide https://clojure.org/guides/spec
2017:03:13 00:17:58 nickik I scrolled threw this multiple times, but I did not see merge
2017:03:13 00:18:04 nickik Thank you.
2017:03:13 00:19:10 seancorfield (there's only three mentions of merge and it's easy to miss squished in between a couple of big code blocks)
2017:03:13 00:20:40 nickik Works like a charm 🙂
2017:03:13 00:36:10 nickik If I have something like this: (spec.test/check `my-fn)
2017:03:13 00:36:22 nickik Where should I put this so it will be run with other unitests?
2017:03:13 00:36:39 nickik Should I just put it like that into the test namespace?
2017:03:13 00:37:33 seancorfield You can run the result thru spec.test/summarize (I think) and you can write a deftest around that to assert that the summary shows success.
2017:03:13 00:38:50 seancorfield However, one thing to consider is that generative tests are not "fast" like most unit tests should be -- so you might want to think about having a separate task to run generative tests, so that your unit tests stay nice and fast.
2017:03:13 00:42:01 nickik Ok. So I have to write a deftest myself. Thats fine given how long it takes to run them.
2017:03:13 00:45:21 seancorfield summarize-results -- I had to go look it up.
2017:03:13 21:43:58 dpsutton question about tooling: since spec is intextricably linked to 1.9-alphas no tooling can exist for repl's until they are separated. To include spec otherwise would necessarily require hijacking the specified clojure version to 1.9, which would possibly violate people's expectations. Does this sound true and do I have any misconceptions?
2017:03:13 21:44:29 dpsutton I'm working on spec tooling in CIDER and I don't know how i could begin without hijacking the clojure version of the project that CIDER is interacting with
2017:03:13 21:47:05 ghadi I'm not sure I follow your base assumption. Can tooling check for the existence of spec?
2017:03:13 21:48:25 dpsutton well in clojure 1.8, clojure.spec doesn't exist right?
2017:03:13 21:48:39 dpsutton ah so it would have to be wrapped in lots of conditional checks?
2017:03:13 21:49:25 dpsutton i was just wondering about
(ns nrepl-spec
(require [clojure.spec :as s])
wouldnt' this throw an error on any clojure < 1.9?
2017:03:13 21:53:15 dpsutton
1. Unhandled java.io.FileNotFoundException
Could not locate clojure/spec__init.class or clojure/spec.clj on
classpath.
2017:03:13 21:54:09 ghadi instead check for the existence of a var
2017:03:13 21:56:29 ghadi the spec namespace is automatically loaded in clojure 1.9
2017:03:13 21:56:38 ghadi that may change, we'll see
2017:03:13 21:56:48 dpsutton yeah that makes the problem worse, right?
2017:03:13 21:56:52 ghadi (at least when running a clojure repl in 1.9)
2017:03:13 21:57:08 ghadi I guess I don't get what the problem is
2017:03:13 21:57:46 dpsutton i need middleware to lookup specs to provide a spec browser
2017:03:13 21:57:48 dpsutton in cider
2017:03:13 21:57:53 ghadi you're gonna need conditionals
2017:03:13 21:57:58 dpsutton but if you jack in from a clojure 1.7 project it's gonna blow up
2017:03:13 21:58:11 ghadi well yeah the middleware needs to not be dumb
2017:03:13 21:58:20 dpsutton when they decouple spec from 1.9 i can just load spec in any clojre 1.7 > which is what cider supports
2017:03:13 21:58:25 ghadi sure
2017:03:13 21:58:30 dpsutton but until then i can't even "talk" about it, i thought
2017:03:13 21:58:39 dpsutton like the require statement
2017:03:13 21:59:20 ghadi
(if (spec-available?)
(do-cider-stuff)
:pass)
2017:03:13 21:59:37 ghadi a common thing is to try/catch around require
2017:03:13 21:59:55 dpsutton i guess that will have to be the case anyways, as I'm sure spec versions will be mismatched
2017:03:13 22:00:03 dpsutton thanks for the feedback ghadi
2017:03:13 22:00:05 ghadi spec versions?
2017:03:13 22:00:35 dpsutton i'm assuming when it's released, CIDER will have clojure.spec 1.8 and someone might be using 1.4? or will it be like clojure.string or set where it's built in
2017:03:13 22:00:57 dpsutton i'm not really clear about that
2017:03:13 22:02:29 dpsutton didn't know if it's something that will be specified in the project file with a version or if it will be more baked into the language like set and string, etc
2017:03:13 22:02:41 ghadi more conditionals
2017:03:13 22:02:45 ghadi 😆
2017:03:13 22:02:48 dpsutton haha hooray
2017:03:13 22:03:46 ghadi there isn't a guarantee that clojure.spec will be released outside of 1.9 -- it's been mentioned before, but until you see a commit SHA, can't make assumptions
2017:03:13 22:04:19 dpsutton i sat next to stuart at dinner at the conj and he said that's what was holding up the release of 1.9. I was going off of that but i understand it's not official
2017:03:13 22:04:35 tbaldridge This does bring up a interesting point in the discussion of middleware and nrepl. As long as tools insist on installing new deps into the repl there's going to be problems like this.
2017:03:13 22:04:50 dpsutton sorry not stuart but stu
2017:03:13 22:04:57 ghadi tooling should degrade gracefully
2017:03:13 22:05:06 dpsutton not if i'm writing it
2017:03:13 22:05:08 dpsutton 🙂
2017:03:13 22:23:15 ghadi @tbaldridge you using cider or inf-clojure?
2017:03:13 22:25:49 tbaldridge I use Cursive, you insensitive clod 😛
2017:03:13 22:26:05 ghadi hehheh
2017:03:13 22:26:30 dpsutton haha
2017:03:13 22:26:33 tbaldridge but I do wish all clojure tooling could move away from having middleware in the JVM repl
2017:03:13 22:27:02 ghadi what are your reasons for that?
2017:03:13 22:27:11 ghadi i can think of several, just curious
2017:03:13 22:28:06 tbaldridge it just causes a lot of problems, for no real need. The dependencies used by middleware muck with the resolution order of your repl. Stuff like this spec discussion are due mostly to not being able to use spec in tooling because it might be used in a version of clojure that doesn't support it
2017:03:13 22:28:34 tbaldridge and nrepl is too complex for something so simple, lol
2017:03:13 22:29:50 tbaldridge all I needed was netcat + a tcp port...and now I have a RPC server inside my program
2017:03:13 22:31:08 tbaldridge I guess it's half a rant against middleware and half about nrepl
2017:03:13 23:05:01 seancorfield I must admit, I too find it frustrating to need different dependencies in a project based on what a particular editor or other tooling needs. CIDER is great but with the “regular” stack of DEV middleware it takes ages to start an nREPL server. I’ve recently switched to Atom/ProtoREPL and that requires different libraries in play. For non-DEV work, we use the Socker Server but that’s “too light” for useful tooling (unless tooling was rewritten to assume a more neutral Clojure environment and just expected to execute “regular” Clojure over-the-wire).
2017:03:13 23:05:21 seancorfield (at the risk of continued a somewhat off-topic discussion!)
2017:03:13 23:39:00 ghadi might be off-topic for spec but not in general - being parsimonious about dependencies is very important. gem install hairball applies to our community too
2017:03:13 23:43:58 seancorfield I assume you knew that was a Rich Hickey reference? 🙂
2017:03:14 00:10:29 tbaldridge That's awesome. Although I'm disappointed that it's not a gem that downloads the most recent of every other gem...
2017:03:14 01:58:30 ghadi gem install hairball is a quote from simple made easy
2017:03:14 00:12:59 cfleming FWIW Cursive doesn’t use any middleware at all.
2017:03:14 00:13:21 cfleming I’m a big fan of loading the server-side code I need over the REPL connection when it’s opened.
2017:03:14 00:14:53 cfleming But tooling like CIDER that uses the REPL more than Cursive does are very prone to this sort of problem, right.
2017:03:14 00:15:23 tbaldridge Never thought of it that way, static analysis saves you from having to do a lot of the server side stuff.
2017:03:14 00:15:29 cfleming I don’t think there’s a solution other than saying “this functionality won’t work unless you’re using Clojure 1.9+"
2017:03:14 00:15:34 cfleming Yeah
2017:03:14 00:16:06 cfleming Generally I try to make sure that Cursive will work independently of the version of Clojure the user is using.
2017:03:14 00:16:53 cfleming I’m actually looking at some REPL improvements right now, and if I want to use EDN then that means a Clojure 1.5+ project. Fortunately I know which version the user is using and can warn them, but it’s a little ugly.
2017:03:14 00:18:40 cfleming @tbaldridge BTW pure socket REPLs are on the way.
2017:03:14 00:18:44 cfleming (in Cursive)
2017:03:14 00:19:32 cfleming But again, there I have to check that the user is using 1.8+
2017:03:14 02:40:10 devth is there a way to grab the :ret spec out of an fdef? trying to validate the return value of a fn in a test
2017:03:14 03:33:59 alexmiller Yep, just call get-spec on the symbol to get the function spec
2017:03:14 03:34:30 alexmiller That spec implements ILookup and can be invoked with :ret
2017:03:14 03:35:06 alexmiller So, (:ret (s/get-spec 'full/symbol))
2017:03:15 13:32:34 devth excellent. thanks.
2017:03:16 03:33:23 english What's a good way to spec a sequence with a static subsequence? For example, if I want to spec a seq that contains 1, 2, 3 somewhere in the middle, is there a better way to do it than the following?
2017:03:16 09:46:43 thheller (some background on what a language server is in the gh issue above)
2017:03:16 09:46:56 thheller naturally I want to leverage clojure.spec for this task
2017:03:16 09:47:53 thheller ie. if I have (defn x :foo) in a file or REPL I get an exception with Call to clojure.core/defn did not conform to spec and ex-data containing the usual information
2017:03:16 09:48:44 thheller that information however is not enough for me to provide good highlighting for the error that occurred since it contains no source information
2017:03:16 09:51:51 thheller the question is whether or not this is something the error should/could contain or not? I imagine all tools will require it somehow as the given exception doesn't tell you anything about the actual location of the error.
2017:03:16 09:52:40 thheller @alexmiller you said you were looking into the error presentation details a while back, any progress on that?
2017:03:16 09:53:33 thheller for now even just the form that caused the fdef to fail would be a start
2017:03:16 10:11:07 thheller found :clojure.spec/args but neither that list or the items in there contain any meta data from the reader
2017:03:16 10:37:33 lwhorton is it possible to use spec to generatively test stateful fns? for example, if I have a fn that takes an in and out core.async channel, where it performs some logic on the values in the in channel (when they show up in the future), and perhaps puts values on the out channel, can I generate any meaningful tests?
2017:03:16 12:28:42 alexmiller @thheller yeah I made some progress on it but got blocked by not having access to the root value in the explain-data. I have a ticket and patch pending about that and had a brief conversation about it with Rich last week. I believe there is a separate ticket re spec source meta, certainly there is some overlap with the idea of spec doc meta.
2017:03:16 12:33:29 thheller @alexmiller ah thx, found the JIRA issue. will try with the patch and see if I can do what I have in mind.
2017:03:16 12:34:51 thheller although it still doesn't contain the original form, only the args
2017:03:16 14:09:13 alexmiller should have been changed with this commit
2017:03:16 14:09:23 alexmiller that’s definitely a bug
2017:03:16 14:12:24 alexmiller so that happened in 1.9.0-alpha12
2017:03:16 14:24:13 thheller @alexmiller maybe we could add the last known form to CompilerException(String source, int line, int column, Throwable cause, Object form) as well
2017:03:16 14:26:30 thheller there are many places where a compiler exception is thrown and the form is known
2017:03:16 14:26:31 alexmiller oh, that’s not there already. that needs to be a separate idea then.
2017:03:16 14:27:22 alexmiller that needs some thought. when you’re throwing arbitrary data like that you can introduce some weird problems
2017:03:16 14:27:48 alexmiller I have been burned by that a number of times over the years
2017:03:16 14:28:51 alexmiller I hate that the only exception in Clojure that can attach location info is CompilerException… grr
2017:03:16 14:30:09 thheller hehe a 2nd constructor adding the form where possible seems like the simplest non-breaking change
2017:03:16 14:30:43 thheller not sure how the CLJ compiler/reader is implemented but if the form contains lazy things that will indead cause strange errors
2017:03:16 14:33:18 thheller I will attempt to port the clojure.core.specs over to CLJS soon, it would be amazing if the exceptions generated on errors where somewhat similar
2017:03:16 14:35:35 alexmiller well when you have a form, it’s been read and should not be lazy (unless perhaps it was generated by a prior macro?)
2017:03:16 14:36:14 alexmiller do you really want the form or do you want a string representation of the form?
2017:03:16 14:37:29 thheller the form with metadata would be perfect
2017:03:16 14:37:38 alexmiller yeah, that’s what I was wondering
2017:03:16 14:37:46 thheller with that I can get the string if I need it
2017:03:16 14:38:42 thheller I have never actually worked with the Clojure analyzer, only the tools.analyzer or cljs.analyzer. so I need to figure out how the CLJ metadata looks
2017:03:16 14:38:50 thheller but I take everything I can get at this point
2017:03:16 14:39:14 alexmiller well, nothing in this area is going to change quickly :)
2017:03:16 14:39:50 thheller hehe no doubt but the change to CompilerException is simple enough that I can try in a branch
2017:03:16 14:39:58 thheller and report if thats actually as useful as I think it is
2017:03:16 15:44:23 hiredman forms can be very large, at the repl you just type (/ 1 0) and that fails, and it seems like a good idea to report the form, but I really have no need to see the macroexpansion of a 100 line go block
2017:03:16 20:05:22 timgilbert Say, is there any way to get (s/keys) to validate a map with non-namespaced keywords? Trying to add some validation around an external library I don't control
2017:03:16 20:06:27 timgilbert Oh, sorry, just noticed the section in the guide around :req-un, will read that more carefully
2017:03:16 21:26:36 zane How does Transit intersect with Clojure Spec (if at all)?
2017:03:16 21:57:42 ghadi i don't think it does directly
2017:03:16 21:58:10 ghadi However, Transit is a great way to pass around namespaced keys, which is the norm for things used by clojure.spec
2017:03:17 14:22:00 hkjels I might have to use a macro I guess
2017:03:17 14:25:26 alexmiller if you start from the symbol, it’s easy enough to get the meta from there
2017:03:17 14:26:54 alexmiller (meta (resolve s))
2017:03:17 14:26:59 bronsa might not be that easy from cljs
2017:03:17 14:27:07 alexmiller ah, true, didn’t notice that
2017:03:17 14:30:26 hkjels yeah, unfortunately that doesn’t work
2017:03:17 18:39:15 waffletower Just starting with custom generators, and I have only looked at the underlying code in clojure.test.check.generators a bit, and didn’t notice if there was a more straight-forward way to use custom generators without input arguments. clojure.test.check.generator’s own uuid is an interesting example, but it relies on an internally passed random number (while ignoring size) and defines itself with namespace internal functions.
2017:03:17 19:42:39 gfredericks @waffletower this is generally discouraged since it sidesteps most of the test.check orchestration -- it should be possible for you to write any generator you need using the built in combinators
2017:03:17 19:43:05 gfredericks Is your specific problem about generating a string that matches a regex?
2017:03:17 20:05:09 waffletower the generator doesn’t need shrinking as with gen/uuid, and it currently uses RNG internally. I could use RNG from the generator namespace instead — which seems to suggest I could continue using gen/fmap
2017:03:17 20:10:10 gfredericks you can opt-out of shrinking by wrapping with gen/no-shrink
2017:03:17 20:10:30 gfredericks using the randomness from test.check means that at least your generator (and any composition including it) is deterministic
2017:03:17 20:11:51 gfredericks @waffletower if you'd like to share the details of the generator you already have I could suggest how to rewrite it
2017:03:18 15:08:37 tbaldridge @yonatanel not a super bad idea, but add an additional function to all your conformers (`s/conform can take two args), so that we can unconform as well.
2017:03:18 16:37:41 yonatanel @tbaldridge I don't think there's a need for unform here. I use a few predefined conformers for coercion and they yield either a coerced value or ::s/invalid, and you can chain them with s/and without using unform.
2017:03:18 17:16:05 yonatanel Unless you need the original value which wasn't a goal for this lib. Perhaps I could add it as meta of the coercion result. I never used meta so I don't know if it works for nil and primitives. Anyway it needs to go into a better thought out lib than this.
2017:03:19 11:11:18 Yehonathan Sharvit When one uses clojure.spec to validates the body of an API request, how can we returns user-friendly errors in case the body doesn’t conform?
2017:03:19 11:11:36 Yehonathan Sharvit I mean s/explain-str is nice for developers but not for users!!!
2017:03:19 11:23:34 yonatanel @viebel What about using s/explain-data and formatting it however you want?
2017:03:19 11:30:23 Yehonathan Sharvit How would you do that @yonatanel ?
2017:03:19 11:30:38 Yehonathan Sharvit Let’s take a simple example of a signup request
2017:03:19 11:41:17 yonatanel I guess it's the author responsibility to define higher level predicates that can be later examined more easily than (>= (count %) 8)
2017:03:19 11:42:05 yonatanel Something like long-enough?
2017:03:19 11:43:47 Yehonathan Sharvit Nice idea.
How would you manage the mapping between predicates/specs and a string that describe them?
2017:03:19 11:44:44 yonatanel Not sure right now, but I think you are right in that explain is not really meant for public data-oriented apis.
2017:03:19 11:45:29 Yehonathan Sharvit It means that I have to manually conform the data? 😞
2017:03:19 11:46:02 yonatanel I'm not sure what you mean by that and how it's related to the error message
2017:03:19 11:46:42 Yehonathan Sharvit I mean that I can write code that checks if the password is long enough and returns an appropriate error message
2017:03:19 11:47:15 Yehonathan Sharvit I’d need to write such a piece of code in addition to the specs definitions - which is quite disappointing
2017:03:19 11:48:23 yonatanel Would you like each predicate to optionally have an explanation string or something like that?
2017:03:19 11:48:37 Yehonathan Sharvit Maybe
2017:03:19 11:48:55 Yehonathan Sharvit But I am curious to know if spec is designed for that
2017:03:19 12:13:24 yonatanel You kinda have all the building blocks for that, and it might be good to separate error message from spec itself. You could have multiple mappings from a spec to error message, one for public api, another for a kafka consumer etc, all using the same registered (s/def ::long-enough ...) spec.
2017:03:19 17:16:02 gklijs I still have to go about trying spec out for myself, but for a similar usecase I made a simple validation function, returning a list with either only true, or false and an end-user friendly error message.
2017:03:19 17:16:48 Yehonathan Sharvit Could you be more specific @gklijs
2017:03:19 17:20:15 gklijs Well, I wouldn’t know since I’m not using spec yet, I just did it like this:
(defn valid-registration-map?
[registration-map]
(cond
(not (map? registration-map)) '(false "Input is not a map")
(not (contains? registration-map :username)) '(false "Input is missing the :username key.")
(not (contains? registration-map :password)) '(false "Input is missing the :password key.")
(< (count (:username registration-map)) 8) '(false "Username should have a minimal of 8 characters")
(< (count (:password registration-map)) 8) '(false "Password should have a minimal of 8 characters")
:else '(true)))
2017:03:19 17:45:11 yonatanel In spec you explain separately from validating
2017:03:19 18:26:28 dominicm With forms, there's also the issue that if you validate the whole map, you can't indicate that there's an error with a particular key in the same way s/keys does (unless that's changed?)
2017:03:19 18:39:53 seancorfield We use explain-data and a map of symbols or forms to messages -- and a generic function that walks the explanations and finds matching messages. That way you can choose how generic or how specific the message is by how much of the form of the spec it matches. For example, if you just want a generic bad password message, just map 'password to the message. Otherwise map more pieces of the spec forms to different messages.
2017:03:19 18:44:00 Yehonathan Sharvit @seancorfield could you share a code snippet?
2017:03:19 18:47:18 seancorfield I'm on my phone so, no. I'll try to remember tomorrow when I'm back at work. But this is what the Clojure/core folks have been encouraging us to do with spec: walk the explanation data and map it to messages or error codes or whatever. You have a lot of useful information there.
2017:03:19 18:48:00 seancorfield The path, the predicate, etc.
2017:03:19 18:48:14 Yehonathan Sharvit That looks very cool!
2017:03:19 18:48:27 Yehonathan Sharvit I’ll ping you tomorrow @seancorfield !
2017:03:19 19:31:29 Yehonathan Sharvit @alexmiller any interesting insights of how to generate user-friendly error messages from spec. See ⏫
2017:03:19 20:48:10 alexmiller Advice from Sean above seems right
2017:03:19 20:48:40 alexmiller Or bind a custom explain printer
2017:03:20 02:13:12 bbloom @gfredericks recalling out discussion about generators, monads vs applicatives, annoyances in generative testing, etc - i’ve finally gotten around to reading “Constraint Propagation” by Guido Tack"
2017:03:20 02:13:55 bbloom some snippets:
2017:03:20 02:13:59 bbloom > The generate-and-test solver is too inefficient to solve real-world problems, as it requires enumeration of all assignments. We now improve on this naive method by pruning the set of enumerated assignments using propagation.
2017:03:20 02:14:07 bbloom > A propagation-based solver interleaves propagation and enumeration (search).
2017:03:20 02:15:03 bbloom seems like the way generators defined now is the output of a single manual propagator pass - ie the hand written generators are just domains
2017:03:20 05:17:48 Yehonathan Sharvit @alexmiller what do you mean by “bind a custom explain printer”?
2017:03:20 08:07:34 yonatanel @viebel Maybe like how explain-str does it with with-out-str, but it doesn't sound right.
2017:03:20 08:22:26 yonatanel Ok, got it (from spec source):
(def ^:dynamic *explain-out* explain-printer)
(defn explain-out
"Prints explanation data (per 'explain-data') to *out* using the printer in *explain-out*,
by default explain-printer."
[ed]
(*explain-out* ed))
(defn explain
"Given a spec and a value that fails to conform, prints an explanation to *out*."
[spec x]
(explain-out (explain-data spec x)))
2017:03:20 11:16:55 gfredericks @bbloom how much of this 200 pg paper do I need to read to know what you're talking about
2017:03:20 12:54:55 mpenet would it make sense for (s/conformer f) to return :clojure.spec/invalid on exception?
2017:03:20 12:55:18 mpenet seems like I wrote this a few times already
2017:03:20 16:54:07 stathissideris when defining :args in fdef is there a good way to enforce some relationship between the arguments, for example (fn [a b] (+ a b)) but (> a b) (contrived example)
2017:03:20 16:54:21 stathissideris I have a hard time expressing this as a spec
2017:03:20 16:56:45 bbloom @gfredericks just like any paper like this, you can skip all the math 😛 i didn’t read all of it, just the intro and some prose in the major chapters
2017:03:20 16:59:47 bbloom @gfredericks or you can rewatch sussman’s talk from strange loop a few years back about “we really don’t know how to compute” which may be more accessible to the propagator idea. the only really extension beyond that is that if you combine propagation and search, you get a sound/complete constraint solver. think soduku: if you have badly made puzzle (for humans) that doesn't have enough info to solve by inferences, you can take guesses, and then make more inferences. if your guesses don’t pan out, you can backtrack and make different guesses.
2017:03:20 16:59:56 bbloom right now spec makes some inferences from stuff like s/and etc and composes together some generators, and those things are making guesses
2017:03:20 17:00:56 bbloom that paper mentions that the propagators that provide inferences also act as the predicates in a generate and test system, but in spec, no new inferences occur after the initial construction of the generators
2017:03:20 17:02:25 bbloom not sure if there’s any way to capitalize on this insight in a spec context tho - but the goal would be to shrink the search space to reduce how many examples need to be generated only to be thrown out by predicates
2017:03:20 18:03:15 madstap @stathissideris You can use s/& to apply additional predicates to the conformed values of a regex spec.
2017:03:20 18:24:15 gfredericks @bbloom thanks, I'll noodle it a bit and get back to you
2017:03:20 18:25:06 seancorfield @viebel Slackbot just reminded me about sharing details of our explain mapping code — nice 🙂
2017:03:20 18:26:27 seancorfield There are two parts to it: one is a function that reduces the explain data to a vector of relevant pieces to match on, from most specific to least specific, and then for each spec, we have a map from symbols/forms to the specific error code or message.
2017:03:20 18:28:06 seancorfield The key is that we have two paths through the code, one for when :path is empty and one for when :path is non-empty. The latter provides more context (a top-level key, for example) so we get better error messages that way. With the former, we can sometimes only get generic errors, unless the predicates are sufficiently specific.
2017:03:20 18:35:24 seancorfield We look at :path and :pred (only), and only the first element of :path (at present). We turn the predicate into a canonical form by taking the first three subforms and removing % and the path element (e.g., given a :path of [:email] and a :pred of #(re-find email-regex %) we’d get (‘re-find ‘email-regex) and a path of :email) and then build a sequence of possible matches [[:email ‘re-find ‘email-regex] [:email ‘re-find] [:email] [‘re-find]] — and we’d look in the map of forms to errors for each of those three in turn.
2017:03:20 18:35:51 stathissideris @madstap didn’t know about s/& - thanks, I’ll check it out!
2017:03:20 18:41:12 seancorfield So a given map might be
— so 11302 is for photoid failing the ->long conforming predicate and 11301 is for “other” photoid failures (which is just required field missing in this case). 11402 is for caption being empty, 11403 is for caption being too long, and 11401 is for caption in general (in this case just “missing”).
2017:03:20 18:50:39 Yehonathan Sharvit I was hoping that clojure.spec would give something out of the box
2017:03:20 19:09:21 seancorfield I don’t think that’s really possible since the spec forms can be both arbitrarily complex and application-specific.
2017:03:20 19:11:00 seancorfield We deliberately structure our specs so that our “decoding” function works — i.e., if we find a particularly spec doesn’t decode well, we refactor it (usually just a matter of creating named predicates).
2017:03:20 19:13:02 seancorfield It also takes a bit of trial and error sometimes to get the right set of structural keys in the error code map for certain errors we want to break out specifically.
2017:03:20 19:15:16 seancorfield All told, our decoding logic runs about 50 lines of somewhat dense code. Then we have a map for each spec that needs specific error messages.
2017:03:20 19:16:05 seancorfield And it’s taken two or three iterations on the decoding logic to be both general enough for all our specs and also specific enough to get the level of detail we need in error messages.
2017:03:20 19:19:25 seancorfield The nice thing about this approach is that specs are still clean and readable — and independent of the error messages we need to produce — and we can change the level of detail in how we report errors just by changing the map we pass into the decoding function (e.g., in the map I showed above, if we remove the [:photoid '->long] key, then :photoid will still be matched and we can treat it as “photo ID is required and must be numeric” instead of two separate errors — without changing the spec itself!).
2017:03:20 20:24:43 stathissideris wasn’t there something in spec that allowed you to define a modification of the conformed input?
2017:03:20 20:27:35 stathissideris ah, conformer!
2017:03:20 20:28:27 stathissideris or maybe not
2017:03:20 20:28:49 stathissideris 🤔
2017:03:20 20:42:42 stathissideris maybe it used to be there - I remember a bit of documentation saying that you have the option to do it, but you shouldn’t in case a consumer of your spec output needs the original info
2017:03:20 20:46:26 bbloom @stathissideris i think you’re thinking of conformer - why doesn’t it work for you / what do you need?
2017:03:20 20:47:37 stathissideris so I have this s/or that produces [:match …] vectors, and I’d like to get rid of the “tag” around the actual data
2017:03:20 20:48:03 bbloom you mean that calling conform produces the tagged value?
2017:03:20 20:48:11 bbloom you can’t change the result of conform - only the input in to conform
2017:03:20 20:48:12 seancorfield So you want to add (s/conformer second)?
2017:03:20 20:49:34 seancorfield example:
(s/def ::boolean (s/and (s/or :b boolean?
:s (s/and #{"true" "false"}
(s/conformer #(Boolean. %))))
;; we don't care about the path, just the value
(s/conformer second)))
2017:03:20 20:50:13 seancorfield (not necessarily good practice — because downstream consumers of your spec are forced to take the conformed value and lose the original)
2017:03:20 20:50:23 seancorfield For some situations it’s “right” however.
2017:03:20 20:50:40 stathissideris @seancorfield exactly, but I was reading the documentation wrong and ended up using it incorrectly
2017:03:20 20:50:45 stathissideris many thanks for the example
2017:03:20 20:50:51 bbloom that’s a neat trick @seancorfield 🙂
2017:03:20 20:51:18 seancorfield I found the description of conformer very confusing on the first few reads… but I’m not sure how to describe it any better so I haven’t submitted a PR.
2017:03:20 20:51:30 seancorfield It’s just an odd concept at first.
2017:03:20 20:52:16 stathissideris the reason I wanted that is that I’m using s/& in my function spec to enforce constraints between args, but the conformed args don’t play very well with my predicates
2017:03:20 20:55:07 seancorfield Yeah, we use it to hide the “implementation” of certain specs.
2017:03:20 22:18:41 seancorfield Some thoughts: Instead of comparing against ::s/invalid, why not use the predicate s/invalid?; your conform-or-throw seems to be the built-in s/assert; I’d consider a macro to reduce the boilerplate of those last set of very similar predicates.
2017:03:20 22:21:41 seancorfield We have a macro that takes a predicate, a coercion, and an optional reverse coercion that lets us define specs of that pattern: satisfies “pred” or is a string that coerces to something that satisfies “pred” — and can be generated (by generating for the “pred” and then reverse coercing back to a string).
2017:03:20 22:37:37 yonatanel When do you reverse-coerce?
2017:03:20 22:47:05 seancorfield As part of generation. The default for most of our “API specs” is just str so we produce strings.
2017:03:20 22:47:28 yonatanel @seancorfield regarding s/assert I wrote my own that's independent of *compile-asserts* or check-asserts because it was part of my application logic and not just to find bugs. Anyway I hardly throw anymore and instead put the explain data in a stream.
A macro combining pred and coercion is nice. I'll think about it.
2017:03:20 22:47:46 seancorfield But we have a few specs where we accept or coerce to a keyword so the reverse-coerce is name instead of str.
2017:03:20 22:48:07 seancorfield Ah yes, good point about *compile-asserts* etc.
2017:03:20 22:49:17 yonatanel I'm not into gen testing yet so this whole use case escapes me.
2017:03:20 22:51:39 yonatanel I used to allow strings padded by whitespace and trim them before coercing to keyword. A reverse coercion would miss that, wouldn't it?
2017:03:20 22:53:52 yonatanel But it sounds like a regex spec would know how to generate these things instead of reverse coercion. Could you eliminate reverse-coerce in your case?
2017:03:20 23:01:56 seancorfield With regex as the “spec”, I use test.chuck’s regex generator.
2017:03:20 23:03:17 seancorfield And, yes, if my spec trims whitespace before validation, I would likely not bother randomly generating strings with additional whitespace (I would instead have a single test-by-example that had whitespace to verify that validation accepted it).
2017:03:20 23:05:01 seancorfield It really depends on what aspects of the spec I consider important enough to be part of the generated test cases. For example, we have a spec for validating member-supplied passwords, but we use a regex-generator for it that produces only a subset of possible valid input values.
2017:03:20 23:05:48 seancorfield (it’s not worth the effort to accurately generate all possible edge cases in that situation since the random-but-subset-conforming password values as “good enough” as test data)
2017:03:20 23:12:12 yonatanel Makes sense
2017:03:21 00:36:46 alexmiller @stathissideris @seancorfield regarding your conformer question above, use (s/conformer val) not second
2017:03:21 00:37:42 alexmiller Or if you're feeling bold you can wrap the semi hidden and possibly going away (s/nonconforming ...)
2017:03:21 00:37:53 seancorfield Ah, yes, it’s a MapEntry not a vector.
2017:03:21 00:38:04 alexmiller It's both! :)
2017:03:21 00:38:16 seancorfield I would use s/nonconforming but you keep saying it might go away 😛
2017:03:21 00:38:42 seancorfield So why does it matter whether we use val or second?
2017:03:21 00:39:04 seancorfield (I’m happy to “do the right thing” but want to know why my current code is “wrong”)
2017:03:21 00:39:06 alexmiller val is direct field lookup
2017:03:21 00:39:16 seancorfield Ah, so it’s faster.
2017:03:21 00:39:26 alexmiller second will convert to a seq and pull two values
2017:03:21 00:39:35 alexmiller Should be much faster
2017:03:21 00:41:56 seancorfield I suspect I use second in other places where val would be better...
2017:03:21 00:50:37 alexmiller Just don't ever use second :)
2017:03:21 00:51:13 seancorfield Yeah, I have a whole bunch of (->> … (group-by something) (map second) …) forms!
2017:03:21 00:52:39 seancorfield What about destructuring where you likely have (map (fn [[k v]] …) some-map)
2017:03:21 00:54:10 seancorfield better to do (map (fn [me] … (key me) … (val me) …) some-map)?
2017:03:21 00:54:49 alexmiller I'd use the one that you find more readable
2017:03:21 00:55:02 alexmiller I'd destructure
2017:03:21 00:55:38 alexmiller Id have to look at code to guess at any perf difference
2017:03:21 00:56:24 alexmiller If destructuring uses nth then its probably a wash
2017:03:21 06:56:32 stathissideris @alexmiller thanks, I wasn't aware that using second instead of val for map entries is less performant
2017:03:21 13:20:53 iGEL Thanks. We missed that you still need to pass namespaced keywords, but the expectation is non namespaced. Now it works
2017:03:21 13:23:30 danielneal 👍
2017:03:21 13:48:13 alexmiller @yonatanel no, those are internal and subject to violent change without warning :)
2017:03:21 13:54:12 iGEL Another question: I wanted to use spec in my tests, where I do want to expect most keys in my map to be exactly this, but others like the id or created_at, which I can't predict, to just conform to predicate. For example, given this map:
{:id 4
:customer_id 5
:section "lala"}
I want to see that customer_id is exactly 5 and section is "lala", but for id it just want to know that it's an integer. Finally, it would be nice if the spec would fail if the result has other keys, but that's not required. Is that possible and how?
2017:03:21 13:58:55 moxaj @igel for the first part, you can use sets as predicates, like #{5} or #{"lala"}. For the second part, you can do (s/and (s/keys ...) #(every? #{:your :keys} (keys %)))
2017:03:21 14:13:44 iGEL Thanks 🙂
2017:03:21 14:00:05 moxaj (but it's generally advised against)
2017:03:21 14:03:15 Yehonathan Sharvit Another question related to (s/keys …)
Imagine I run this:
2017:03:21 14:05:16 Yehonathan Sharvit If foo.core is somehow defined in my spec registry, then its specs will run when calling (s/valid? {:foo/core “aa”})
2017:03:21 14:05:31 Yehonathan Sharvit But it is not my intent at all
2017:03:21 14:05:53 Yehonathan Sharvit foo/core might be a namespace defined by a library that I am requiring
2017:03:21 14:06:03 Yehonathan Sharvit Is there a security risk?
2017:03:21 14:06:19 Yehonathan Sharvit Unexpected code can run based on user input
2017:03:21 14:06:47 Yehonathan Sharvit imagine I parse a JSON with {”foo/core” “aa”} and decide to keywordize the keys
2017:03:21 14:07:44 yonatanel @viebel that's why it's recommended to use namespaces you own, otherwise you break others and others break you.
2017:03:21 14:08:18 yonatanel @viebel But yes, that global registry is a pain in the ass.
2017:03:21 14:09:01 Yehonathan Sharvit The problem is that once I require a library, I cannot prevent it to add specs in the registry
2017:03:21 14:09:02 Yehonathan Sharvit !!!
2017:03:21 14:09:32 Yehonathan Sharvit So when I parse the JSON, I must be careful when keywordizing the keys !!!?!?!
2017:03:21 14:09:58 yonatanel @viebel I'm not sure spec is meant for portable data validation and coercion at all.
2017:03:21 14:10:25 mpenet why not?
2017:03:21 14:10:38 moxaj @viebel you can still preprocess the keywordized map, before validating it
2017:03:21 14:10:59 yonatanel Because of how it's built. I'm using it for JSON all the time but it feels wrong.
2017:03:21 14:11:21 mpenet I do the same and I don't have any issue with it. Not sure I follow
2017:03:21 14:12:38 bronsa I think viebel has a point
2017:03:21 14:12:54 yonatanel @mpenet I don't like prefixing all my keywords with a company name for example.
2017:03:21 14:12:54 bronsa suppose you have a spec whose conformer does heavy-ish work
2017:03:21 14:13:09 bronsa if you conform user input against a spec that uses keys
2017:03:21 14:13:41 bronsa it will force that conformer to run
2017:03:21 14:13:58 bronsa might be an opportunity for dos attack
2017:03:21 14:14:17 bronsa forcing cpu bound work while parsing user data
2017:03:21 14:14:36 mpenet if you accept input with keywordized keys and use spec you must follow some rules yes (proper namespaces), otherwise just use the *-un variants
2017:03:21 14:14:55 mpenet (I do the later)
2017:03:21 14:15:21 mpenet the endpoint dicts the context already, no need for namespaces usually
2017:03:21 14:15:21 bronsa well, the point is that if you're using keys and accepting user data you don't have control over what the user will pass you
2017:03:21 14:15:48 bronsa and might malignantly pass namespaced keys that force conform using an expensive conformer
2017:03:21 14:15:52 mpenet if you take edn directly yes that would matter. but arent' we talking about json ?
2017:03:21 14:16:18 bronsa are we?
2017:03:21 14:16:30 mpenet I think so
2017:03:21 14:16:39 Yehonathan Sharvit we are talking about JSON
2017:03:21 14:16:49 bronsa well, point still stands
2017:03:21 14:16:56 bronsa suppose I was talking about edn/transit :)
2017:03:21 14:16:58 Yehonathan Sharvit that you parse with keywrodizing the keys which is quite common practice
2017:03:21 14:17:28 mpenet you then get un-namespaced keywords
2017:03:21 14:17:34 mpenet so no risk for "injection"
2017:03:21 14:18:56 yonatanel Unless you have data like {"event/type" "blabla"}
2017:03:21 14:19:57 bronsa still, if you do something like (comp clojure.walk/keywordize-keys cheshire.core/parse-string) over your input data
2017:03:21 14:20:11 bronsa & then conform
2017:03:21 14:20:11 mpenet ah right, didn't realize that (keyword "event/type") would actually create a ns keyword, bummer
2017:03:21 14:20:15 bronsa yeah
2017:03:21 14:20:31 mpenet another argument for never keywordizing json I guess
2017:03:21 14:20:54 bronsa (doesn't solve this issue with edn/transit over the wire)
2017:03:21 14:22:41 mpenet this kind of sucks indeed, I wonder why keyword allows this at all
2017:03:21 14:22:44 yonatanel @bronsa I guess you shouldn't blindly use (s/keys) on user input, or do anything blindly with user input.
2017:03:21 14:22:55 mpenet there's the 2 arity version for ns stuff already
2017:03:21 14:23:06 bronsa @mpenet it's been like that since forever and it's used quite often
2017:03:21 14:23:17 mpenet still very unfortunate
2017:03:21 14:23:26 bronsa it's quite useful
2017:03:21 14:23:59 yonatanel @mpenet I read somewhere that keyword accepts anything exactly for this reason, to keywordize user input.
2017:03:21 14:24:12 bronsa @yonatanel meh, i don't buy the argument "you should be careful with what you do while parsing user input" tho -- i'm conforming user input to check for valid data
2017:03:21 14:24:15 mpenet that said you could write a safe keywordize-keys
2017:03:21 14:24:25 mpenet using find-keyword
2017:03:21 14:24:34 bronsa i don't want to manually check for invalid data and then conform it
2017:03:21 14:25:51 bronsa what if I do want to use namespaced keys in my user data tho
2017:03:21 14:26:26 bronsa then it's either live with the possibility of people exploiting this for possibly overload cpu while validating user input or don't use namespaced keys at all
2017:03:21 14:27:12 mpenet seems so
2017:03:21 14:27:40 bronsa I would consider this a significant limitation
2017:03:21 14:28:26 mpenet I guess this is a +1 for the "closed" key-set definition people are asking for
2017:03:21 14:28:34 mpenet "a la Schema"
2017:03:21 14:28:50 bronsa or using a private registry
2017:03:21 14:28:53 idoa01 Hey, new here and new to clojure-spec, I was discussing with @viebel on this issue before he came here.
2017:03:21 14:30:00 idoa01 the thing that struct me is that the (s/valid?) will call unexpected code on various input. as we saw here, it might be used to run malicious code on the server
2017:03:21 14:30:53 bronsa I wouldn't say malicious code as the user can't really inject code
2017:03:21 14:37:34 idoa01 in that case, the library has to have a way to clean the input
2017:03:21 14:37:43 bronsa
-Namespaced keyword specs can be checked even when no map spec declares those keys
This last point is vital when dynamically building up, composing, or generating maps. Creating a spec for every map subset/union/intersection is unworkable. It also facilitates fail-fast detection of bad data - when it is introduced vs when it is consumed.
2017:03:21 14:42:19 yonatanel You're asking spec to sanitize your inputs and conformers to be your lossy coercions. From what I gather it's not meant to do that (I do use it that way though :))
2017:03:21 14:43:53 idoa01 I'm asking spec to not run code I didn't specifically intend to run.
2017:03:21 14:45:11 dergutemoritz By that reasoning you'd also have to prohibit polymorphism across libraries
2017:03:21 14:45:44 idoa01 that's taking it too far, and when I use polymorphism, I intend to use polymorphism.
2017:03:21 14:45:45 bronsa don't think that's the same at all
2017:03:21 14:46:19 dergutemoritz It's similar in a way that you potentially run code that you didn't know you were going to run from the point of calling a function
2017:03:21 14:46:37 bronsa this issue implies that you can't run s/valid? on user-provided input before validation (which is what s/valid? is for?).
Now suppose one of your libs defines a spec for some internal validation and it's bugged in a way that causes certain inputs to never terminate parsing.
Now somebody discovers it, and maliciously provides those inputs to you and you have no way of avoiding that to run
2017:03:21 14:46:47 bronsa I think it's a very valid issue to raise
2017:03:21 14:47:07 bronsa let's wait to see what @alexmiller thinks about it
2017:03:21 14:47:11 dergutemoritz Yeah, that spec is not good for running on untrusted inputs is a good obversation
2017:03:21 14:49:04 bronsa this to me feels similar to the problem that clojure.edn solved for parsing user input
2017:03:21 14:49:43 alexmiller but you are asking spec to run validation using s/keys, which will validate keys
2017:03:21 14:49:47 alexmiller if you don’t want that, don’t do it
2017:03:21 14:51:06 dergutemoritz Good point, hehe
2017:03:21 14:52:31 idoa01 then what is the recommended way to validate a map?
2017:03:21 14:53:00 idoa01 or more specifically, a JSON
2017:03:21 14:54:09 alexmiller I think you’re implicitly conflating multiple things into one question
2017:03:21 14:54:33 alexmiller spec has capabilities for validating data
2017:03:21 14:55:24 alexmiller one of them is to define a spec for a map as a collection of attributes
2017:03:21 14:56:26 alexmiller recognizing that many current maps have unqualified keys, s/keys provides :req-un and :opt-un options for validating unqualified keys
2017:03:21 14:56:57 alexmiller and that is one approach to validating json
2017:03:21 14:57:15 idoa01 but :req-un and :opt-un still accepts fully qualified namespaces, which will cause the same issue
2017:03:21 14:57:41 alexmiller so don’t rely on spec to solve every problem for you
2017:03:21 14:58:24 alexmiller its your responsibility as an application developer to think about how you handle untrusted inputs from the external world
2017:03:21 14:58:28 bronsa >but you are asking spec to run validation using s/keys, which will validate keys
right, but even if I'm not while validating input data, I might in some internal function that my valid input data might get threaded through
2017:03:21 15:00:36 bronsa >its your responsibility as an application developer to think about how you handle untrusted inputs from the external world
which is why I'm validating it with spec :) it feels a bit odd to me that we have to be careful about passing user data to validating functions
2017:03:21 15:01:11 bronsa if "spec is not the right tool at this level" is the answer I guess I'll live with that but I can imagine a lot of people will be confused by this answer?
2017:03:21 15:02:03 alexmiller I’m not saying it’s not the right tool, just that you should think critically about what you’re doing. sorry not sorry about that.
2017:03:21 15:02:07 bronsa esp because people are using schema for doing that right now and will be tempted to switch from schema to spec w/o realizing the implication
2017:03:21 15:02:23 alexmiller you control the libraries you’re using, the code you’re running, and when and how you validate data
2017:03:21 15:03:15 alexmiller all your specs call predicate functions, maybe from other libraries - how is that any different?
2017:03:21 15:03:37 bronsa that's all true, it still doesn't make me feel any less weird about having to validate my data in order to validate it through spec
2017:03:21 15:03:49 bronsa it's not different, conform vs predicate functions is not the issue
2017:03:21 15:04:16 idoa01 it's a bit like adding eval to the code, without telling the non-expert user that you're doing it.
I was really surprised when @viebel pointed this feature to me.
2017:03:21 15:04:33 bronsa the issue is it appears it's not safe to validate/conform user-provided data with spec w/o.. validating it before passing it to spec
2017:03:21 15:04:42 idoa01 so it may work especially well for anyone in the "knows" but will shot the regular developer in the leg.
2017:03:21 15:08:27 alexmiller can you give an example of a spec that would be “not safe” in this way?
2017:03:21 15:09:17 bronsa not safety is not an issue, I don't think. I'm thinking of some malformed spec that causes non termination with very simple inputs, which is not unlikely to happen
2017:03:21 15:09:29 alexmiller so, example?
2017:03:21 15:10:12 bronsa one sec
2017:03:21 15:10:47 alexmiller if it’s a bug in a predicate or spec, then you would fix it, just like you would with any bug that has that effect in your code
2017:03:21 15:10:54 alexmiller I’m questioning the premise of this problem
2017:03:21 15:11:04 bronsa correct, it would be a bug of a spec
2017:03:21 15:11:10 bronsa which I don't have control over
2017:03:21 15:11:17 alexmiller you do - you chose to load it
2017:03:21 15:11:22 alexmiller you can choose to not load it
2017:03:21 15:11:26 alexmiller or to replace it
2017:03:21 15:11:33 bronsa yes, but I might not know about the bug in this spec
2017:03:21 15:11:44 bronsa nor might the library author
2017:03:21 15:11:46 alexmiller and you might not know about a bug in a function in a library you call
2017:03:21 15:11:52 bronsa but an attacker might discover it
2017:03:21 15:11:57 alexmiller same for any function
2017:03:21 15:12:01 Yehonathan Sharvit The weird thing is that even if the spec is just there - defined in the lib - without even being in use, it will be in the registry and corrupt my code
2017:03:21 15:12:14 alexmiller it will not be unless you load the code
2017:03:21 15:12:23 alexmiller you control the registry
2017:03:21 15:12:36 Yehonathan Sharvit Should I check all the specs defined in all the libs I load?
2017:03:21 15:12:48 alexmiller should you check all the functions defined in all the libs you load?
2017:03:21 15:12:52 bronsa right, it seems highly more likely to be explotable in a spec tho that finding the correct data that causes an invocation of that function to trigger the bug
2017:03:21 15:12:56 Yehonathan Sharvit All the functions I use
2017:03:21 15:13:03 Yehonathan Sharvit But not the funtions I don’t use
2017:03:21 15:13:12 alexmiller all the functions your functions call?
2017:03:21 15:13:16 Yehonathan Sharvit Yes
2017:03:21 15:13:28 alexmiller then yes, you should check all the specs too
2017:03:21 15:13:48 alexmiller this is not different
2017:03:21 15:13:54 idoa01 say I'm using a common library (say cheshire) and it upgraded with a buggy spec.
now anyone who uses that lib, are open to that bug, without explicitly calling the spec. just by loading cheshire
2017:03:21 15:14:14 alexmiller yes, exactly like if cheshire had a function that was buggy
2017:03:21 15:14:17 bronsa I understand what you're saying, I don't agree that the two have the same severity tho
2017:03:21 15:14:28 Yehonathan Sharvit I cannot discover this bug with any kind of unit tests
2017:03:21 15:14:42 Yehonathan Sharvit (Even using test.check !)
2017:03:21 15:14:53 alexmiller that does not make sense
2017:03:21 15:15:59 bronsa I have control over what functions I use and how I use them, I don't have control over what (loaded) specs some user data will invoke unless I validate that data before I spec/validate it (or don't use keys)
2017:03:21 15:16:22 rauh At the easiest you can trigger a stackoverflow if a user has a recursive schema somewhere defined.
2017:03:21 15:16:40 bronsa anyway, it seems like the answer is "be careful", not sure it satisfies me but I can live with it
2017:03:21 15:16:48 rauh I'd love to vote on a ticket if one is created, because until 10min ago I didn't realize s/keys checked other keys implicitly.
2017:03:21 15:16:58 bronsa that's in s/keys docstring
2017:03:21 15:18:10 yonatanel using an empty (s/keys) beats defining every object in even a medium system.
2017:03:21 15:19:10 alexmiller @rauh if the spec throws an error, then it is doing it’s job in telling you that it’s invalid data
2017:03:21 15:19:24 idoa01 the implicitly of (s/keys) opens the code for trouble.
an in-between solution would be that (s/keys) will invoke only locally namespaced specs, and not library defined specs.
2017:03:21 15:19:59 alexmiller those are not two differentiable things
2017:03:21 15:20:06 donaldball Every once in a while I wish for a s/keys version that allowed a whitelist of spec keywords or namespaces that it checks
2017:03:21 15:20:25 bronsa they are if you allow using a local registry rather than a global one
2017:03:21 15:21:03 bronsa the local registry could be defined as a subset of the global one, becoming effectively a whitelist
2017:03:21 15:21:23 bronsa (not sure if it's a good idea)
2017:03:21 15:21:25 alexmiller not doing that
2017:03:21 15:21:54 idoa01 used with (s/keys :ns ... )
2017:03:21 15:22:37 alexmiller s/keys combines multiple things. enhancement requests to do individual parts of that might be worth considering.
2017:03:21 15:23:38 alexmiller separating required/optional key checks from validation of specified keyed attributes from validation of all keyed attributes
2017:03:21 15:24:12 rauh @alexmiller A Stackoverflow which can easily be triggered by user input isn't something most programmers catch, and certainly valid? is expected to return true/false. Stackoverflow doesn't even get caught with (a pretty wide) (catch Exception _)
2017:03:21 15:24:40 rauh It goes under VirtualMachineError and will likely kill the thread.
2017:03:21 15:27:21 alexmiller again, how is this different from having a bad function that throws a StackOverflowError?
2017:03:21 15:27:49 alexmiller if it’s a bug, fix it
2017:03:21 15:27:50 idoa01 the difference is that the code only needs to be loaded, not referenced anywhere
2017:03:21 15:28:05 alexmiller which is not at all different than multimethods or protocol extensions
2017:03:21 15:28:23 alexmiller there are several Clojure constructs that create runtime state on load
2017:03:21 15:29:11 rauh Because the attack is much easier.
2017:03:21 15:29:48 alexmiller how is it any different than a bug in the validation function you were going to hand write instead?
2017:03:21 15:30:42 alexmiller the attack (from outside) is identical
2017:03:21 15:30:46 idoa01 the question is, if a programmer which isn't in this slack room, uses clojure-spec, will he think about sanitizing the json before passing it to (s/valid?) if not, then the buggy behavior will be wide spread.
2017:03:21 15:31:10 alexmiller what is the exposure?
2017:03:21 15:31:11 idoa01 was this behavior of the (s/keys) been discussed in regards to this issue before? if not, I think that it qualifies as a "surprising" side effect of clojure-spec
2017:03:21 15:31:41 alexmiller it’s been discussed many times. it’s discussed in the guide, and in the spec rationale, and in the doc string.
2017:03:21 15:31:56 idoa01 every one who uses clojure-spec to validate REST API calls
2017:03:21 15:32:05 alexmiller no, I mean for a particular application
2017:03:21 15:32:22 alexmiller if someone passes bad input, it could yield an error response
2017:03:21 15:32:27 alexmiller how is that different than any bad input
2017:03:21 15:32:36 alexmiller that’s the whole point of validation?
2017:03:21 15:32:46 idoa01 because I'm using clojure-spec to validate the input is not bad.
2017:03:21 15:32:51 alexmiller but it IS bad
2017:03:21 15:33:02 idoa01 so clojure-spec should tell me it is.
2017:03:21 15:33:07 alexmiller it does?
2017:03:21 15:33:16 idoa01 not run arbitrary code that might harm the machine
2017:03:21 15:33:25 alexmiller how can it harm the machine?
2017:03:21 15:33:32 alexmiller you chose to load the “arbitrary code”
2017:03:21 15:33:44 alexmiller this is not code some attacker is supplying to you
2017:03:21 15:33:50 idoa01 stackoverflow, memory consumption, cpu overload. you name it.
2017:03:21 15:34:11 alexmiller all things that can happen from an invalid input also sent to a “bad” validation function
2017:03:21 15:34:26 idoa01 it all depends on what is written in a faulty spec, that i didn't say i want to use.
2017:03:21 15:35:05 alexmiller I don’t see any way to actually cause memory consumption or cpu overload in dangerous ways. stackoverflow maybe, although I don’t have an example of that either.
2017:03:21 15:35:37 bronsa i think the claim being made is that the surface area of the possible "attack" (for lack of a better word) is potentially way larger with bugged specs than other bugged constructs
2017:03:21 15:35:42 alexmiller I have no examples of “faulty specs” that can cause improper machine resource usage.
2017:03:21 15:35:59 alexmiller what are the alternatives?
2017:03:21 15:36:19 alexmiller 1) don’t validate and don’t be aware of invalid data
2017:03:21 15:36:34 alexmiller 2) validate by using functions in either your code or libs
2017:03:21 15:37:01 alexmiller 1 does not seem better and 2 does not seem effectively different to me
2017:03:21 15:37:02 idoa01 allow me to not run any arbitrary spec code implicitly if i don't want to 😕
2017:03:21 15:37:12 alexmiller and why is that harmful?
2017:03:21 15:37:20 mpenet a spec predicate that does something over the network (call a db?), parses string content (or whatever resource heavy operation you can think of)
2017:03:21 15:37:25 idoa01 why would I want to run it?
2017:03:21 15:37:42 mpenet not your average predicate, but not too crazy either
2017:03:21 15:42:18 rauh IMO it'd be nice to have multiple repositories, for instance, I'm using :db/id, :object/id and :file/id in my application code. Down the road when many libraries are spec'ed this will get trampled and lead to issues. Or am I missing something?
2017:03:21 15:43:27 mpenet that's not so much of an issue with ns aliases + proper namespacing
2017:03:21 15:44:47 mpenet it's the same problem with project names/ns/package. that said multiple repos would be nice for other reasons
2017:03:21 15:51:42 alexmiller @rauh you are missing the use of proper namespacing :)
2017:03:21 15:57:19 yonatanel @alexmiller Would you say datomic attributes should be qualified with a namespace you own, or is :album/year enough?
2017:03:21 15:58:50 yonatanel I see spec and datomic go hand in hand. Correct me if I'm wrong.
2017:03:21 16:04:37 alexmiller @yonatanel same advice as spec. if you’re providing data for use with others, you must use a qualifier that you “control" (reverse domain, trademarked entity, etc). if the data will be used only in an environment that you control, it must only be “sufficiently unique"
2017:03:21 16:05:37 alexmiller so in a generic open source library, use a qualifier you control. If confined in your app, do whatever you want. If in an organizational context, you might need to ensure uniqueness in your organization.
2017:03:21 16:14:41 moxaj @alexmiller care to comment on the snippet above? How should one defend against an 'attack' like this?
2017:03:21 16:27:13 alexmiller Don't call valid?, don't load this spec, don't use s/keys, or use select-keys to pre-filter what you look at
2017:03:21 16:34:14 alexmiller Again, also compare this with what you would do without spec too - is that prone to the same issue?
2017:03:21 16:35:15 moxaj well, without spec, I wouldn't check for an ::x key within my input, so no
2017:03:21 16:46:55 alexmiller so is it better to notice the bad input or to pass 10000 nested maps around your system?
2017:03:21 16:50:55 dergutemoritz (provided your JSON parser didn't blow up with a StackOverflowError before that point anyway, hehe)
2017:03:21 16:51:25 dergutemoritz (or whatever your input format is)
2017:03:21 16:51:28 tbaldridge And "attack" is a bit of a strong word here, considering it's an exception, not a granting of root privileges or something like that.
2017:03:21 17:00:31 moxaj It isn't a bad input though. My spec says it might have an 'a' key and that's it. I do not care about the rest, if I did, I would have specified that in my spec.
2017:03:21 17:01:52 moxaj @tbaldridge I agree about the attack part, hence the quotes (I'm pulling a trump here lol)
2017:03:21 17:31:13 alexmiller @moxaj that’s not what spec says that spec means
2017:03:21 17:32:00 alexmiller that spec says “a map that might have an ::a key, and where all values are valid according to the spec of their keys"
2017:03:21 17:32:17 sparkofreason Is there any way to define coll-of specs such that the associated generator would yield a collection type such as PersistentQueue, sorted-set, etc? I assume I could do it with a custom generator, just wondering if there's a shorter path.
2017:03:21 17:32:29 alexmiller @moxaj if you want what you said, then just map? is sufficient
2017:03:21 17:33:03 alexmiller @dave.dixon look at the :into key
2017:03:21 17:34:20 moxaj The keys spec is perfect for me, except for the implicit part. But I guess that's not subject to change, so no point in arguing :)
2017:03:21 17:35:43 alexmiller well as I said above, an enhancement ticket that separates parts of keys seems like a reasonable idea
2017:03:21 17:37:37 alexmiller I do not know how Rich would react to it, but “decomplecting” is usually a good thing :)
2017:03:21 17:38:19 otfrom I do a lot of ETL stuff where I start with a pile of strings and turn them into some sort of seq of data structures. The strings encode things like a map of key to string or key to set of numbers.
I've had some reasonable success in using spec to check that the strings are of a format that is coercable to something "100" -> 100 or 100,101,102 -> #{ 100 101 102 }
Is this a terrible, no good, pls stop use of spec?
2017:03:21 17:39:10 alexmiller are you checking that they are coercible or actually doing the coercion?
2017:03:21 17:40:36 otfrom I've got a function that checks s/valid? using the spec and then s/conform using the same spec
2017:03:21 17:41:05 alexmiller so you are transforming the data via the conform
2017:03:21 17:41:09 otfrom yes
2017:03:21 17:41:14 otfrom (which I think is the bad bit)
2017:03:21 17:41:28 alexmiller so the general caveat for stuff like that is not that it’s necessarily bad but that it has consequences that you should understand
2017:03:21 17:41:48 alexmiller namely that users of that spec cannot recover the original data (as you could via s/unform normally)
2017:03:21 17:42:08 alexmiller and if you put that spec in the registry, you’ve made that choice for consumers of the spec
2017:03:21 17:42:26 alexmiller if “consumers of the spec” == you, then that’s up to you :)
2017:03:21 17:43:40 alexmiller if you’re using conformers, and if the function is reversible, you can supply an unform function in conformer to retain the reciprocal nature of that (assuming it is worth the trouble for you, which it may not be)
2017:03:21 17:45:29 otfrom I've not seen an unform example that does that yet. That would be interesting. After talking to seancorfield about it quite a bit I've been building up quite a few generators for testing things that have done a good job of driving out some bugs in my functions.
2017:03:21 17:46:02 otfrom would I be able to use an unform function like that inside a generator for testing?
2017:03:21 17:46:22 otfrom (and this is gold dust to me so thx. I'm glad it is a trade off I should think about rather than a terrible idea)
2017:03:21 17:46:52 otfrom (and I realise this spark/hadoop/ETL strings -> data is not every domain)
2017:03:21 17:48:01 sparkofreason @alexmiller Thanks, that worked. Docs make it sound like :into is limited to [], (), {}, #{}, should have just tried it.
2017:03:21 17:49:24 alexmiller it’s just functions yo
2017:03:21 17:49:41 otfrom 🤗
2017:03:21 17:50:51 alexmiller Rich would also caution you against treating spec as a transformation engine rather than something that validates declarative statements about your data (ie “it’s not a meat grinder”).
2017:03:21 17:51:07 alexmiller but that capability is there for your abuse :)
2017:03:21 17:51:24 otfrom I hadn't seen that conformer done that way
2017:03:21 17:51:44 alexmiller @dave.dixon actually, I may have led you into the path of undefined behavior there
2017:03:21 17:52:11 alexmiller I think you’re right that the intention was to only support that fixed set in :into for the time being
2017:03:21 17:53:11 otfrom alexmiller this is the "use clojure.core for this kind of thing" comment I've seen floating about
2017:03:21 17:53:32 otfrom IIUC
2017:03:21 17:54:08 alexmiller yeah
2017:03:21 17:54:53 otfrom I think I'm still searching for a better way of turning other peoples strings into data.
2017:03:21 17:55:29 otfrom (and dividing the invalid from the valid ones and reporting on the invalid ones well)
2017:03:21 17:55:52 otfrom I quite liked the error msgs that I could get out of spec around what had failed when I tried to validate
2017:03:21 17:56:45 alexmiller @dave.dixon there is some ambiguity about whether (I think due to impl churn) about whether :kind is supposed to be a function or a spec. Only a function works now, but the doc implies a spec (which could itself have a custom generator). There is a ticket regarding this issue and I haven’t yet gotten Rich to respond to me about it. :)
2017:03:21 17:57:03 otfrom thx alexmiller
2017:03:21 17:57:09 alexmiller np!
2017:03:21 17:59:06 sparkofreason @alexmiller Thanks, that answers my next question. Will probably just go with it for now since it seems to work, and the only other option would appear to be to write a custom generator for every "non-standard" collection spec.
2017:03:21 18:46:18 yonatanel What's that book/paper spec was inspired by?
2017:03:21 18:51:08 hiredman are you talking about the parser?
2017:03:21 18:55:21 hiredman racket has a contract system which, while I don't think I have heard anyone on the core team say was inspiration, a lot of people just assume it must have been
2017:03:21 18:56:41 hiredman the way spec validates sequential data structures is by "parsing" them using a parser based on parsing with derivatives (also a racket connection there)
2017:03:21 19:00:09 yonatanel yes, parsing with derivatives. Thanks!
2017:03:21 19:42:04 Yehonathan Sharvit @yonatanel Here is an interactive article that I wrote about the basics of “parsing with derivatives"
2017:03:21 19:42:13 Yehonathan Sharvit With interactive clojure code snippets
2017:03:21 19:52:33 yonatanel @viebel Yeah it's one of the first google results for parsing with derivatives...
2017:03:22 08:30:11 mpenet spec made me use derive for the first time in 7 years 🍾
2017:03:22 08:32:27 yonatanel @mpenet What did it solve? :)
2017:03:22 08:34:18 mpenet trying a new approach to generate json-schemas from specs, I have a registry of specs/preds -> json-schema forms that's a multimethod and that allows me to have some form of inheritance
2017:03:22 08:36:24 yonatanel Can you show an example? I'm not sure what json-schema is or how you convert from spec to json-schema.
2017:03:22 08:36:45 mpenet json-schema is the format used by swagger/open-api
2017:03:22 08:36:52 mpenet well part of
2017:03:22 08:37:23 ikitommi does the end result look different than the spec-tools conversion? (also uses a multimethod, but no inheritance atm)
2017:03:22 08:38:10 mpenet not sure, I didn't look into spec-tools internals too much. What do you mean by end result?
2017:03:22 08:40:18 mpenet looks very similar, but to be able to use derive you need to have ns'ed keywords
2017:03:22 08:40:33 mpenet the same can be achieved with a macro tho, nothing too fancy
2017:03:22 08:41:18 mpenet stuff I did is also an experiement with CLJ-2112
2017:03:22 08:41:48 mpenet I am using a slightly modified/fixed version of that patch
2017:03:22 08:42:35 yonatanel @ikitommi That's a nice lib :+1:
2017:03:22 08:47:41 ikitommi derive looks nice. should we at some point merge the working stuff into XYZ? would like to copy-paste the some of the missing mappings from Spex already.
2017:03:22 08:48:02 mpenet feel free to do that, spex is just a test-bed really
2017:03:22 08:49:42 mpenet seems like spec-tools covers a lot more already tho
2017:03:22 08:51:19 ikitommi it’s missing the Coercion protocol (instead of dynamic conforming), after it I’m kinda happy with what tries to resolve. the “drop-extra-keys from maps” things etc.
2017:03:22 08:55:36 ikitommi the 2112 would be awesome, it’s messy without that. Just realized yesterday that one can use and and or with s/keys...
2017:03:22 08:56:39 mpenet these are different approaches really, spec-tools seems to wrap all specs with it's own types to add "metadata", coercion etc, I just force the user to register its specs (and provide some defaults) and don't bother with coercion at all at this level (imho these are 2 different things)
2017:03:22 08:56:48 ikitommi @yonatanel thanks!
2017:03:22 08:57:06 mpenet spec-tools is full of cool stuff, didn't see the spec maps format thing
2017:03:22 08:57:32 mpenet yeah or/and in keys is a bit of a pita, not sure it's feasible to translate in json-schema
2017:03:22 08:58:15 mpenet CLJ-2112 code conforms it happily, but it's just ignored for convertion down the road
2017:03:22 08:58:54 ikitommi spec-tools doesn’t force to wrap into own records, JSON Schema is extracted from normal spec forms. Own records can be used to add metadata.
2017:03:22 09:03:46 mpenet I guess for or/and in keys one should/could just flatten all of it when generating a schema
2017:03:22 12:48:44 mpenet I think I like spec-tools approach better, records make all of this way nicer internally, too bad core.specs are not defined this way
2017:03:23 12:15:49 danbunea Hi, I am trying to do a simple function check using spec in ClojureScript
(:require
[cljs.spec :as s]
[cljs.spec.test :as test]...
(defn my-index-of
"Returns the index at which search appears in source"
[source search]
(clojure.string/index-of source search))
(s/fdef my-index-of
:args (s/cat :source string? :search string?)
:ret nat-int?
:fn #(<= (:ret %) (-> % :args :source count)))
and (test/check 'my-index-of) instead of returning a problem , always returns [] meaning basically that it doesn't do anything?
2017:03:23 12:40:23 thheller @alexmiller is that a bug? I suspect the problem is in with-name which is called in def-impl. deftype isn't IObj, defrecord is, as is reify which is used everywhere in spec
2017:03:23 12:40:49 yonatanel I've added a keyz spec that's like s/keys but with :deny :rest that will allow only keys in :req, :req-un, :opt and :opt-un
2017:03:23 12:41:43 yonatanel It's currently just a hack and doesn't have nice explain data. That will require implementing Spec just for the deny part.
2017:03:23 12:48:50 thheller @yonatanel was that intended for me?
2017:03:23 12:52:13 yonatanel @thheller No, but now that I saw what you wrote, are we working on the same thing?
2017:03:23 12:53:07 thheller hehe had me confused for a second there 🙂
2017:03:23 12:53:41 thheller not exactly the same thing, but I thought about adding the do not allow unspecified keys thing
2017:03:23 12:54:35 thheller I just have a bunch of old data and otherwise open maps that I can't easily create namespaced keys for
2017:03:23 12:55:32 thheller you could just use s/and
2017:03:23 12:55:35 yonatanel @thheller Isn't :req-un enough for that?
2017:03:23 12:56:03 thheller :req-un means that I add ::foo which I must s/def somewhere
2017:03:23 12:56:21 thheller if I have :foo with 2 different meaning that gets annoying
2017:03:23 12:56:41 yonatanel Regarding just using s/and I am doing it, combining s/keys and a set predicate on the map keys, but the explain message is crap.
2017:03:23 12:57:49 thheller ah right, well if you stick to defrecord and avoid the headache that caused the impl for explain/conform is rather straightforward
2017:03:23 12:57:52 yonatanel @thheller Got you. I wanted to do the same thing, so would be nice if you release your inlined map spec in github.
2017:03:23 12:58:47 thheller need to figure out unform and the gen stuff
2017:03:23 12:58:55 thheller but conform/`explain` work just fine
2017:03:23 13:02:46 thheller ah didn't know about that, thx
2017:03:23 13:05:01 yonatanel I like how you implemented explain unlike spec which creates a contains? pred for each key.
2017:03:23 13:06:52 thheller yeah dunno map-spec-impl looked way too complicated for what I want
2017:03:23 13:07:26 thheller started out by using it but made things way too complicated
2017:03:23 13:12:20 hospadar funny that ya'll are talking about this right now
2017:03:23 13:12:30 hospadar I am implementing something very similar as we speak
2017:03:23 13:19:04 hospadar I have to validate a datastructure that already exists in some systems that's
a) not using namespaced keys
b) not always using keywords as the keys for maps
2017:03:23 13:20:00 hospadar and sometimes the value types of the (non-namespaced) keys are conditional on something else
2017:03:23 13:21:15 hospadar i.e.
{:job-type :simple
:output "is a string}
{:job-type :complicated
:output {:is "a map of something}}
2017:03:23 13:21:16 thheller @yonatanel added the closed version
2017:03:23 13:22:34 thheller might not be the best idea but I want to use that to validate react component props, it might actually be better to use closed maps there
2017:03:23 14:33:14 yonatanel @hospadar Are you using multimethods for that?
2017:03:23 14:39:31 yonatanel @thheller Do you think the closed map explain data should show all the allowed keys instead of just the illegal one from the input?
2017:03:23 14:57:24 hospadar @yonatanel I tried using a multi spec that points and keys specs in different namespaces but that seemed to not work the way I needed
2017:03:23 15:04:17 hospadar ^ that basically does what I want
2017:03:23 15:04:24 hospadar but I didn't like the namespace hopping
2017:03:23 15:04:50 hospadar ohhh, I guess I could probably s/def with namespaced keywords instead of using :: to autoresolve?
2017:03:23 15:04:52 hospadar duh
2017:03:23 15:06:34 hospadar learning all the time 🙂
2017:03:23 15:14:32 hospadar ohhhh yeah that's a lot nicer
2017:03:23 15:14:47 thheller @yonatanel not sure about the explain data, good error messages are hard. could put anything in there really
2017:03:23 16:46:35 spieden @hospadar i think that’s typical usage as specs are meant to augment a separate namespace that contains the keywords
2017:03:23 16:46:49 spieden although i do use them inline sometimes too
2017:03:23 17:35:09 anmonteiro how does clojure.spec/instrument play with direct linking?
2017:03:23 17:35:32 anmonteiro wondering if someone has tried that before I go and play with it
2017:03:23 17:36:48 alexmiller it won’t do anything
2017:03:23 17:37:00 alexmiller because direct linking doesn’t use vars
2017:03:23 17:37:46 anmonteiro @alexmiller got me a little confused there. by “won’t do anything” do you mean it won’t instrument or it won’t be impacted by direct linking?
2017:03:23 17:46:50 alexmiller I mean it won’t instrument
2017:03:23 17:48:15 alexmiller or rather it will instrument, but direct linked calls don’t use either the original var or the instrumented var, so instrumentation will have no effect on those call sites
2017:03:23 17:48:42 alexmiller new, non direct linked invocations will use the instrumented vars
2017:03:23 17:51:00 alexmiller @thheller re your question yesterday, there is an implicit assumption in the code that spec impls also implement IObj. so in one sense, I’d say your example is at fault of that. And in another sense that spec code could fall through to throwing an error in that case to better let you know that.
2017:03:23 17:53:24 anmonteiro thanks
2017:03:23 18:13:58 mpenet Direct linking in dev makes little sense anyway doesnt it?
2017:03:23 18:14:47 mpenet (since instrument is also dev time)
2017:03:23 18:14:48 seancorfield Agreed. We only enable direct linking on QA/prod. It gets in the way of REPL-driven development.
2017:03:23 18:17:12 thheller @alexmiller didn't know about the implicit assumption about IObj, took me about 2h to figure that out. an error would have helped a lot.
2017:03:23 18:48:44 alexmiller Ticket would be fine
2017:03:23 18:49:20 alexmiller @mpenet right, there are no plans to change wrt direct linking
2017:03:23 19:54:31 anmonteiro @mpenet @seancorfield we have some functions for which we want to turn instrumentation on in prod, hence my question
2017:03:23 19:54:35 anmonteiro we don’t have direct linking in dev
2017:03:23 19:55:21 mpenet Dont do that. Instrument can/will trigger gen. Use pre/post
2017:03:23 19:55:36 anmonteiro that would be my next question
2017:03:23 19:55:53 anmonteiro we’re currently using Plumatic Schema
2017:03:23 19:56:08 anmonteiro and we have (s/set-fn-validation! true) for the namespace
2017:03:23 19:56:20 anmonteiro apparently instrument won’t do what I expect it to
2017:03:23 20:02:04 seancorfield I would say that if you want the checks to always be performed (i.e., in production) then explicitly code that into the functions themselves — that gives you more control over the error handling anyway.
2017:03:23 20:04:29 seancorfield As @mpenet notes, even if you did instrument functions in production, any higher-order functions that get instrumented would trigger generative testing, and instrumentation doesn’t check function returns anyway, just arguments.
2017:03:23 22:49:52 olivergeorge @anmonteiro would s/assert be a better choice for production code?
2017:03:23 22:51:12 anmonteiro hrm I thought s/assert‘s point was to be turned off in Prod
2017:03:23 22:51:35 anmonteiro Currently I’m thinking about going with a combination of s/valid? and s/explain that we send to Sentry
2017:03:23 23:51:44 olivergeorge Sounds sensible.
2017:03:24 10:37:10 manu I've just started learning clojure-specs. My question is:
what is the right place to put them? in a different namespace or right next to the production code?
2017:03:24 11:05:13 mpenet Personally I like to put them in a separate namespace, but there's no rule about it
2017:03:24 12:05:42 Yehonathan Sharvit what are the pros/cons of having in same/different namespaces?
2017:03:24 12:32:35 mpenet easier to exclude if it's in another namespace
2017:03:24 12:33:01 mpenet other than that it's a matter of taste, diff languages do it both ways (ex haskell vs ocaml)
2017:03:24 12:52:05 hospadar Q: is there any documentation of the Spec protocol?
I ask because I'm building a couple implementations of it to wrap up some of my more complex predicates. Reason being I want to generate custom error messages that are more descriptive
2017:03:24 12:52:48 hospadar I think I've mostly figured it out, but I'm mainly just guessing based on my reading of the implementations already in clojure.spec
2017:03:24 12:54:25 hospadar specific problem: I'm validating a tree-structure that's defined in an EDN file. Not only do I want to validate that the nodes are all structurally OK, I need to validate certain properties of the graph as a whole like "are all the edges pointing at valid nodes" "is it acyclic?"
2017:03:24 12:54:58 hospadar My first approach was to just write those validators completely outside of spec, but that makes them hard to compose (like if I pack this datastructure inside of another one)
2017:03:24 12:56:16 hospadar My second thought was "just use predicates", but an error message like {:pred (not (contains-cycles? graph)) :obj <the whole graph>} is super unhelpful to a user trying to figure out which node is invalid
2017:03:24 12:57:39 hospadar So instead I packed the predicates into a (reify Spec ...) and it's GREAT but it took me a while to figure out
a) that there was even a protocol that I could implement
b) how to implement it
2017:03:24 12:58:23 mpenet I did something similar (spec that validates a dsl: parses, potentially generates error messages with line/num syntax helpers etc)
2017:03:24 12:58:48 mpenet basically a reified Spec with a custom explain*
2017:03:24 12:58:56 mpenet + some caching not to parse again for explain
2017:03:24 12:59:06 hospadar what I guess I'm really angling at is "If I wanted to contribute some helpful documentation of the spec protocol, would there be interest in receiving that, and how would I go about doing it?"
2017:03:24 13:00:07 hospadar I have some other related minor thoughts like "maybe explain-1 shouldn't be private so that Spec implementors can use it without having to #'hack/it"
2017:03:24 13:00:33 hospadar yeah @mpenet same thing for me too
2017:03:24 13:00:56 hospadar I have a little custom lispey transform language I'm using to do some basic filtering that gets packed inside other specs
2017:03:24 13:02:23 mpenet I am not sure what's the status of the Spec protocol, might be considered an (unreliable) implementation detail. Not sure but at the time @alexmiller advised me to do this so I hope not
2017:03:24 13:03:20 hospadar yeah I suspected maybe that was the case
2017:03:24 13:03:25 hospadar (or might be)
2017:03:24 13:07:32 hospadar maybe a better approach would be extending clojure.spec/spec to take an (optional) custom explain function
2017:03:24 13:07:40 hospadar in the same way it takes an optional generator
2017:03:24 13:07:57 hospadar it would probably make my code look a little simpler 😛
2017:03:24 13:13:30 mpenet it would be nice yes
2017:03:24 13:13:32 alexmiller @hospadar the Spec protocol is subject to violent change without warning and we’re not interested in documenting it at the current time
2017:03:24 13:14:04 alexmiller we may not even keep it
2017:03:24 13:16:10 alexmiller I don’t think Rich is interested in having custom explain messages (but I could be wrong)
2017:03:24 13:23:16 alexmiller I think Rich’s conception is that in general, people should not be writing their own spec impls, they should be composing the provided specs
2017:03:24 13:27:56 hospadar Use that trick in my project.clj to parameterize versions
2017:03:24 13:29:21 dergutemoritz That only works if the macro explicitly allows it
2017:03:24 13:29:21 hospadar @alexmiller I feel that, I'd really rather not be writing custom preds. But in some cases I have to and it's nice if I can pack them into my specs with more error detail than just "this predicate failed"
2017:03:24 13:31:08 mpenet well you can (eval (s/def ::foo ~(...)), which is quite horrible
2017:03:24 13:31:24 souenzzo
(eval `(s/def ::my-keys (s/keys :opt ~my-keys)))
works, but (s/def ::my-keys (s/keys :opt ~my-keys)) no
2017:03:24 13:32:10 mpenet hopefully in the future we can conform the spec form, modify it, then unform it
2017:03:24 13:32:31 yonatanel @souenzzo What's your use case for defining keys separately? Are you reusing them?
2017:03:24 13:33:03 mpenet wouldn't hold my breath tho
2017:03:24 13:52:18 yonatanel @hospadar Would recursive spec solve your issue of informative tree validation explain message?
2017:03:24 13:54:40 alexmiller CLJ-2112 is work that Rich asked me to do - that’s something we will definitely have
2017:03:24 14:33:22 hospadar @yonatanel sometimes yes, sometimes no (at least I'm not sure how I'd implement that right now?)
2017:03:24 14:33:30 hospadar I'd love to be proven wrong though
2017:03:24 14:35:04 hospadar that's a super-dumb simple example of the kind of problem I'm trying to validate
2017:03:24 14:36:18 hospadar it's pretty easy for me to write a validator (in the form of a predicate) that just reduces over the edges to check their validity, and I can s/and that together with the structural spec at the top level
2017:03:24 14:37:02 hospadar but I want to get a more helpful error message that says "this edge failed validation because this node isn't valid"
2017:03:24 14:38:38 hospadar hence why I've started writing implementations of Spec that let me emit more specific error messages
2017:03:24 15:13:07 yonatanel @hospadar I wonder if there should be a line between what you should validate with spec and what should go to another validator, other than runtime dependencies such as a database. In your case the rules might be a bit complex but you don't need anything but the data, so spec sounds good, but you can't compose yourself out of every case, so if spec already supports custom predicates and generators, why not explainers?
2017:03:24 15:45:00 hospadar yeah that's my thought - if I was going out over the wire during validation ("only valid if exists in DB" situations) I'd probably not attempt to jam it in the spec
2017:03:24 15:46:24 hospadar spec is very attractive to me for this kind of problem for me though since it makes it easy to build composable validators and still get helpful robust error reporting
2017:03:24 15:50:09 hospadar I feel like spec should be able to validate (with useful errors) any data structure where the correctness of the data-structure doesn't rely on any external factors?
2017:03:24 16:02:13 hospadar (i.e. the validation is very deterministic data->spec->answer is the same every time)
2017:03:24 19:10:01 luxbock I kind of wish that a spec defined using s/and did not perform conforming while passing the value through its specs
2017:03:24 19:11:21 luxbock I find myself defining named predicates that add additional constraints on what the value should be like using s/and, and it would be more natural for those predicates to receive the unconformed value
2017:03:24 19:13:07 luxbock I know I could in theory use s/unform inside my predicates, but often times I already have an existing predicate that I use elsewhere as a part of the business logic of the program, so now I have to create a wrapper in order to use it with spec
2017:03:24 19:22:30 luxbock
user> (s/explain (s/and (s/cat :a int?) (fn [[a]] (> a 1))) [2])
UnsupportedOperationException nth not supported on this type: PersistentArrayMap clojure.lang.RT.nthFrom (RT.java:962)
2017:03:24 19:41:56 alexmiller @luxbock we have talked about providing both flowing and non-flowing versions of s/and
2017:03:24 19:44:06 seancorfield @alexmiller and something for s/or that doesn’t produce pairs? (so we don’t need (s/conformer val) when we don’t care about the label)
2017:03:24 19:44:40 alexmiller well s/nonconforming exists as a theoretical answer to that
2017:03:24 19:44:51 alexmiller but I’m not sure whether Rich considers it a good idea
where fn-string-from-regex is a wrapper for test.chuck‘s string-from-regex generator.
2017:03:24 20:57:03 seancorfield It generates some really wild email addresses 🙂
2017:03:24 20:57:48 tjtolton ooh
2017:03:24 20:57:52 tjtolton string from regex
2017:03:24 20:57:54 tjtolton that's neat
2017:03:24 21:12:35 seancorfield @tjtolton Depending on what you’re doing with generated emails, you might want a simpler regex for the generator portion — here’s just two examples from s/exercise: ”𧶊@[858.0.376.98]" ”\"𱢆\"@5cl84.e.a2.QN-.hpr1s.H.Ikp"
2017:03:24 21:12:56 seancorfield Hmm, the unicode didn’t survive the copy’n’paste...
2017:03:24 21:13:39 seancorfield Anyway, you get the idea.
2017:03:24 21:19:59 tjtolton Yeah, I do, thanks!
2017:03:24 21:20:09 tjtolton Test.chuck is pretty neat
2017:03:24 21:33:32 don.dwoske Newbie here. I’m trying to write a multi-spec to handle a case like this:
For a particular type value, I want the acceptable properties keys to be different, while every instance has an :id, :type and a few other fields. The properties are variable, but not the parent/core keys. In the examples I’ve seen for multi-specs the type is always inside the variable collection itself.. e.g. I’d need to move type into properties instead of leaving it at the top level.
2017:03:24 21:59:54 kidpollo I think that you can perfectly use a multi-spec with the type where it is. Your entity specs will all have id, type and properties. No need to complicate things IMO
2017:03:24 22:23:36 don.dwoske likely - but as I said - newbie - I don’t know how to do it. I’m fooling around with using a nested s/spec in there to get the properties map returned for each type in the multi-spec.
2017:03:25 13:01:32 yonatanel @don.dwoske Where do you get this data from? Can you use :person/properties and :house/properties instead or do you treat ::properties as just a bag of key-value pairs? Or maybe use :properties without namespace so you can have the same keyword with different specs such as (s/def :person/properties ...) and then (s/keys :req-un [:person/properties] ...).
2017:03:25 13:03:59 yonatanel Or perhaps be strict about a flat structure such as {::id 2, ::type "house" :house/color "red" :house/style "Victorian"}
2017:03:25 13:33:30 don.dwoske I’ve got a flat structure working decently now. Your idea to use different namespaces for each properties list is a good one, that should work I think. The spec is used to validate data with unqualified keys, so it fits. I only need the properties map name to be consistent across all types. We have not yet finalized our design for these payloads, but if I can get the specs working both in flat and nested properties format, that would be excellent.
2017:03:25 13:38:34 don.dwoske BTW, for those interested, I’m building out a proof-of-concept to compare using clojure.spec on an enterprise level to define and validate our entity types vs. something homegrown vs. XSD, vs. JSON Schema vs. Avro vs. whatever else. Barring any setbacks, I may propose using clojure.spec as the place-of-truth for these, and using tools to translate to any other format we may need.
2017:03:25 13:41:26 don.dwoske I’d be curious also to know if there is a way to use clojure.spec to define a web API contract and then auto-generate swagger docs from it.
2017:03:25 15:52:15 mpenet The json schema stuff in spex is far from finished. Wouldnt rely on it
2017:03:25 15:52:18 mpenet We might just use graphql instead actually
2017:03:26 12:23:01 don.dwoske @U050SC7SV we are looking at graphql as well - any particular database or libraries in mind?
2017:03:26 12:25:11 mpenet We use a mix of ES and cassandra, but graphql usage doesnt really depend on a particular datastore. I am experimenting with the lib from Walmartlabs, so far impressed
2017:03:25 15:54:01 devth I thought you could lookup :ret/`:args` on a spec, e.g. (:ret (s/get-spec 'full/symbol)) but this is just returning nil 🤔
2017:03:25 15:54:57 devth the type of the spec is clojure.spec$or_spec_impl$reify__13856
2017:03:25 16:29:04 devth right so i wasn't trying it on an fdef spec. i was trying it on an s/keys spec. however, still can't seem to get much useful info out of it. (e.g. which keys are specified as :req vs :opt)
2017:03:25 23:47:06 hiredman You have to call s/form and parse the form
2017:03:26 23:33:33 hueyp warning: new to spec 🙂 I discovered conformer and have a case where I’m using it purely to transform the conform output. Found this about that pattern (re: hiding the spec inside conformer): "You could keep the conversion out of the spec and do that via a transformation or via a non-registered spec too.”. Anyone have an idea on the ‘via a non-registered spec’ option? 🙂
2017:03:27 00:07:47 hueyp unsure if good / bad idea 🙂
2017:03:27 02:39:24 seancorfield @hueyp As someone who uses s/conformer quite a bit, I'll caution against specs that transform their data (beyond the conforming that the specs themselves do). Alex Miller and other Cognitect folks also caution against mixing transformation into your specs. The problem is that once you put that in a spec, all client uses of the spec have no choice about the transformation -- they all have to deal with the conformer in the spec.
2017:03:27 02:40:24 hueyp I’m building a parser and was kind of blown away that with conformer you could totally have it output the AST 🙂
2017:03:27 02:40:36 seancorfield We do it for a very specific use case: where our input is always strings (from an HTTP request) and we want numbers or keywords out of the specs (and they're coded to accept numbers or keywords too, as well as strings).
2017:03:27 02:41:02 seancorfield Yeah... just because you can do something, doesn't mean you should 🙂
2017:03:27 02:41:24 hueyp exactly why I’m here ! 🙂 thx for feedback~
2017:03:27 02:42:11 seancorfield It's likely to be better for you down the road, to separate the transform to AST from the specs. Otherwise you're complecting validation (specs) with transformation. If you keep them separate, they'll be easier to test and potentially easier to reuse.
2017:03:27 02:45:18 seancorfield Spec is great tho'... we've been using it in production code for quite a while (we have Clojure 1.9.0 Alpha builds in production).
2017:03:27 17:20:08 jiangts i’m specing a datastructure in clojurescript
2017:03:27 17:20:26 jiangts and the keys are coming in as strings
2017:03:27 17:20:57 jiangts I’d like to write the spec with s/keys but that only supports keywords. Am I missing something simple?
2017:03:27 17:30:46 jiangts thanks! I haven’t used every-kv, but as far as I know using map-of I can’t specify that certain keys are required and others are optional
2017:03:27 17:32:02 jiangts can that be done with every/every-kv?
2017:03:27 17:40:56 xiongtx You’ll have to use some logical combinations w/ s/and and such, I think
2017:03:27 17:43:15 zane Pretty sure the recommended approach is to conform the keys to keywords. Are you worried about performance?
2017:03:28 04:25:11 jiangts yeah, worried about performance, a lot of nested maps with strings as keys
2017:03:28 04:26:05 jiangts perhaps I can replace keys with keywords with specter or something
2017:03:28 04:27:02 jiangts but yeah, will be converting a couple thousand string keys to keywords, nested to ~20 levels deep, so would like to avoid if possible
2017:03:27 17:21:27 jiangts I’d rather not coerce the data to use a bunch of keywords cause I have a lot of data
2017:03:27 21:33:14 liamd hiya are there any good reference projects or tutorials on how to integrate spec into real world projects
2017:03:27 21:33:29 liamd most of the examples i’m digging up explain how it works well but seem “contrived” for lack of a better term
2017:03:27 21:33:40 liamd (e.g. modeling decks of cards)
2017:03:27 21:49:27 seancorfield @liamd I suspect most “real world projects” are closed source and can’t easily be discussed...
2017:03:27 21:49:57 liamd i mean is spec being used in any popular open source libs?
2017:03:27 21:50:23 liamd i guess i’m looking for “best practices”. do i create a new file for my specs, do i do runtime checks with them, etc.
2017:03:27 21:58:44 seancorfield "popular open source libs” are nearly all remaining compatible with earlier Clojure versions...
2017:03:27 21:59:04 seancorfield It’s too early for much in the way of best practices with spec to be codified.
2017:03:27 21:59:29 seancorfield We’re unusual (at World Singles) in being willing to run Alpha versions of Clojure in production.
2017:03:27 22:00:32 seancorfield For libraries that need to support earlier versions of Clojure, specs must be in a separate file so that folks on Clojure 1.9 can require it, but folks on earlier versions can ignore it.
2017:03:27 22:00:50 seancorfield For apps that run on Clojure 1.9, they can have specs inline with functions.
2017:03:27 22:01:46 seancorfield We’ve found the most value so far comes from specifying data not functions. We use spec to validate input parameters for our REST APIs and we’re starting to use it to validate user input in some web apps as well.
2017:03:27 22:02:08 seancorfield So, yes, for us it is runtime checking in production.
2017:03:27 22:03:54 seancorfield But it’s also part of testing — that’s where we instrument code (prior to running our test suite, for example) and some limited automation of generative tests (beware of the increased time such testing takes — consider running it separately from your “unit tests”, so that it can be run “as needed” by developers and as long form testing in CI perhaps).
2017:03:27 22:04:06 seancorfield Does that help @liamd ?
2017:03:27 22:04:17 liamd yes absolutely
2017:03:27 22:04:41 liamd so what i’m gathering is that standard practices haven’t been established yet necessarily because it’s really still in alpha
2017:03:27 22:04:53 liamd i suppose in that case i just have to play around and see what works for me
2017:03:27 22:05:56 liamd it seems like there’s lots of space for libraries to be developed on top of spec to enforce certain uses and patterns, like perhaps ring middleware, or test suite integration or cleaner errors from explain
2017:03:27 22:08:08 seancorfield Our experience so far re “cleaner errors” is that requires application domain knowledge, so we’ve created some custom code that maps from explain-data to our application domain, using some heuristics that work for the sorts of spec failures we might reasonably get.
2017:03:27 22:08:39 seancorfield As for the rest of it, yeah, I think it’s just too early for much to have solidified.
2017:03:27 22:08:54 liamd that’s actually pretty exciting, i guess i’ll just have to jump in
2017:03:27 22:08:58 liamd thanks!
2017:03:27 22:10:19 seancorfield Will you be at Clojure/West later this week?
2017:03:27 22:15:22 liamd Nope, I’m in Chicago and don’t have the spare cash. Looking forward to the screen casts that come out of it though.
2017:03:27 23:00:06 sparkofreason Is there any way to control the length of keywords generated for the keyword? predicate?
2017:03:28 00:11:29 danboykis Can anyone explain this?
This works:
((fn []
(if-let [k nil]
k
:other-keyword)))
=> :other-keyword
But this doesn't:
((fn []
(if-let [k nil]
k
:clojure.spec/invalid)))
CompilerException clojure.lang.ExceptionInfo: Call to clojure.core/if-let did not conform to spec:
In: [2] val: :clojure.spec/invalid fails at: [:args :else] predicate: any?
...
The only difference being a keyword. I ran into this writing a conformer.
2017:03:28 00:14:27 seancorfield :clojure.spec/invalid is a special value — you have to be really careful about code that produces it
2017:03:28 00:15:33 seancorfield I think you can do (keyword “clojure.spec” “invalid”) as one way to get around that (i.e., generate it at runtime).
2017:03:28 00:15:47 seancorfield (I suspect there’s a cleaner way)
2017:03:28 00:17:22 danboykis seancorfield, I agree that it's special, I am using it in a conformer
2017:03:28 00:18:10 danboykis this is a stripped down example
2017:03:28 00:18:10 seancorfield It’s special to spec itself. And in 1.9, spec is used to syntax-check various macros.
2017:03:28 00:19:11 danboykis in your opinion, is this expected behavior?
2017:03:28 00:19:41 seancorfield Another option is
2017:03:28 00:23:44 danboykis oh ok great, someone filed it
2017:03:28 00:24:03 seancorfield Known since last June and no indication that it really is a problem that needs to be fixed (and it has an easy workaround — that I’d simply forgotten!)
2017:03:28 00:24:54 seancorfield (ironic that I was the one that provided the easy workaround in that ticket but I’d forgotten)
2017:03:28 00:25:13 seancorfield so (if-let [k nil] k ':clojure.spec/invalid)
2017:03:28 00:25:48 danboykis that's a cool workaround 🙂
2017:03:28 00:26:25 seancorfield Since macros are just functions on code-as-data, spec can’t tell the difference between a conforming spec that produces :clojure.spec/invalid — the special value — and a literal :clojure.spec/invalid in the actual code.
2017:03:28 00:26:25 danboykis i hope it get's fixed
2017:03:28 00:27:08 danboykis the docs should probably change then
2017:03:28 00:27:10 seancorfield I suspect the answer from the Clojure/core folks will be: if you’re writing a spec or a conformer that needs to produce the special invalid value, you must quote it.
2017:03:28 00:27:38 danboykis it's not at all obvious that you're supposed to quote it
2017:03:28 00:27:49 seancorfield You can return :clojure.spec/invalid in all sorts of constructs just fine. You only need to quote it in some situations.
2017:03:28 00:29:21 seancorfield We have all sorts of specs and conformers that produce :clojure.spec/invalid as a literal value — we just don’t have code around it that is a macro that is checked by spec in the compiler.
2017:03:28 00:30:13 seancorfield (if (some-check v) v :clojure.spec/invalid) works fine, as does (try (some-coercion v) (catch Exception _ :clojure.spec/invalid))
2017:03:28 00:34:28 danboykis i wonder what will happen once cond is spec'ed too
2017:03:28 00:34:50 danboykis you can return it now
2017:03:28 00:34:52 danboykis but not later?
2017:03:28 01:19:49 danboykis coming back to our discussion the crux of the issue is the following
2017:03:28 01:20:48 danboykis that's a contradiction
2017:03:28 07:11:32 athos Sorry for breaking into the discussion. Does anyone know it's a known limitation or an unintended behavior that multi-spec in general cannot be unformed?
2017:03:28 14:48:08 tjtolton Is there some kind of core.spec function for "generate a view of this data structure that conforms to this spec"?
for instance
2017:03:28 14:49:35 tjtolton conform is supposedly for destructuring, but perhaps my understanding of destructuring is wrong, because intuitively I feel like "select this known data pattern out of this unknown data" is what destructuring is for
2017:03:28 14:55:00 tjtolton Basically, the use case here is "web request should conform to this data, but if you pass in other data, that's perfectly fine. But when we store the data off in the database, we should only select the part that we understand and care about."
2017:03:28 14:56:01 tjtolton so, the spec's don't disallow keys, but we select the parts we're interested in for passing along
2017:03:28 14:59:04 borkdude Recently I sometimes see error messages like these:
adzerk.boot_cljs.util.proxy$clojure.lang.ExceptionInfo$ff19274a: ERROR: Attempting to call unbound fn: #'clojure.spec/macroexpand-check at file /Users/Borkdude/.boot/cache/tmp/…/app/1x42/fg7eat/public/js/app.out/ajax/core.cljc, line 591, column 1
I think it was after upgrading clojure.future.spec to alpha15
2017:03:28 14:59:45 borkdude Not sure where it comes from
2017:03:28 15:09:31 borkdude Now I get:
Attempting to call unbound fn: #'clojure.spec/macroexpand-check at line 2164, column 1 in file /Users/Borkdude/.boot/cache/tmp/.../app/1xx5/-x902hh/public/js/app.out/cljs/pprint.cljs
2017:03:28 15:12:37 borkdude Hmm, sorry, this was with alpha-14. Now I upgraded to clojure.future.spec alpha 15.
2017:03:28 15:47:39 tjtolton this is crazy, it doesn't even make any sense to me.
the guide here https://clojure.org/guides/spec seems to imply that conform is used for destructuring. Here's the example it gives:
We can see that "input" is returning from "conform" as "parsed". But it doesn't actually modify the input in any way, unless the input was nonconforming, in which case it returns a special value, which it then explicitly checks for
2017:03:28 15:48:14 tjtolton why not just do (if (s/valid? ::config input)?
2017:03:28 16:39:26 seancorfield Destructuring plucks out the parts you care about — but leaves all the other parts in place (which you can see with :as).
2017:03:28 16:40:08 seancorfield Clojure takes an inherently “open for extension” stance on that sort of thing: it’s considered good practice to be able to accept (and ignore) additions to data structures that still conform.
2017:03:28 16:40:55 seancorfield In order to save stuff to a database, you need to explicitly restrict the keys in a map — either with select-keys (ignoring the extra keys) or something that checks you’ve only get the expected keys and no others.
2017:03:28 16:42:31 seancorfield In general, when saving to a database (with java.jdbc), we tend to build the map we need from the conformed input map, so we explicitly select the columns we need and ignore the rest. ^ @tjtolton
2017:03:28 16:47:03 tjtolton yeah, that's a shame @seancorfield I'm totally down with the "open for extension" concept, but I'm also interested in reducing the cost of detection with spec problems.
2017:03:28 16:47:53 tjtolton Basically, permissive specs are great, but they let you do stupid things, like mispelling an optional key
2017:03:28 16:48:23 tjtolton if you mispell an optional key, you have no way of knowing about it. Nothing will ever fail it
2017:03:28 16:53:13 seancorfield I would expect code that called a function incorrectly like that to be caught by tests on the calling function.
2017:03:28 16:54:27 seancorfield i.e., if func-a calls func-b with misspelled arguments, tests for func-a should catch that.
2017:03:28 17:27:07 tjtolton so, its interesting, because I basically just want a select-keys, but one that is recursive and spec aware
2017:03:28 17:28:42 tjtolton It's still very strange to me that it isn't available. Even in an "open for extension" system, this seems like an important utility.
2017:03:28 17:30:35 tjtolton Especially for web programming. I only want to display these values on screen, I only want to return these values in the api call, I only want to save these values to the kv store
2017:03:28 17:30:41 tjtolton all things that spec is great at
2017:03:28 17:30:47 ikitommi @tjtolton in spec-tools there is a conformer that drops extra keys out of keys-specs. Will blog about those soon.
2017:03:28 17:30:51 tjtolton I should be able to reuse that functionality
2017:03:28 17:33:44 ikitommi tjtolton: yes, metosin. Just finishing up first release. Comments welcome!
2017:03:28 17:34:01 tjtolton awesome, will take a look!
2017:03:28 17:38:52 tjtolton in the strip-extra-keys function, what is that first argument? is that something that gets generated by core.spec?
(defn strip-extra-keys [{:keys [:keys pred]} x]
(if (map? x)
(s/conform pred (select-keys x keys))
x))
2017:03:28 19:08:05 ikitommi it's the Spec Record. Sadly, clojure.spec/Specs are not extendable and we need to wrap them to make them extendable. There is a example in the readme.
2017:03:28 17:32:41 tjtolton metosin?
2017:03:28 20:38:06 ddellacosta is there a way to enforce that a pair of keys exists in a map, or else neither? That is, if one of them exists the other is required, however they are both optional otherwise.
2017:03:28 20:41:20 schmee ddellacosta something like
2017:03:28 20:59:15 dergutemoritz Yeah, it's handy but has been the source of surprise in this channel more than once 🙂
2017:03:28 20:59:22 ddellacosta I can imagine
2017:03:28 20:59:41 dergutemoritz I actually wonder why and and or are not supported in :opt now ...
2017:03:28 21:00:17 dergutemoritz Your case makes it seem like a reasonable thing to expect
2017:03:28 21:26:36 seancorfield Note that or in :req isn’t mutually exclusive: (s/conform (s/keys :req [(or ::a ::b)]) {::a 1 ::b 2}) succeeds and produces {::b 2 ::a 1}
2017:03:28 21:27:17 seancorfield It just means "at least one of these must be present”. Both can be present in the normal “open for extension” model that spec has.
2017:03:28 21:29:05 ddellacosta @seancorfield yeah, in this case the problem is that I want these two to either be both present, or not at all
2017:03:28 21:29:12 ddellacosta but yeah, good to note
2017:03:28 21:29:28 seancorfield Since :opt just means “and here are some more keys you may encounter that have associated specs” — because additional keys are always accepted, they just won’t be assumed to have associated specs to conform to.
2017:03:28 21:29:51 ddellacosta right, that makes sense now that I’ve worked through this
2017:03:28 21:31:21 seancorfield @ddellacosta Yeah, the best you can do here is specify them both as :opt if you want them conformed, and then s/and a key check predicate around s/keys. We’ve run into this a lot with mutually exclusive keys — we have to have a separate predicate that checks at most one of them is present. Not sure whether I feel this should be built-in or not...
2017:03:28 21:31:54 ddellacosta yeah, that’s exactly what I ended up doing. Not the most elegant thing but it works well.
2017:03:28 21:32:17 ddellacosta I mean, I feel like @dergutemoritz ‘s suggestion would have done that very elegantly
2017:03:28 21:32:29 ddellacosta but maybe that’s challenging to implement, I dunno
2017:03:28 21:32:43 ddellacosta haven’t looked at the actual clojure.spec code so dunno
2017:03:28 23:56:36 mattly I'm using spec/conform to destructure & validate a largish data structure from a client, and at some point it's using coll-of. But in the process, conform seems to be reversing the contents of the list its given. Is this expected/normal?
2017:03:29 13:54:18 alexmiller Yes, sounds like that - that’s been fixed (can’t remember which alpha it was in). You might also look at every-kv (which will not conform its input at all).
2017:03:29 00:05:13 mattly weird, I can't reproduce it in the repl
2017:03:29 00:10:44 mattly it's definitely happening in the call to conform though
2017:03:29 00:14:08 mattly upgrading from alpha14 to 15 made it go away
2017:03:29 15:20:21 alex.ter.weele What’s the easiest way to make a spec that matches {:a 1 :b 1}, {:a 1}, and {:b 1}, but not {}?
2017:03:29 15:25:09 alex.ter.weele is that regular or or spec/or?
2017:03:29 15:29:49 bronsa it's s/keys syntax that expands to clojure.core/or
2017:03:29 15:30:24 alex.ter.weele ok, I got it. Thanks!
2017:03:29 17:28:27 manderson Is there a way to provide more detailed explain output for a custom conformer? In alpha15 I get a message like: val: "k" fails predicate: (conformer conform-int). I see I can supply an optional "unform", but no arity to supply an "explain"...
2017:03:29 17:47:38 bja is there a way to turn off fdef validation for defn temporarily in -alpha14? I'm seeing a failure in a third party lib that works in another project.clj so I think I'm having classpath issues and I need to load the namespace so I can interogate some objects via the classloader
2017:03:29 17:53:27 hospadar @manderson you can implement the Spec protocol to provide a custom explainer
2017:03:29 17:53:56 hospadar word on the street is that the protocol may be changed in the long run
2017:03:29 17:54:03 hospadar but for now it works fine
2017:03:29 17:55:48 hospadar if you look at the Spec implementations included in the lib (that are used for keys and tuples, etc) you can get an idea of what kind of output they return
2017:03:29 17:56:20 hospadar if errors, return a list of errors (like what you get from explain-data), otherwise return empty list/nothing
2017:03:29 17:56:51 hospadar that example assumes you don't want to do any special conformation
2017:03:29 17:57:35 manderson Gotcha, thanks @hospadar. Figured that might be the only way, but was hoping for a "published" approach instead of looking behind the curtain. I'll give that a try.
2017:03:29 18:14:01 hospadar yeah I feel that
2017:03:29 20:25:04 alexmiller @bja you can s/def your own spec over the top of the standard one
2017:03:29 22:53:40 bja @alexmiller thanks. that's what I eventually settled on
2017:03:29 22:54:26 bja found out that I had a conflict with an older version of potemkin generating non-conforming functions.
2017:03:30 14:59:04 cgrand Hi! Going meta: has anyone worked on generating (from a spec) the spec of it’s conformed data?
2017:03:30 15:17:59 zane No, but I've thought about writing that function and would use the hell out of it.
2017:03:30 18:53:49 alex.ter.weele on a related note, I’ve been wrestling with the "entry points” to spec. Once I learned that s/conform produces a new structure, e.g. when going through an s/or, I switched to using s/valid?. So what is the use case for s/conform? It sort of works like s/explain that produces an explanation on success too.
2017:03:30 19:56:38 adambros @alex.ter.weele my main usage of conform has been in macros to parse the body in a way that I can emit things in a much simpler way
2017:03:30 20:05:27 seancorfield @alex.ter.weele Producing a new, annotated data structure is really useful when your code needs to be able to branch on those or tags.
2017:03:30 20:07:10 seancorfield For us (World Singles), we use conform to process our REST API inputs (all string values) and process keywords, numbers, dates, and strings -- with annotations around ranges of values etc.
2017:03:30 20:07:14 cgrand @alex.ter.weele validate AND transform
2017:03:31 01:36:04 derwolfe Using spec, is there a way to specify something like a future or deferred that should contain a map with a set of keys? For instance, I'm using manifold/futures to make some calls to AWS; I'd like to write specs for these, but if they only represent that the functions return instances of manifold/futures, I'm not sure how valuable that would be.
2017:03:31 02:05:30 derwolfe Per the mailing list, it seems like this is similar situation to trying to spec protocol methods which seems to be discouraged in favor of wrapping protocol methods with functions and speccing those.
2017:03:31 02:10:04 seancorfield Yeah, it makes more sense to spec the functions involved that use the underlying data structures.
2017:03:31 02:29:39 derwolfe Thanks 😄
2017:03:31 09:46:10 yenda is there a way to remove everything that is not speced in a datastructure ? i.e the datastructure is valid but the legacy system behind it won't like extra fields to be present.
2017:04:03 19:49:28 mobileink yenda: the way i do this with maps is, read the registry to get all deffed kws, then walk your data, and when you hit a map, iterate over the entries. for each key, if it is not in the registry, discard the entry.
2017:03:31 10:15:49 yonatanel @yenda You need to use something like select-keys yourself
2017:03:31 10:16:54 yenda the problem with this solution is that it is repeating what is already in the spec, also it is hard/not-really-readable to recursively pick the keys in the structure
2017:03:31 10:23:04 mpenet spec-tools does I think
2017:03:31 10:23:14 yenda thanks
2017:03:31 10:23:48 yonatanel @mpenet I just like to link to your lib and see you apologize for it not being complete :)
2017:03:31 10:24:08 mpenet 🙂 I should add a notice in the readme
2017:03:31 19:56:03 richardh Got a somewhat general question here — I am trying to do a spike to see what it would be like to convert one of our services from prismatic schema to clojure spec.
There is a clojure.test-compatible fixture in schema which runs all the functions in a test suite through a validator, and throws an error if their arguments and return values don't match their declared schemas.
It strikes me that there's probably no way to do this easily with spec -- that the reason this works in schema is that the functions themselves were created using schema/defn, so they were already set up for this kind of thing. And that the only way to do something similar using spec is if you can find or write all the necessary generators to run the function through its paces in generative tests. (Which in our case seems like a daunting task as -- for better or worse -- there is a lot of hairy Java and Scala interop, futures, server responses, etc. that is all painstakingly mocked out in the tests.)
Does this sound about right? I think this points to what @seancorfield was saying in a recent post, which is that the most leverage in spec comes from making schemas for data, not functions. At least not all the functions.
2017:03:31 20:02:12 seancorfield You can instrument (turn on spec checking) when you run tests but, if you have higher-order functions spec'd, you'll get generative testing on those. And instrument only checks arguments not return values. The idea is that instrument is for checking that functions are called correctly, not that they behave correctly. Generative testing is for behavior and is considered to be a separate type of task to "unit testing".
2017:03:31 20:05:01 seancorfield Alex was talking last night (spec unsession at Clojure/West) about an experimental generative test runner Cognitect are working on which would run continuously and watch for source changes, and analyze which functions (and therefore which specs) need to be exercised as a result of each source change -- rather than running all tests each time.
2017:03:31 20:05:36 richardh That sounds awesome
2017:03:31 20:05:57 seancorfield At World Singles, we've definitely focused on spec'ing data structures (and, for us, particularly using spec for validation and conformance of REST API parameters). Spec is very strong for that.
2017:03:31 20:08:05 seancorfield The generative testing is also amazing but, yeah, if you've got a lot of interop, it's really hard to generate those things. But the normal flow of running (unit) tests doesn't sit well with generative testing: unit tests run very quickly, generative tests often run slowly. We haven't yet settled on a good workflow for running those (we run a few generative tests from unit tests, where they run quickly, but we run them manually -- via check for example -- "from time to time").
2017:03:31 20:11:06 richardh @seancorfield Interesting, thanks for the responses. Yeah, I’ve been playing around with spec and test.check for a while in toy projects and thinking, this is awesome, we should use them on the codebase at work! But our actual production code base is another story. Would probably be easier if we were thinking in terms of spec and generative tests when it was originally being written.
2017:03:31 20:11:41 richardh We’re basically writing Scala in Clojure, is what’s happening.
2017:03:31 20:12:03 seancorfield Yeah, following a TDD / BDD workflow leads to nicely testable code. Adding tests later can be much harder.
2017:03:31 20:12:53 richardh yup
2017:03:31 20:14:22 richardh @seancorfield Do you know what is the rationale behind having instrument not be able to check the return value? That’s basically the core of my original question, I think.
2017:03:31 20:15:15 richardh Also, can you turn on instrument for all functions for which specs exist, or do you have to instrument the functions one by one?
2017:03:31 20:17:27 seancorfield You can (instrument) to turn on checks for all functions that are currently loaded (so it won't instrument namespaces that haven't yet been required!).
2017:03:31 20:17:42 seancorfield Or you can (instrument sym-or-syms) for specific functions.
2017:03:31 20:17:46 richardh that sounds good
2017:03:31 20:20:18 seancorfield The args vs ret thing is philosophical: Clojure/core folks believe that verifying behavior of a function is best done by generative testing based on stated properties. rather than test-by-example. Whereas instrumentation is something you want in place while you're developing / exercising code under test, to check calls are being made correctly.
2017:03:31 20:21:01 seancorfield (I hope that accurately captures what @alexmiller has been communicating about why there's this dichotomy in spec?)
2017:03:31 20:21:44 richardh Yeah, that makes sense. So instrumentation should be thought of as a dev tool, not something that you run in your automated test suite.
2017:03:31 20:22:26 alexmiller Yes. And I'll note that opinions on this vary even in the core team. :)
2017:03:31 20:23:40 richardh @alexmiller ha ha. And by that do you mean that opinions and dev practices vary, or that the dev practices are constrained because of the current state of the tools, and some people are of the opinion that that state should change so that they could use different dev practices?
2017:03:31 20:24:25 alexmiller The latter
2017:03:31 20:24:31 richardh interesting
2017:03:31 20:24:47 alexmiller I think it would be useful to have a way to instrument with ret checking
2017:03:31 20:25:10 richardh You know this Slack record is being recorded. 🙂
2017:03:31 20:25:36 alexmiller No worries :)
2017:03:31 20:26:14 seancorfield When I started spec'ing clojure.java.jdbc, I set it up so when the test suite runs, all functions are instrumented. We don't do that consistently at World Single (but we don't have a huge number of functions spec'd -- see above about our focus on data structure specs).
2017:03:31 20:26:56 alexmiller I agree with the philosophy above but I do not think that stest/check is the only possible kind of test where ret checking would be useful
2017:03:31 20:28:14 seancorfield (I was vocal and upset when :ret-checking stopped being done via instrument -- but I accepted Rich's explanation of why that was changed)
2017:03:31 20:28:49 richardh It used to be there and it was removed?
2017:03:31 20:29:18 seancorfield Yup, in an early alpha.
2017:03:31 20:30:21 richardh Do you have a link to any posts detailing the explanation you’re referring to?
2017:03:31 20:30:43 richardh If not, no worries
2017:03:31 20:32:38 richardh I think one of my main issues is probably something that it’s not the Clojure team’s job to fix, which is that I work in a place dominated by Scala, and it’s a lot easier to justify things that bear some resemblance to the way they do things, like being able to easily validate return types in example tests. But that’s not Clojure’s problem!
2017:03:31 20:33:10 richardh @seancorfield and @alexmiller , thank you very much for taking the time to answer my questions.
2017:03:31 20:35:22 seancorfield I think the most promising avenue will be new forms of test runners that are spec-aware and therefore some new workflows.
2017:03:31 20:37:19 richardh @seancorfield Yes, I’ve been thinking the same thing. That there is an amazing base available now, on top of which some tools could be built.
2017:03:31 20:38:24 seancorfield It's something I'll probably explore with Expectations, now that I maintain that...
2017:03:31 20:39:12 seancorfield We have an #expectations channel and input is always welcome on both the test library itself and the supporting tooling ecosystem!
2017:03:31 20:39:18 richardh cool!
2017:04:01 08:57:42 lmergen does anyone know whether there’s some “clojure.spec cookbook” ?
2017:04:01 08:59:35 lmergen (i.e. a collection of spec examples / community reviewed)
2017:04:01 11:22:49 ggaillard Hello !
I have a set of keywords like (def ks #{::email ::password}) and I would like to build a spec from it, like
(s/def ::cred
(s/keys :req-un (into [] ks)))
But s/keys and s/def are macros so (into [] ks) is not evaluated… So I tried
(s/def ::cred
`(s/keys :req-un [
But the inner part seems to not be expanded and passed "as is" to s/def. I have a feeling telling me that I'm doing something wrong here. Is it sane ? Is there a way to expand the body before s/def is called ?
2017:04:01 11:27:05 gfredericks You have to write your own macro
2017:04:01 11:28:49 ggaillard A one that wrap s/def and s/keys ?
2017:04:01 11:49:26 ggaillard Ah ! I get it ! Sorry for that 😕 Thank you !
2017:04:01 12:51:46 mike_ananev Hi! Sorry for dumb question, but how i can run my spec tests as unit tests using clojure.test? My goal is to run tests in some CI tool using lein test.
2017:04:01 17:41:34 gfredericks @mike1452 write a regular test that calls spec and asserts about the response
2017:04:01 21:15:58 dviramontes hello all, having a little trouble getting up and running with clojure.spec in clojurescript
have these dependencies in my project.clj
2017:04:01 21:18:42 dviramontes let me know if you can spot the error, much appriciate
2017:04:01 21:20:24 dviramontes tldr; the generator doesnt work which works in clj and does not work in cljs, not sure why or what im missing
2017:04:02 11:14:22 mike_ananev Hi. if i have code (gen/sample (s/gen ::basic/string) 1000))), how i can reproduce the data generation? Where to put seed value?
2017:04:02 12:07:26 gfredericks @mike1452 what are you using this for?
2017:04:02 12:45:37 mike_ananev @gfredericks well, i want to have unit tests for my functions. I have bunch of specs for data, and i have specs for my fn's. I want to pass spec'ed data to spec'ed fns to write unit tests.
2017:04:02 12:46:02 mike_ananev i found some solution
2017:04:02 12:46:20 mike_ananev for spec'ed data i can
2017:04:02 12:47:14 mike_ananev i know that (s/valid) here is unnecessary
2017:04:02 12:48:35 mike_ananev and for spec'ed fn example.core/average i can
2017:04:02 12:51:02 mike_ananev i wanted to have reproducible results in unit tests
2017:04:02 20:32:34 didibus Question: Why is cat always used for args in an fdef?
2017:04:02 20:32:56 didibus I've used tuple sometimes, and that works well, would it be incorrect to do so?
2017:04:03 09:12:51 didibus Other question: Is there an any? that means anything except nil?
2017:04:03 09:29:30 bronsa some?
2017:04:03 10:37:46 lsenjov (complement nil?)
2017:04:03 15:46:09 alexmiller @didibus regex specs are generally a good match for all of the various argument calling needs, but really any spec that can match an arg collection is fine
2017:04:03 17:00:09 didibus @bronsa Ah, should have thought of that one, thx.
2017:04:03 17:02:34 didibus @alexmiller I have to say you're an awesome community manager. I'm always impressed how active you are in all the forums and chats and conferences. I don't know how you find time to know everything about Clojure on top of that :+1:clj💯
2017:04:04 11:14:23 tjtolton I said the same thing to him once. His response was "It's literally my job."
2017:04:04 16:02:27 tjtolton Anyone know of a clean way to get the list of defined arguments for a function spec?
2017:04:04 16:05:11 tjtolton basically I want to pull arguments out of a request body and pass them into a function based on that function's spec
2017:04:04 16:05:23 tjtolton I'd bet a dollar somebody's done this before me
2017:04:04 16:33:32 tjtolton Wow, this isn't as straightforward as I thought.
2017:04:04 17:28:41 minimal @tjtolton does this help? (s/form (:args (s/get-spec `my-fun)))
2017:04:05 09:28:28 qqq here I could also do (s/map-of any? (s/map-of any? any?)) but the inner (s/map-of any? any?) is equiv to map? right?
2017:04:05 09:28:30 qqq this is neat
2017:04:05 09:38:37 thheller @qqq (s/def ::map (s/map-of any? ::map) to make it recursive
2017:04:05 09:41:52 qqq @thheller: but that'd be wrong 🙂
2017:04:05 09:45:06 shafeeq Is there a way to send clojure spec over the wire?
I’m looking to dynamically fetch specs from a clojure service via a HTTP request, into my clojurescript app that needs to use these specs for validation, etc.
2017:04:05 09:46:32 shafeeq Unfortunately, I can’t share these specs via cljc, because the server, and the client apps should be decoupled. So it’s a runtime dependency.
2017:04:05 09:46:32 thheller @shafeeq you can put your specs into a .cljc file so they can be shared, but serializing a spec will not work
2017:04:05 09:46:39 thheller doh 😉
2017:04:05 09:47:41 qqq @thheller: I (perhaps incorrectly) thought specs are just data and can be passed around like data.
2017:04:05 09:50:43 ikitommi the s/form is the serialization format for specs. You need to recursively fetch all the related specs too.
2017:04:05 09:50:56 thheller the specs yes but the predicates they use are not
2017:04:05 09:51:26 thheller (s/def ::foo #(= % :foo)) cannot be serialized and transferred over the wire
2017:04:05 09:51:41 ikitommi spiked that with a custom rmi-style “spec-loader” that is used to negotiate the specs over the wire.
2017:04:05 09:59:29 qqq @thheller: ah; besides functions, are there other things that need to be serailized? (and it seems like functions can be serialized if you have a giant hashmap of keyword to functions)
2017:04:05 09:59:40 qqq [I'm not saying this is a good idea; I'm now just curious whether it can be done.]
2017:04:05 10:12:59 thheller that is reading not eval
2017:04:05 10:13:27 thheller you have a symbol clojure.core/string? but now what
2017:04:05 10:13:29 ikitommi does the upcoming spec-of-specs help here?
2017:04:05 10:13:43 thheller no
2017:04:05 10:14:27 thheller as @qqq suggested you could make this work if turn everything into keywords and then define those keywords on both ends
2017:04:05 10:14:43 thheller but really ... use .cljc (just turn the specs into a library the client/server can share)
2017:04:05 10:15:17 ikitommi oh, true @thheller i had test eval’ing the forms, but only in the clj-side.
2017:04:05 10:17:51 shafeeq @thheller from a higher level perspective: the specs on the server side will evolve with time, and we shouldn’t really have to compile, and build the client. The client should ideally be oblivious to changes in the specs. It should pull it down (via http), and use it.
2017:04:05 10:18:05 shafeeq that’s the idea, at least.
2017:04:05 10:18:34 thheller then validate on the server and send the explain-data back, that is just data
2017:04:05 10:20:37 shafeeq well, the specs are my API specification at the moment. Apart from getting a catalog of requests, I’m building and coercing requests on the clojurescript side based on the spec.
2017:04:05 10:21:00 shafeeq pretty much swagger.json, but in spec
2017:04:05 10:21:40 shafeeq I already have this functionality implemented for swagger.json. I’m moving to spec now.
2017:04:05 10:32:21 qqq @thheller: I'm impressed by how flexible spec is. I can't imagine even asking this question in Haskell. 🙂
2017:04:05 12:32:18 odinodin Is it possible to get full spec instrumentation during ClojureScript development in your running app? (using Figwheel). I’ve run cljs.spec.test/instrument which returns a list of fdef’ed functions, but none of the fdef functions trigger when the contract is broken.
2017:04:05 12:37:26 thomas @rovanion can't you use the seq regex's for that?
2017:04:05 12:37:43 thomas or is that not what you need?
2017:04:05 12:38:07 shafeeq I’m rewriting my prismatic-schema spec into clojure.spec. There’s a lot of it. Has anyone here heard of or used some utility to help with this?
2017:04:05 12:38:47 rovanion thomas: I'm able to match them using the seq regexes, but not generate data since they generate lists and I can't seem to be able to message them that I want vectors instead.
2017:04:05 12:39:51 thomas @rovanion just looked at our SO post... and if you do something like gen/vector ?
2017:04:05 12:43:25 rovanion thomas: I'd have to look up how that works first. The docs are very empty when it comes to explaining its use.
2017:04:05 14:34:14 alexmiller it’s not uncommon to run into this when spec’ing a macro (the need exists in a few places in the clojure.core specs) and as you can see in the example, it’s quite tedious to properly solve (even that example is missing what you really want for describe/form). We’ve been discussing something like s/vcat for this but Rich has several competing ideas for how to implement it and I’m not sure where he will end up going with it.
2017:04:05 17:47:21 ghadi Are they design alternatives or impl alternatives @alexmiller ?
2017:04:05 18:06:34 slipset Following http://dev.clojure.org/jira/browse/CLJ-2141, it seems to me that Clojure is missing a predicate namespaced? which returns true if a thing is, well, namespaced.
2017:04:05 19:00:33 ikitommi what kind of data does a (s/coll-of string? :into {}) expects?
2017:04:05 19:31:47 alexmiller @ikitommi if using :into {}, you should be using a spec for a map entry (like a 2 element tuple)
2017:04:05 19:33:18 alexmiller Like (s/coll-of (s/tuple string? string?) :into {})
2017:04:05 19:34:03 alexmiller @slipset I don't think that's generally useful enough
2017:04:05 19:34:27 alexmiller @ghadi design
2017:04:05 19:40:24 ikitommi thanks alex
2017:04:05 20:11:53 alexmiller That's effectively how map-of is implemented
2017:04:05 23:20:46 takeoutweight Is there any (idiomatic) way to attach custom s/explain data for user specs? Use case is a predicate that supplies its own context information that I'd like to have preserved in the failure message.
2017:04:05 23:38:07 alexmiller no
2017:04:05 23:38:55 ghadi @alexmiller can you elaborate around design alternatives for vcat?
2017:04:05 23:39:56 alexmiller @ghadi at this point I just don’t remember the details (and Rich did not fully elaborate on them anyways)
2017:04:05 23:40:11 alexmiller we did talk about it for a while. I mostly remember that he didn’t like any of my ideas. :)
2017:04:05 23:40:41 ghadi sounds legit
2017:04:07 10:08:31 triss hey all. so are spec’s searchable in anyway?
2017:04:07 10:09:54 triss let’s say I have (s/def ::a string?)
2017:04:07 10:10:26 triss and (s/def ::b (s/keys :req-un [::a])
2017:04:07 10:11:18 triss and I want to know if ::b contains any values that are strings?
2017:04:07 10:11:38 triss is it serious effort to find this out?
2017:04:07 10:28:51 ggaillard @triss I don't think there is a function for that purpose. But (s/form spec) give you the spec form, so in your case the list (s/def ::b (s/keys :req-un [::a]). You could walk it like a tree and recursively s/form any spec you find until you get a predicate that match you search criteria … I do something similar to extract the set of keys used by nested s/merge specs. Maybe there is a cleaner way to do it though…
2017:04:07 10:37:58 ggaillard Do you know if there is a way to make spec generate a minimalist sample for a particular spec ?
What I mean by minimalist is "the first simplest value validating the predicate". If I have (s/nilable string?) I'd like to get nil, if I have string?, I'd like to get "", if I have (s/coll-of ::foo :kind vector?) I'd like to get [].
I don't know (yet) how spec works internally but I would expect it could be able to find the shortest path to the first and simplest value validating the predicate. It would be really useful to generate kind of placeholder data waiting to be filled but still validating the spec. Especially when building forms with Re-Frame or similar frameworks.
2017:04:07 11:56:33 borkdude Has anyone seen a similar error message like this lately?
Attempting to call unbound fn: #’clojure.spec/macroexpand-check
We get it in our ClojureScript build. We’re using clojure.future.spec alpha15.
2017:04:07 12:17:06 alexmiller @triss there is no easy way to figure that out right now
2017:04:07 12:18:24 alexmiller @ggaillard you can call gen/sample and take the first example. Samples "grow" in complexity as you use a generator so the first few are usually "simple"
2017:04:07 14:18:21 thomas hi, I have a (defspec tester 100.... in my test file... but when I run either lein test or from the REPL it seems to run only once. Any idea what could cause this?
2017:04:07 15:17:07 fossifoo how do you fdef a fn without args?
2017:04:08 23:53:01 twashing 1. Trying to run s/exercise-fn passing in N arguments, instead of a collection. How can I get s/exercise-fn to pass in a collection?
2017:04:08 23:53:10 twashing 2. Also, how are people using the pinned defspec-test. Running (defspec-test test-name [sut/myfn]) always passes, even if there’s bad code.
2017:04:08 23:59:38 twashing Ok, I’ve sorted 1.… But 2. is still a thorn.
2017:04:09 00:01:17 twashing Basically, clojure.spec.test/check returns an empty sequence, even though :ret number? should break the function.
2017:04:09 01:14:57 twashing Ok got it. The invocation needed to be (clojure.spec.test/check <backtick>myfn) , which I was messing up.
2017:04:09 02:44:11 xiongtx Does it need to be backtick? Regular quote should do
2017:04:09 03:40:40 twashing Hey Tx
2017:04:09 03:41:47 twashing For whatever reason, the check macro needs
A) (st/check `ranged-rand)
B) (st/check ‘ranged-rand) will yield an empty result.
2017:04:09 03:42:20 twashing Presumably it has to do with how the symbol is expanded?
2017:04:09 04:01:16 seancorfield It needs to be a fully-qualified symbol name - which back tick does - or you could use my.ns/my-fn
2017:04:09 07:03:25 fossifoo when i stest/check my database access functions, the real db implementation gets called. i don't want that. i currently try to with-redefs-fn those. is that the way to go?
2017:04:09 07:09:06 fossifoo seems to work, i was just too dumb to get the signiture right. time for more specs i guess 😉
2017:04:09 09:40:50 fossifoo hmm. actually i'm still stuck. does sb have working examples of instrument with :stub & :gen or :spec?
2017:04:09 17:03:10 fossifoo thanks, but only stubbing is not enough in my case
2017:04:09 17:03:34 fossifoo i either need to use :gen or :spec too because i need to scope the fn spec differently
2017:04:09 17:04:09 fossifoo but in the mean time i also found an actual bug in the spec i wanted to check
2017:04:09 17:04:16 fossifoo so it might've been that all along
2017:04:09 17:13:01 fossifoo na, no luck. i always get Couldn't satisfy such-that predicate after 100 tries. {}
2017:04:09 17:13:37 fossifoo but the docs make it really hard to know what should be put into :gen or :spec
2017:04:09 17:13:53 fossifoo and the code is not very straight forward either
2017:04:09 17:34:41 alexmiller :gen or :spec where?
2017:04:10 07:53:16 srihari Is it possible to dynamically (at runtime) define a spec in clojurescript?
2017:04:10 11:30:07 rovanion The macro clojure.spec/keys and clojure.spec/keys* expects a seq of keys as the value of its named arguments :req and :opt. I want to use the same list of keys for two different specs, one using keys and one using keys*. But the macros get passed the quoted symbol for the variable and not the value of it. Is it possible to unpack the value pointed to by a symbol at read time like the common lisp #. syntax?
2017:04:10 14:51:29 frank I think that depending on how the macro was implemented, you may be able to
2017:04:10 15:48:07 donaldball
(eval `(s/def ::record (s/keys :req ~ks)))
is what I’ve been using when generating specs from data
2017:04:10 20:51:03 tjtolton I'm sorry, I'm having trouble finding this in the docs (which leads me to believe core.spec has some kind of strong opinion on this) -- is there a canonical way of saying "a predicate that matches anything". essentially a predicate (fn [thing] true)
2017:04:10 20:51:23 tjtolton looking for, essentially, (s/.)
2017:04:10 20:51:54 donaldball any?
2017:04:10 20:52:27 tjtolton is that a thing?
2017:04:10 20:52:40 tjtolton are you sure you're not thinking of prismatic?
2017:04:10 20:52:56 schmee it is new in clojure 1.9
2017:04:11 02:51:24 olivergeorge I do look forward to instrumentation on clojure.core
2017:04:11 02:52:03 olivergeorge Perhaps I'm wishing but imagine it would have saved me this headache. (cljs.reader/read-string (str (keyword "3asdf")))
2017:04:11 02:52:28 olivergeorge That throws a mysterious error since we're trying to read a keyword which starts with a number.
2017:04:11 02:53:16 olivergeorge (my use case was using js->clj keywordize-keys to turn json into clojure data structures and later serialising to js/localStorage)
2017:04:11 02:53:43 olivergeorge A spec on the keyword function would have picked up the illegal input (I believe)
2017:04:11 03:23:32 hiredman Nah
2017:04:11 03:24:56 hiredman The core team has repeatedly stated that those keywords are not illegal, the keyword function is just able to create a wider range of keywords than the reader
2017:04:11 04:09:08 olivergeorge Is that right. Huh. It was a bugger to find my unexpected keyword. I guess I'd be happy for the error thrown in read-string to improve.
2017:04:11 04:11:58 stathissideris @madstap yes, that would be the basic version, but I was thinking about something that actually binds local vars in the same way that core.match does
2017:04:11 04:17:04 stathissideris With var names derived from the keyword names in s/cat
2017:04:11 04:18:48 cfleming @olivergeorge It’s worse than that - using the function you can create keywords with spaces in them, or anything else your heart desires.
2017:04:11 04:19:06 cfleming Feel like a keyword with a newline in it? No problem.
2017:04:11 04:24:02 olivergeorge Yeah that's a nasty surprise. Seems like a spec instrument on clojure.core/keywords would help to protect me nicely.
2017:04:11 04:24:25 olivergeorge As it is I'll log a ticket for improving read-keyword so that it's blindly obvious when the specific issue I came up with occurs.
2017:04:11 04:26:28 olivergeorge @alexmiller I gather adding spec's to the core ns's is complicated by aot compilation (or similar). Does that mean user defined optional core ns specs will be difficult/troublesome?
2017:04:11 04:26:56 olivergeorge @cfleming have you found any good opportunities for Cursive to integrate/leverage specs?
2017:04:11 04:27:33 olivergeorge (I wondered if it could be used to improve autocomplete suggestions)
2017:04:11 04:28:10 cfleming There are plenty, but I haven’t had time to investigate it yet. It could definitely be used to improve autocomplete, yes, assuming the spec machinery provides enough hooks to get that info out.
2017:04:11 04:29:20 olivergeorge @cfleming good stuff
2017:04:11 07:40:59 ikitommi Hmm.. s/merge seems to report problems of qualified keys twice:
2017:04:11 08:05:49 akiel As I read the doc on s/& this (s/conform (s/& (s/+ char?) #(apply str %)) [\s \p \e \c]) should return "spec" but returns [\s \p \e \c]. Am I wrong?
2017:04:11 08:41:44 akiel (s/conform (s/& (s/+ char?) (s/conformer #(apply str %))) [\s \p \e \c]) returns "spec". I debugged the code and s/& calls dt with cpred?nil. So maybe it should use cpred?true here?
2017:04:11 09:18:43 wottis greetings
2017:04:11 09:21:23 akiel Juste require shepherd.spec without any alias and use :principle/type
2017:04:11 09:25:45 wottis @akiel thank you very much!
2017:04:11 12:13:31 alexmiller @akiel s/& just validates the predicate then returns the conformed value of the spec so that is the expected behavior. You could switch it to s/and though and use s/conformer for the last bit
2017:04:11 12:16:04 alexmiller @ikitommi it's reporting the same bug from each branch of the merge. Not sure if those should be deduped or not
2017:04:11 12:18:34 ikitommi I think it should, as it works with unqualified keys.
2017:04:11 12:24:25 alexmiller @olivergeorge you can add user-defined specs to core. Works fine. We have no plans to place a spec on keyword any narrower than string? though. What would be a useful enhancement though is to have either a predicate or "safe" version of keyword that accepted only print-read-safe keywords
2017:04:11 12:25:38 alexmiller keyword-strict or something
2017:04:11 12:26:11 alexmiller And then you could use that in json parsing etc
2017:04:11 12:39:38 akiel @alexmiller The doc of s/& says:
takes a regex op re, and predicates. Returns a regex-op that consumes input as per re but subjects the resulting value to the conjunction of the predicates, and any conforming they might perform.
especially: but subjects the resulting value and any conforming they might perform
I read that as if the cvals of the preds should be returned. The same as s/and is doing, as you say.
2017:04:11 13:00:42 alexmiller Hmm, not sure. I would need to ask Rich.
2017:04:11 13:14:33 akiel @alexmiller Ok. Thanks.
2017:04:11 15:50:00 ikitommi @alexmiller wrote an jira issue of that double problems, also another of s/keys* is returning something that is not a clojure.spec.Spec
2017:04:11 15:51:49 bronsa @ikitommi no regex op return specs tho
2017:04:11 15:52:19 bronsa same for s/cat, s/+ etc
2017:04:11 16:43:59 ikitommi @bronsa oh, that bad.
2017:04:11 17:07:54 ikitommi well, now that I look at the clojure.spec test suite, no wonder there are (form) bugs. @alexmiller, is there a way to contribute more tests? testing all forms & return values as Specs would easily reveal all things that don’t work right now.
2017:04:11 18:13:23 bronsa @ikitommi i think that's by design (regex ops not being specs)
2017:04:11 19:40:43 alexmiller correct, that’s as intended (those respond to regex?
2017:04:11 19:45:38 alexmiller regarding & forms, that problem space is related to the issues with the keys* form (which is implemented with &). would be fine to log though.
2017:04:11 21:02:44 olivergeorge Thanks @alexmiller I'm happy with those options.
2017:04:11 21:06:20 alexmiller @olivergeorge I’ve talked about these before with people but I don’t think it was ever actually logged
2017:04:11 22:03:08 olivergeorge I wonder where that documentation should live.
2017:04:11 22:04:38 olivergeorge Adding a keyfn option to js->clj would make it easier to step safely around the sharp edges
2017:04:11 23:31:30 waffletower If this is a reasonable place to ask about generators… I am trying to understand clojure.test.check.generators/no-shrink. I have tried to eliminate shrinking in this generator example:
2017:04:12 00:11:15 waffletower Will check that out. Trying to get a uniform distribution. Am tempted to use the private (make-gen) and the random namespace as you do with the generator uuid.
2017:04:12 00:11:36 waffletower Is there a reason that make-gen is private?
2017:04:12 00:11:40 gfredericks Uniform among longs?
2017:04:12 00:11:44 waffletower yes
2017:04:12 00:11:54 gfredericks It's private because it's usually not necessary
2017:04:12 00:12:16 gfredericks But we don't have a uniform long generator at the moment
2017:04:12 00:12:21 gfredericks Though choose is close
2017:04:12 00:12:35 gfredericks I'm considering adding one
2017:04:12 00:12:47 waffletower I know I am young to test.check, and you have been really helpful, but I keep bumping into boundaries.
2017:04:12 00:13:13 gfredericks Are you using it for PBT or some other speccy thing?
2017:04:12 00:13:40 waffletower In this case I am making a generator for dates
2017:04:12 00:13:52 gfredericks Why do you want a uniform long?
2017:04:12 00:14:11 gfredericks (I.e., why does that help for dates?)
2017:04:12 00:14:59 waffletower using clj-time and it is convenient to generate a random date from a long rather than its constiuent parts. Want them to be evenly distributed.
2017:04:12 00:15:36 gfredericks You'll get weird dates with >4 digit years that way, won't you?
2017:04:12 00:16:41 waffletower I could use ints instead, but would be nice if they were uniform and not usually centered around 1970
2017:04:12 00:17:43 gfredericks I guess I'm trying to figure out why uniformity is important for this, especially since the domain will inevitably end up rather arbitrary
2017:04:12 00:18:05 gfredericks E.g., dates in the years -1000000 to 1000000
2017:04:12 00:18:47 gfredericks I think date & time generators end up being weird regardless :/
2017:04:12 00:19:50 waffletower Its a question of control. For some data I feel a desire to turnoff the growth and shrinking mechanisms and control the output with localized RNG
2017:04:12 00:20:35 gfredericks gen/choose is the classic tool for turning off growth
2017:04:12 00:21:10 waffletower great lead, I will check it out.
2017:04:12 00:21:15 gfredericks It just has edge cases outside of the 32 bit range
2017:04:12 00:21:38 gfredericks I was thinking of making a variant called uniform-integer that world be more robust
2017:04:12 00:23:16 waffletower you are supremely helpful. thanks again
2017:04:12 00:24:35 waffletower I see gen/scale as well…
2017:04:12 00:36:01 gfredericks Yep. gen/sized is a more general (ha!) version of that
2017:04:12 00:36:40 gfredericks A lot of generators that do nontrivial growth things do it by combining choose and sized
2017:04:12 00:37:28 gfredericks I guess I mean that those two things give you the power to completely customize growth
2017:04:12 00:49:04 fossifoo does sb have working examples of instrument with :stub & :gen or :spec?
2017:04:12 00:57:07 fossifoo the ":replace" works, but i thought i could just change the result of (db/select) so that it fits the spec when testing the function calling it
2017:04:12 00:58:57 fossifoo i know that my datomic/adi db/select will return the correct types, but i'm not sure how to put that knowledge into spec
2017:04:12 01:27:17 fossifoo i think an example of ":gen a map from spec names to generator overrides" in clojure.spec/instrument would help
2017:04:12 01:27:56 fossifoo i got a version to run that uses ":spec", so that's a little nicer
2017:04:12 01:35:14 fossifoo fdef docs says "Once registered, function specs are included in doc, checked by
instrument, tested by the runner clojure.spec.test/run-tests," is there such a thing as this runner?
2017:04:12 02:42:59 wottis does someone know a larger project (present on github) that uses clojure.spec? I want to see how spec is really being used. All of the small, and simple repl example can't answer the questions i have.
2017:04:12 03:47:44 seancorfield As noted in #clojure, it's kinda hard since most real world Clojure projects have not yet moved to 1.9.0 Alpha builds.
2017:04:12 08:36:45 curlyfry @wottis I like this one (in ClojureScript): https://github.com/jrheard/voke/tree/master/src/voke It's a WIP game that uses spec for its entity system. Not super large, but makes good use of fdefs and has a few generative tests! 🙂 Maybe @jrheard can tell us a bit more about it?
2017:04:12 08:46:13 rovanion wottis: Here is a list of all the projects on clojars I've found using spec: systems-toolbox, spex, oops, alia-spec, clj-edocu-users, mistakes-were-made, utangled-client, clj-edocu-help, spec, net, clj-element-type, gadjett, datomic-spec, tappit, tick, deploy-static, tick, spectrum, sails-forth, ring-spec, doh, tongue, diglett, rp-query-clj, pedestal.vase, clj-jumanpp, om-html, crucible, ctim, crucible, wheel, merlion, turbovote-admin-specs, odin, pc-message, active-status, rp-json-clj, curd, rp-util-clj, swagger-spec, invariant, functional-vaadin, untangled-client, flanders, uniontypes, dspec, schpeck, macros, replique, specific
2017:04:12 08:47:23 wottis @curlyfry @rovanion Thank you very much!
2017:04:12 08:47:29 mpenet spandex also has specs
2017:04:12 08:48:41 rovanion Right, libraries that are so "popular" that they're on http://clojure-toolbox.com are: alia, cljs-oops, closp, core.typed, graphql-clj, java.jdbc, onyx,
sablono, spandex, tupelo.
2017:04:12 08:48:49 rovanion Forgot that I had split them into two.
2017:04:12 08:50:35 wottis hrhr that should be enough for now
2017:04:12 09:26:25 rovanion So perhaps start with some library from the latter list.
2017:04:12 09:47:10 rovanion Is it possible to express something like (spec/keys :req-un [(spec/or ::a ::b)]) so that either :a or :b is present in the resulting map, but never the two at the same time?
2017:04:12 10:00:37 dergutemoritz @rovanion Not with s/keys alone
2017:04:12 10:01:05 dergutemoritz You gotta s/and it with a pred of that effect
2017:04:12 10:02:52 fossifoo this seems kinda noisy
2017:04:12 10:03:25 fossifoo i also wondered about that and i have some cases where a thing is either a or b or c or d (or ... some more)
2017:04:12 10:03:39 fossifoo might be bad interface design, but it's exploding fast
2017:04:12 10:25:49 dergutemoritz @fossifoo "is either a or b or c or d" sounds like s/multi-spec could be worth a look for you
2017:04:12 10:28:35 fossifoo ah, yeah, i glanced over that. might work out in my case
2017:04:12 14:10:09 alexmiller @fossifoo regarding your question above about the docstring for fdef - that has already been fixed - should be clojure.spec.test/check. Where are you seeing the version pointing to run-test?
2017:04:12 14:12:18 alexmiller oh, probably the online clojure doc - there is actually a known problem preventing those docs from being regenerated and they are a few alphas behind. There aren’t many changes to the API since then but that’s one.
2017:04:12 14:33:10 fossifoo okay
2017:04:12 14:33:45 fossifoo do you happen to have an example or clarification for instrument {:gen} ?
2017:04:12 14:35:25 fossifoo "a map from spec names to generator overrides" i read that as {:some/spec (gen/string)} or such , but that didn't seem to work for me
2017:04:12 14:36:55 fossifoo i always got the "could'nt generate in 100 tries" error with that. don't have the actual error on hand
2017:04:13 00:37:49 tbaldridge IMO, with-redefs is always bad form
2017:04:13 00:43:47 micahasmith feel like the more i work through this clojure.spec guide, i’m realizing this is more about typing + generative testing than testing
2017:04:13 00:44:17 micahasmith which is strange given then (s/fdef :fn #()) functionality
2017:04:13 00:53:56 micahasmith or does my mocking have to turn into custom generators
2017:04:13 09:40:31 rovanion Has any of you ever tried to s/merge two s/keys*? I can only seem to get it to work for validation and not for generation.
2017:04:13 14:23:16 jfntn Getting some odd validation errors on the :ret with orchestra.spec.test so I’m not sure if the spec is wrong?
2017:04:13 14:33:57 alexmiller comparator fns don’t have to return only -1 / 0 / 1, although a particular one might
2017:04:13 14:35:34 alexmiller they can return any int - the sign is the important thing
2017:04:13 14:36:16 alexmiller comparators also typically have constraints around comparing two of the same kind of things
2017:04:13 14:36:55 jfntn oh I didn’t know about the sign
2017:04:13 14:37:55 jfntn I think the issue is I forget fspec is generating data from the args spec and calling the comparator with it, but I have some tricky invariants between a and b that are not encoded inside the args spec
2017:04:13 14:38:23 alexmiller yep
2017:04:13 14:38:42 jfntn that makes sense, thank alexmiller
2017:04:13 14:49:29 jfntn Trying to resist the urge to shave that yak but I’m curious what would be the right approach here: a and b only have a partial order, the state that’s being closed over by the comparator is a lookup table that’s used to get a total order. So if I wanted the whole fdef to work, I’d have to constrain the fspec:args generator with the value generated for the fdef:args constructor 🤔
2017:04:13 14:51:08 alexmiller yeah, that’s not likely going to be possible to do easily
2017:04:13 14:51:47 alexmiller but what you can do is to a) build a better args spec that uses values from your lookup and b) write a fn spec that verifies something about the comparator
2017:04:13 14:52:05 alexmiller for a, I mean the generator
2017:04:13 14:54:34 alexmiller and then you also need to ask yourself whether the tests you get out of it are worth the effort you’re putting into the spec. it’s ok if the answer is no. :)
2017:04:13 14:55:01 alexmiller higher order stuff is particularly hard to gen test
2017:04:13 14:59:43 jfntn I think the answer is no in this case, but I think I understand how this would work: the fspec for the comparator would be checked in isolation so its args need to have the invariants from the lookup table but we don’t need to somehow ensure it’s lexically the same table. So the gen for the lookup table can build the relationships, and the gen for comparable would bind to it and pluck a couple of values out of the table. Is that somehow correct?
2017:04:13 16:18:51 alexmiller yeah, you’d have to close over some of that stuff I guess
2017:04:14 14:38:39 fossifoo @gfredericks thanks for the example, instrument {:gen (f [] somegen)} works indeed
2017:04:14 17:46:58 alexmiller was just playing with it this morning actually - there’s a function that makes a spec from data
2017:04:14 17:47:13 mpenet So "are" -> "will be"
2017:04:14 17:47:48 alexmiller well I’d say specs “are” data now, just that you need the specs for that data too
2017:04:14 17:48:55 alexmiller the ::ss/or-form in the example above is the only missing piece
2017:04:14 23:30:30 stand Is there some way of specing a situation where a map has mutually exclusive optional keys? That is neither ::a or ::b is required but if ::a appears, ::b may not and vice versa.
2017:04:14 23:43:14 alexmiller You can and a predicate
2017:04:14 23:43:40 alexmiller Or possibly multi-spec is a better match
2017:04:14 23:58:09 stand Can you explain what you mean by "and a predicate?"
2017:04:15 02:00:53 seancorfield A general criteria map must have :weight but can only have one of :value or :values.
2017:04:15 02:02:54 stand I don't believe that would work in my case. I need a situation where neither :value or :values need appear but if one appears the other cannot. I don't think you can do an or in a keys :opt, can you?
2017:04:15 02:06:39 seancorfield Ah, no, you cannot.
2017:04:15 02:07:13 seancorfield But they would both just be :opt keys and you could still s/and the check that they are not both present
2017:04:17 16:47:13 ikitommi @mobileink there are at least two libs on top of spec which allow meta-data on specs (and produce JSON Schemas): https://github.com/metosin/spec-tools & https://github.com/uswitch/speculate. I’m working on the first one, in which the meta-data is written to spec forms, so the are really persisted. Last PR before releasing, this week hopefully.
2017:04:17 16:48:14 mobileink thank you. i did take a brief look at spec-tools and saw some spec->json schema stuff; does it go the other way ’round?
2017:04:17 16:49:21 mobileink anyway, i take it that means no way to do it in plain spec?
2017:04:17 16:51:28 ikitommi no, just one-way currently. @mpenet had plans of doing the other way around, would open up a whole new level of interop with js.
2017:04:17 16:52:03 mobileink so in spec-tools, for readOnly I would do :json-schema/readOnly true?
2017:04:17 16:54:26 ikitommi yes
2017:04:17 16:55:46 ikitommi In plain spec, I guess you could add meta-data to registered specs: read the registry value, add meta, re-register.
2017:04:17 16:57:39 mobileink yeah, was thinking about that but I think I’ll take the path of least resistance and give spec-tools a try. thanks.
2017:04:17 17:24:44 waffletower I am trying to make generative testing of a recursive spec more feasible. Consider the following simple example:
2017:04:17 17:26:43 waffletower Is there an external way to limit the size of the collection produced by s/coll-of, or is a custom generator necessary?
2017:04:17 17:55:34 nfisher @waffletower :min-count :max-count allows you to control the collection size.
2017:04:17 17:58:33 nfisher Is that what you’re looking for?
2017:04:17 18:06:46 waffletower Thanks!
That limits the size supported in the spec though, correct? I am trying to leave the maximum unbounded but place limits to facilitate generation.
2017:04:17 18:08:26 waffletower I have tried this route:
2017:04:17 18:59:51 alexmiller no, but you can pass :conform-keys option for that if you need it
2017:04:18 05:09:40 ikitommi thanks!
2017:04:18 05:15:22 ikitommi still, I find the below bit unintuitive: without :conform-keys the first one informs of error but the second one does not:
2017:04:18 08:43:18 didibus @ikitommi Why would a string conform to a keyword?
2017:04:18 09:52:41 ikitommi because of the conformer. If I set the :conform-keys it conforms ok as Alex pointed out. But without it, the conform returns invalid data, which feels odd given the s/conform docs: “Given a spec and a value, returns :clojure.spec/invalid if value does not match spec, else the (possibly destructured) value.“.
2017:04:18 12:29:22 alexmiller Seems like you're ignoring the map-of docstring
2017:04:18 12:29:49 alexmiller There are several good reasons not to conform the keys by default
2017:04:18 12:36:30 pseud what's the point of clojure spec's fdef ? If I check against my specs using pre & post-conditions, I will get failures at the point of invocation, but having defined a function spec doesn't really prevent the function returning a non-compliant value (and I assume the same is true for providing wrong input)
2017:04:18 12:37:07 pseud (aside from stubbing dependent functions for tests)
2017:04:18 12:37:26 alexmiller For use with stest/instrument and stest/check
2017:04:18 12:38:30 pseud That's it ? The spec guide hints at fdef providing something for development, I can't temporarily get more feedback during development ?
2017:04:18 13:01:08 urbank @pseud Isn't instrument meant for development - you can instrument a function so that it check its input, output when called?
2017:04:18 13:03:01 curlyfry instrument only checks input for fdef if I remember correctly
2017:04:18 13:03:23 curlyfry But yeah, it's meant to give more feedback during development
2017:04:18 17:59:07 seancorfield No. s/and flows the conformed value through subsequent forms.
2017:04:18 17:59:54 seancorfield (s/conform (s/keys :opt [::foo]) {::foo 1 ::bar 9}) ;=> #:boot.user{:foo 1, :bar [:i 9]} — s/keys conforms all key/values if there are specs for them, not just the keys listed in :req and :opt.
2017:04:18 18:00:16 seancorfield ^ @mobileink
2017:04:18 18:02:20 seancorfield You want s/merge here to combine key-specs:
2017:04:18 18:06:44 mobileink d’oh! thanks, he mumbled sheepishly.
2017:04:19 08:25:45 mike_ananev Hi! I would like to add some :doc to my spec? Is there any way to do that? I need some text comments for my spec
2017:04:19 08:26:06 mike_ananev for data, not for fn's
2017:04:19 11:35:43 alexmiller Not currently but maybe eventually
2017:04:19 14:11:40 robert-stuttaford is there anything out there that talks about how to write custom generators for s/multi-spec?
2017:04:19 14:12:57 robert-stuttaford if we write a generator that returns something sufficient for the dispatch fn to use, will the generator code know to keep generating data for the specs returned by the defmethods?
2017:04:19 14:13:38 robert-stuttaford in our case, we dispatch in two different ways based on the value of :type, so we’re thinking we’d gen a map with :type with a valid value from one of the two sets of :type values we keep
2017:04:19 15:19:05 alexmiller Well that's where retag comes in. However I was looking at something the other day that made me question whether multi-spec was working properly in gen for non-keyword retag cases
2017:04:19 15:20:24 alexmiller I guess I would turn your question around and ask whether it's doing what you expect
2017:04:19 16:48:04 mobileink hi. I have a map with a type key (:oic.core/rt, for resource type), whose value is a vector of type keys. how can I tell spec to form the conjunction of those types so that valid maps satisfy them all? Something like (apply s/and [::foo ::bar]), which doesn’t work.
2017:04:19 16:52:21 ghadi No need. spec will automatically check all namespaced keys for which there is a spec registered
2017:04:19 16:53:31 mobileink right, but different types will have different specs; for example, :oic.r/temperature requires ::p/temperature
2017:04:19 16:54:03 mobileink so i need map-level validation. a map might look like
2017:04:19 16:54:53 mobileink this should fail, because :oic.r/humidity requires ::p/humidity
2017:04:19 16:55:30 mobileink it’s easy to use s/get-spec, but how can I then apply the spec?
2017:04:19 16:59:57 ghadi Look at s/merge, which merges map specs
2017:04:19 17:01:31 mobileink yep, know about that, but i think that’s static - here we won’t know what to merge until runtime. at least, i’ve tried to use it but no luck so far.
2017:04:19 17:20:34 mobileink i think i need something like multi-spec with the ability to dispatch multiple times
2017:04:19 17:30:55 ghadi Are you changing the meaning of namespaced keys in different maps?
2017:04:19 17:31:07 ghadi Or are you having different sets of namespaced keys
2017:04:19 17:45:13 mobileink the idea is one schema for each oic.r “resource” at https://github.com/OpenInterConnect/IoTDataModels, but a resource “instance” can have multiple types - listed in :oic.core/rt
2017:04:19 17:48:09 mobileink I think i have a possible solution: I use multi-spec, but my multi-methods recur over the vector of keywords. so given :oic.core/rt [::foo ::bar], the dispatch method will dispatch on ::foo, and the defmethod for ::foo will recur with (rest (:oic.core/rt arg)) before calling s/keys. seems to work with a simple example at least. kinda cool imho.
2017:04:19 17:48:38 mobileink amazing what you can do with clojure
2017:04:20 22:25:43 donaldball I am just now using multi-spec for the first time and am finding it a little weird that it doesn't conform into a variant tuple like or does
2017:04:22 21:52:59 pseud I know there's supposed to be some way by which I can refer to a spec, bar residing in [my.example.foo :as foo] by its alias, but (s/conform :foo/bar ...) doesn't seem to be it. Anyone knows/remembers ?
2017:04:22 21:57:47 dergutemoritz @pseud You gotta use two colons like that: ::foo/bar
2017:04:23 10:40:19 urbank is https://github.com/jeaye/orchestra usable with clojurescript? It requires clojure.spec, not cljs.spec, and uses StackTraceElement->vec, which I can't find in the clojurescript api
2017:04:23 10:54:56 lsenjov Looks to be straight .clj, so probably not
2017:04:23 10:55:56 lsenjov Although it doesn't seem to be using any java interop, so a bit of dependency management and it should work
2017:04:23 11:19:56 urbank It does use (.getStackTrace (Thread/currentThread) ... though I suppose that's not really a necessary function, and it would just be different in clojurescript
2017:04:23 11:25:31 lsenjov Oh, I missed that one
2017:04:23 11:26:27 lsenjov Hmmm, it also uses .applyTo just below it, but that shouldn't be a problem
2017:04:24 12:57:35 hkjels How do I ensure that a spec never generates the same value twice?
2017:04:24 13:17:06 tbaldridge @hkjels you mean like values in a collection are unique? And this is for the generator for a spec?
2017:04:24 13:17:52 hkjels Yes
2017:04:24 13:20:30 hkjels I can probably make a custom generator that checks for each item, but I was hoping that something exists already
2017:04:24 13:28:23 hkjels Actually, I think I have a good solution here
2017:04:24 13:36:38 tbaldridge @hkjels I think that exists
2017:04:24 13:37:54 tbaldridge s/coll-of takes an optional parameter of :distinct
2017:04:24 14:25:11 hkjels @tbaldridge Thank you!
2017:04:24 18:19:03 mobileink any general-purpose spec libs out there? my googling didn't find any. I'm specking routine stuff like href, uri/url, name, uuid, etc. also more complicated but widely used stuff like phone number, postal address, etc. easy enuff, but a well-maintained lib would be better.
2017:04:24 18:36:26 mobileink more generally: lots of schema languages and schemas out there, that can be expressed in spec. i'm working on the OCF/OIC schema; another possibility is FOAF (https://en.m.wikipedia.org/wiki/FOAF_(ontology)). it would be nice to settle on conventions, e.g. spec.foo.bar for a spec implementation of schema foo.bar.
I get :clojure.spec/invalid, explanation: val ["a"] fails...
Is it a bug?
2017:04:25 00:35:26 souenzzo IMHO it should be valid, once spec just looks for what is defined, and on ::m spec's, there is no ::a
2017:04:25 00:50:42 ghadi > When conformance is checked on a map, it does two things - checking that the required attributes are included, and checking that every registered key has a conforming value. We’ll see later where optional attributes can be useful. Also note that ALL attributes are checked via keys, not just those listed in the :req and :opt keys. Thus a bare (s/keys) is valid and will check all attributes of a map without checking which keys are required or optional.
https://clojure.org/guides/spec
2017:04:25 00:51:02 ghadi @souenzzo this is actually by design
2017:04:25 00:53:11 ghadi Specs registered under namespaced keys give the keys global meaning thus are checked whenever they are encountered
2017:04:25 00:54:09 ghadi This is different than other systems. (I like it)
2017:04:25 09:00:36 mbjarland have a clojure.spec question, assume I have a number of specs defined for seq:s of chars, how would I define a string spec which uses them?
i.e given:
how would I write a string spec using the char seq spec:
(s/def ::my-string
(s/and string?
(??? <something using ::myc-char-sequence and perhaps (seq %) > ???)))
2017:04:25 10:09:02 dergutemoritz @mbjarland Inserting (s/conformer seq) after the string? predicate in the ::my-string spec should do the trick!
2017:04:25 10:26:24 mbjarland @dergutemoritz !! ahh...I knew there had to be away to make a transformation before handing it off to spec...that is great!! thanks
2017:04:25 10:29:38 dergutemoritz @mbjarland You're welcome! Note that when using that with s/conform it'll also return a seq of chars. Maybe you want to add something like (s/conform (partial apply str)) at the end of the chain.
2017:04:25 10:30:11 mbjarland seems conformer has a second unform argument function
2017:04:25 10:30:23 mbjarland would that fix it
2017:04:25 10:31:34 dergutemoritz Yeah but that's for use with s/unform for recovering the original value - slightly different approach with different pros and cons but works, too 🙂
2017:04:25 10:32:36 dergutemoritz Another option would be to use string regexen instead of spec regexen
2017:04:25 10:33:41 mbjarland yeah, I was hoping not to have to do string regex
2017:04:25 10:33:52 mbjarland I like the composability of specs
2017:04:25 10:35:31 mbjarland errr...I'm assuming "string regexen" refers to actual regular expression #" " matching?
2017:04:25 10:45:47 Adam Munoz Hi everyone. How do you spec :args of any type? i.e. takes x that can be anything and returns for example boolean.
2017:04:25 10:55:31 Adam Munoz Ok, any?🙂
2017:04:25 10:55:37 Adam Munoz thanks anyway
2017:04:25 10:58:16 mbjarland @adammunoz, I'm a beginner with spec but perhaps something like:
2017:04:25 10:58:56 Adam Munoz @mbjarland Thanks. No I meant that predicate that I want is maybe any? .
2017:04:25 10:59:08 Adam Munoz (any? 1) => true
2017:04:25 10:59:16 mbjarland : ) yes looks like it
2017:04:25 10:59:19 Adam Munoz (any? "whatever") =>true
2017:04:25 10:59:37 mbjarland I figured any would be collection based, but you are right
2017:04:25 10:59:44 Adam Munoz 😉
2017:04:25 12:10:24 dergutemoritz @mbjarland Yeah that's what I meant by string regexen 🙂
2017:04:25 12:11:45 dergutemoritz Note that there are also various libraries which provide a structured representation of regexen similar to that of spec which can be composed just as well and compiled down to java.util.regex.Pattern
2017:04:25 12:20:47 mbjarland @dergutemoritz have any clj ones to share?
2017:04:25 12:33:26 dergutemoritz @mbjarland I seem to remember having seen multiple ones actually, but I fail to find them again!
2017:04:25 12:33:43 mbjarland @dergutemoritz : ) no worries (edit: was on touch screen...they suck)
2017:04:25 12:33:47 dergutemoritz I have used Scheme Regular Expressions with great pleasure in the past, though. Perhaps I'm mixing it up.
2017:04:25 12:34:19 mbjarland and as a side question, how do you organize specs for functions? Right after the function itself? in another namespace? somewhere else?
2017:04:25 12:35:11 mbjarland keeing them close feels good in a sense but also pollutes the code with a lot of "spec noise"...for good or bad
2017:04:25 12:36:04 dergutemoritz Personally, I like to put them before the function definition (if you mean fdefs) but I don't have enough confidence in it to recommend it as a good idea or anything 😉
2017:04:25 12:36:22 mbjarland ok
2017:04:25 12:36:29 dergutemoritz Also, I'm not specing each and every function just for the sake of it
2017:04:25 12:40:10 mbjarland I got another one (starting to feel like a spambot here), do you see any immediate reason why this:
2017:04:25 12:41:08 mbjarland improvements to the specs also welcome, I can for example see that I could remove the (s/and char? in the first one
2017:04:25 12:42:00 mbjarland all of ::align-specifier, ::bracket-expr and ::between-bracket-expr work in isolation
2017:04:25 12:42:21 mbjarland as in I can run explain and valid? on them and they behave as I would expect
2017:04:25 12:42:34 mbjarland ah...complement...maybe I need to be explicit there instead...
2017:04:25 12:43:26 mbjarland I did feel a bit woozy writing (complement #{\[ \}}) : )
2017:04:25 12:51:38 dergutemoritz @mbjarland Looks like you're running into an infinite left recursion because ::between-bracket-expr may not consume any input
2017:04:25 12:58:53 mbjarland @dergutemoritz ok, I will have to marinate on that a bit. Thanks for the pointer
2017:04:25 13:02:27 dergutemoritz Or perhaps not 😄
2017:04:25 13:03:47 mbjarland food for marination
2017:04:25 13:05:58 dergutemoritz Bon appétit!
2017:04:25 13:06:23 mbjarland thanks!
2017:04:25 13:17:13 mbjarland and yes, changing (s/* to (/s+ in :between-bracket-expr fixes the problem. It probably should have been + to begin with
2017:04:25 13:55:39 mbjarland and so I'm lost in the woods again...and not for the first time with spec : )
If I write standalone specs I can run explain/valid?/exercise etc on them. What if I spec a function, do I have to actually instrument the function to test my spec or is there a way to get hold of the function spec and run the normal spec-exercising-functions on it?
2017:04:25 13:56:48 mbjarland I guess I can do a s/def on say the args spec and then exercise that separately...
2017:04:25 14:19:19 alexmiller yes, you can call s/get-spec on the fully-qualified symbol to get the function spec, and the function spec supports keyword lookups for it’s parts (`:args`, etc).
2017:04:25 14:21:24 mbjarland @alexmiller ok, nice and data centric as usual. thank you
2017:04:25 14:22:37 alexmiller or you can go the other way and define the args spec as a standalone spec and then assemble the fdef spec from it (I’ve done this in some cases)
2017:04:25 14:23:22 mbjarland that's what I ended up with, but still good to know it's not a black hole and you have access to the function spec
2017:04:25 14:26:53 mbjarland I find my biggest hurdle with spec so far is not understanding the language but understanding its place in the process and best practice use. Is there for example still a place for using specs as preconditions in functions? I guess it would be nice to see an example of a larger production system using spec (or a similarily reality checked code base) and what kind of usage patterns they ended up with. Any references to presentations or repos with such code base much appreciated....
2017:04:25 16:33:59 sparkofreason Is there any way to use coll-of or every (or some other existing capability in spec short of writing custom predicates and generators) to spec non-Clojure collections, like java.util.HashSet? I feel pretty certain the answer would be "no", but just want to double-check before reinventing any wheels.
2017:04:25 17:02:08 dergutemoritz @dave.dixon Since most (all?) Java collections are seqable, you should be able to make it work by anding in a seq conformer
2017:04:25 17:10:53 sparkofreason @dergutemoritz thanks. Looks like even every works if you specify :kind to be a custom predicate, but still doesn't generate. And just verified every-kv doesn't seem to work with :kind, and keys doesn't give any hooks for custom collections, as far as I can tell. Which seems sensible - can't support everything possible in the JVM with a single API.
2017:04:25 17:12:28 dergutemoritz @dave.dixon Oh, right, generation isn't covered by my suggestion either. Note what the doc string says about a custom :kind predicate, though:
> :kind - a pred/spec that the collection type must satisfy, e.g. vector? (default nil) Note that if :kind is specified and :into is not, this pred must generate in order for every to generate.
2017:04:25 17:14:36 dergutemoritz Hm but I doubt that passing something like :into (HashSet.) will just work either 🙂
2017:04:25 17:14:46 sparkofreason @dergutemoritz not documented, but the collection must also support the Clojure interface for supporting into.
2017:04:25 17:15:27 dergutemoritz I doubt that spec is of much use for mutable collection types
2017:04:25 17:36:40 sparkofreason Actually, I want to use it for bifurcan collections.
2017:04:25 18:56:09 alexmiller covering Java colls was not really a goal of the spec coll preds
2017:04:25 18:58:29 alexmiller I can’t say we’ve really talked about it either way though. doesn’t seem like there’s any technical reason such a thing couldn’t exist. not sure if there are performance concerns in growing/shrinking non-persistent colls in generators.
2017:04:26 12:39:22 mbjarland a followup question to one I posted yesterday, if I have a conformer from string to seq of chars:
2017:04:26 15:37:32 dpsutton spec will be versioned then?
2017:04:26 15:38:40 alexmiller yes
2017:04:26 15:51:53 alexmiller versions will be ala core.async: 0.1.<git-change-derived>
2017:04:26 15:53:43 gfredericks the specs for clojure.core functions/macros will stay with clojure?
2017:04:26 15:54:01 bronsa no, that's what core.specs.alpha is for AFAICT
2017:04:26 15:54:07 dpsutton it'll be interesting to see how this impacts CIDER's support for spec. Now the tooling will have to make sure to match the version of spec that the application uses
2017:04:26 15:54:23 gfredericks @bronsa that's a namespace in the spec artifact?
2017:04:26 15:54:35 bronsa it's a separate contrib lib
2017:04:26 15:54:56 gfredericks oh so you need three artifacts if you want to use clojure with the core specs?
2017:04:26 15:55:15 bronsa I'm assuming clojure will depend on core.spec.alpha but dunno
2017:04:26 15:55:38 gfredericks I'm just trying to figure out if/how the specs can drift from the implementations
2017:04:26 15:55:51 gfredericks if the specs stay with clojure then that's not an issue
2017:04:26 15:59:31 gfredericks > Additionally, this is a first step towards increased support for leveraging dependencies within Clojure.
I don't know what that means
2017:04:26 16:02:53 alexmiller @bronsa yes, clojure will depend on the two new libs
2017:04:26 16:03:13 alexmiller but you can also specify a newer version and override the one its using
2017:04:26 16:03:48 alexmiller @gfredericks towards demonolithing clojure
2017:04:26 16:03:59 bronsa FWIW i have a similar dependency tree with t.a/t.a.jvm and I found that it's easier to just depend on the leaf dependency (t.a.jvm or in spec's case core.spec.alpha) and release concurrently when the root dependency needs a change
2017:04:26 16:13:38 mobileink just curious: why not put the “alpha” bit in the version string instead of the ns?
2017:04:26 16:15:04 tbaldridge (note: I'm speaking as a Clojure user here, not as someone who has authority on the subject)
2017:04:26 16:16:16 tbaldridge But the gist is you could have alpha, beta, v1, v2, v* code all in the same artifact if you wanted. That just because spec moves to "beta" or "RC" doesn't mean the alpha API has to go away.
2017:04:26 16:22:29 alexmiller at some point there will be namespaces that are not alpha. the alpha stuff will continue to work.
2017:04:26 16:23:38 mobileink ah, so this is just until we get to v1?
2017:04:26 16:24:07 alexmiller well, same idea may also apply to any breaking changes post v1
2017:04:26 16:24:15 mobileink ok, thanks
2017:04:26 16:24:27 alexmiller whether that’s done at the fn level or ns level is a future decision point
2017:04:26 16:36:14 mikerod The only problem I see with making libraries like spec (not part of lang) are “jar hell” dep issues
2017:04:26 16:36:42 mikerod in JVM land libraries using deps always leads to trouble because you end up using 2 libs with 2 different version of the same dep and they are incompatible
2017:04:26 16:36:46 mikerod (eventually)
2017:04:26 16:37:37 mikerod However, I know Rich’s talk at the last Clojure/conj was about never making a breaking change essentially. So maybe clojure.spec will be robust enough to avoid the problem and you can always just “take the newest version” of the dep when multiple lib deps diagree on version.
2017:04:26 16:38:15 mikerod I just know that I ran into issue with say, Plumatic schema when several libraries were using it and I was using those libraries and then schema came out with a set of breaking api changes
2017:04:26 16:39:42 mikerod This libraries having dependencies conflict problem has been one of the greatest pains of mine on the JVM it seems. I don’t think I’m alone there though. It can get difficult to deal with. I think it often encourages libraries to not use any dependencies. Which I feel is unfortunate for code-reuse or using useful utilities for documentation like spec/schema.
2017:04:26 16:40:17 mikerod Or in Java-land sometimes people do build-time repackaging of classes to some “internal” package name. I think it is harder in clj to confidently add a prefix to a set of namespaces though in a way that doesn’t miss anything.
2017:04:26 16:57:00 seancorfield My experience — six years of production Clojure with a code base of now 50K lines — is that API version conflicts are actually pretty rare in Clojure libraries in the wild.
2017:04:26 16:57:27 seancorfield They are more common with Java libraries, true, but Clojure people seem to be better about API changes.
2017:04:26 16:57:53 seancorfield (says the man who’s guided clojure.java.jdbc through several breaking API versions, however)
2017:04:26 17:00:53 bronsa seancorfield: c.j.j is different than most libs in that it's used much more by application code than by other libs so it's easier to avoid getting into dependency issues IMO
2017:04:26 17:03:31 seancorfield Yes, and that’s in line with why I think the utility libraries tend to be the worst offenders here. It’s why I have serious concerns about relying on Timbre, Carmine, Faraday, etc — even tho’ Peter seems really careful about not breaking the API across versions.
2017:04:26 16:59:37 seancorfield I think the danger comes mainly from Clojure libraries that themselves have a large number of transient dependencies where they rely on Swiss Army Knife utilities libraries for just one or two “cool functions” — which is an indictment of the questionable validity of such libraries, IMO. They should be broken up into much smaller libraries so you don’t have to pull in the whole thing for just one or two useful functions.
2017:04:26 17:00:00 seancorfield But this is a case of “do one thing and do it well” which such libraries inherently don’t follow 😞
2017:04:26 17:13:23 tbaldridge Right, part of the issues with "jar hell" are really problems with non-additive changes to APIs.
2017:04:26 17:19:22 mikerod Yeah, that’s sort of what I was thinking was the answer to this concern here. It’d be great to see that workout.
2017:04:26 17:19:58 mikerod It’s the “secret weapon” 😛
2017:04:26 17:46:13 alexmiller the whole idea is to make all versions of a library compatible
2017:04:26 17:47:15 alexmiller but the important thing here is that alpha is a time before this applies and breaking stuff is ok (you as a user have to track the changes)
2017:04:26 18:34:42 seancorfield @alexmiller I asked on the list but I’ll ask here too: will the new org.clojure/spec.alpha artifact be available for a few days before Clojure master SNAPSHOT actually removes the namespaces?
2017:04:26 18:57:58 seancorfield Ah, guess it just hasn’t been indexed yet? It doesn’t show up on https://search.maven.org yet. OK, got it, thanks!
2017:04:26 18:35:28 alexmiller as of just a few moments ago
2017:04:26 18:35:40 seancorfield Thank you! Much appreciated!!
2017:04:26 18:35:54 alexmiller but we are moving (slowly) towards an updated alpha release of clojure that uses it today
2017:04:26 18:36:15 seancorfield I’ll get our codebase switched over today then…
2017:04:26 20:01:13 dominicm @alexmiller I'm a little curious about this shift. If you depend on "1.9.0", is it part of clojure's api to expose it's transitive dependency on spec? Or should I always explicitly depend on spec if I'm using it in my lib?
2017:04:26 20:03:24 seancorfield I would expect core 1.9 will conditionally use spec at this point… So if you have spec loaded, you’ll get better macro error messages?
2017:04:26 20:03:35 alexmiller @dominicm clojure will have a compile-time dependency on spec.alpha so you can always rely on it being available. So you don’t need to explicitly list the new library as a dependency. You may do so though in order to specify a newer version than Clojure is depending on.
2017:04:26 20:05:16 alexmiller @seancorfield it is not currently conditional in any way
2017:04:26 20:05:26 pdlug Is there a mechanism in spec for namespacing keys as part of a conform? For example, a map parsed JSON will not have namespaced keys so I'm applying some functions that namespace them for me but this feels like a pretty common use case which must have a better solution
2017:04:26 20:05:57 alexmiller s/keys has :req-un and :opt-un options for this
2017:04:26 20:06:13 seancorfield So Clojure 1.9 will transitively bring in the spec library automatically? But you can bring in a different version yourself? How will that work if the spec library breaks the API?
2017:04:26 20:06:59 alexmiller @seancorfield yes. yes. then you’re broken (see “alpha = things may change”).
2017:04:26 20:07:39 alexmiller sorry, have to jump in the car but can pick up later
2017:04:26 20:08:53 pdlug @alexmiller Any pointers as to where? I've been through that a few times, I see the examples on spec'ing unqualified keywords but not coercing to namespaced keywords as part of a conform
2017:04:26 20:09:03 seancorfield Ah, so for clojure.spec to break Clojure itself, you’d already have to have a new public release of Clojure available (even if it is 1.10 alpha perhaps at this point)?
2017:04:26 20:09:55 seancorfield I’m hoping you wouldn’t make a new release of clojure.spec that then required users of it to rely on SNAPSHOT builds of Clojure — you’d at least have a new alpha of Clojure available by that point?
2017:04:26 20:10:28 seancorfield (as an active user of clojure.spec in production, this is all a bit frustrating and a little worrying!)
2017:04:26 20:57:54 seancorfield @pdlug I think maybe Alex misread what you were trying to do? There's no automated way to produce qualified keys from unqualified ones.
2017:04:26 20:58:35 pdlug @seancorfield Thanks, that's exactly what I was asking, that's unfortunate because this seems like a really common use case (JSON APIs, JSONB in PostgreSQL, etc.)
2017:04:26 21:00:38 seancorfield Since any namespace qualifier you would be adding is arbitrary, I don't see how spec plays into this: you control that arbitrary key qualification.
2017:04:26 21:02:33 seancorfield Somewhere you're converting string data to a Clojure representation of JSON. Why not introduce the qualifier at that point -- then spec with qualified keys directly?
2017:04:26 21:03:11 seancorfield That's why clojure.java.jdbc provides a :qualifier option, for example.
2017:04:26 21:03:29 mobileink i do it by typing (on the keyboard).
2017:04:26 21:04:52 mobileink for a json schema i'm working with i can come up with namespaces for everything, but automating it seems like a lot more work than just typing it in.
2017:04:26 21:09:30 alexmiller @seancorfield if you were asking about spec breaking Clojure, I don't see how we would do that. The integration points are pretty small.
2017:04:26 21:10:39 seancorfield @alexmiller I was considering a situation where you made a breaking API change in clojure.spec.alpha that could affect Clojure itself (since it depends on the earlier, non-breaking API)
2017:04:26 21:11:00 mobileink e.g.json schema puts "di" in schema "foo.bar". in spec: :foo.bar/di.
2017:04:26 21:13:39 pdlug I think this would be easier if there were more functions for manipulating maps to add namespaces to keywords. I keep copying the same keywordize-keys-ns function in every project to coerce incoming JSON into keywords w/ namespaces. Would be nice to also have a rename-keys that could do unqualified to qualified mappings en masse.
2017:04:26 21:13:44 seancorfield But if the integration surface is small, I guess that breakage is unlikely — and what I meant was that you’d already have to have a version of Clojure somewhere that used the new (changed) API in order to test/use it anyway. I just wanted to check that if you ever made such a breaking change to clojure.spec.*, you’d ensure there was a *non-SNAPSHOT* version of Clojure made available at the “same time”.
2017:04:26 21:14:09 bronsa @seancorfield the only thing clojure depends on from spec atm is s macroexpand-check, I find it really hard to imagine a situation where that could be broken
2017:04:26 21:15:03 alexmiller I would consider that broken :)
2017:04:26 21:16:05 seancorfield Well, the World Singles’ main codebase now uses clojure.spec.alpha with an explicit version dependency so I’m “happy” 🙂
2017:04:26 21:18:08 thheller @alexmiller any particular reason why the core.specs.alpha is a separate package?
2017:04:26 21:18:30 alexmiller Vs?
2017:04:26 21:18:57 thheller keeping it in clojure.spec.alpha or clojure.core I guess
2017:04:26 21:19:53 thheller just wondering if there are extended plans for it
2017:04:26 21:20:35 alexmiller You mean why in a different ns or why in a different project/artifact?
2017:04:26 21:20:45 thheller different artifact
2017:04:26 21:21:07 alexmiller It's logically separate and can evolve at its own rate
2017:04:26 21:22:27 thheller ok I'm still confused how the whole fdef / macroexpand-check integration is going to work, guess I'll have to wait and see
2017:04:26 21:22:31 alexmiller We made the decision long enough ago that I've forgotten the details tbh
2017:04:26 21:22:53 alexmiller @thheller it's exactly the same?
2017:04:26 21:23:12 alexmiller The code is just in a different artifact
2017:04:26 21:25:28 thheller ok, I keep thinking there is a circular dependency somewhere
2017:04:26 21:26:02 alexmiller There is
2017:04:26 21:26:47 alexmiller spec.alpha depends on Clojure
2017:04:26 21:26:59 alexmiller Clojure depends on spec.alpha
2017:04:26 21:27:32 alexmiller There is a bootstrap aspect to it
2017:04:26 21:29:38 thheller ah right, forgot the current code already does a var lookup.
2017:04:26 21:30:08 alexmiller I find it's best not to think about it too hard :)
2017:04:26 21:30:28 thheller hehe yeah I'm thinking more about CLJS at the moment
2017:04:26 21:30:47 thheller I was using the core.specs for CLJS but I guess that is just as easy with them in a lib
2017:04:26 21:30:58 alexmiller It's easier
2017:04:26 21:37:26 alexmiller The specs are not the same though for things like ns
2017:04:26 21:37:52 thheller well, everything besides ns is though
2017:04:26 21:38:39 thheller but still trying to come up with a good way to intergrate them with CLJS properly. might need to copy them anyways for self-hosted.
2017:04:26 21:59:09 alexmiller Open to helpful ideas
2017:04:26 22:01:18 thheller still trying to solve the "helpful errors" issue, currently the spec explain for something wrong in a let doesn't look much more helpful than the actual compiler error
2017:04:26 22:16:38 alexmiller Well more to come on that
2017:04:26 22:17:51 seancorfield @thheller I think the benefit will come from familiarity — all spec-powered errors will have a similar output format and folks will quickly get used to them (both from let and other core constructs, as well as from general libraries that use spec). And any improvements to how explain works will improve all errors from code that uses spec.
2017:04:26 22:18:13 seancorfield I mean, yeah, it’s a lot of output, but it’s very regular and structured.
2017:04:26 22:24:39 thheller not too sure about that
2017:04:26 22:25:06 thheller the most helpful part is demo/errors.cljs at 3:1😉
2017:04:26 22:27:25 thheller the problem in this case is that 1 is a number which doesn't have reader metadata, so you cannot narrow the cause down further
2017:04:26 22:28:07 thheller (defn x 1) is the actual source so 3:1 is the start of the (defn not the 1 which would be more accurate
2017:04:26 22:28:58 hiredman I don't follow
2017:04:26 22:29:05 hiredman that seems pretty clear
2017:04:26 22:29:43 hiredman 1 isn't a vector, 1 isn't that other big long thing
2017:04:26 22:30:03 hiredman each case gives the value that failed, and the predicate it failed
2017:04:26 22:30:33 hiredman and which spec the predicate came from
2017:04:26 22:30:46 thheller @hiredman the predicate can get cryptic as seen in the second message
2017:04:26 22:31:20 hiredman the grammar is cryptic 🙂
2017:04:26 22:31:38 thheller I think it is alright as well, a beginner might not be so inclined
2017:04:26 22:32:12 thheller but my issue for not is more about more accurate source locations
2017:04:26 22:32:32 thheller for the most part they are accurate enough though
2017:04:26 22:38:55 alexmiller The lack of reader metadata is annoying but not an endpoint necessarily
2017:04:26 22:42:44 thheller yeah coupled with the spec errors you can usually figure out whats wrong pretty quickly
2017:04:26 22:44:12 thheller it kinda breaks apart if the printed val gets too large though
2017:04:26 22:48:53 seancorfield I read that first error as “arg-list is not vector?” which is pretty clear. Even the second one is pretty straightforward: “args+body” is not a sequence of “args followed by body” — and if you (source defn) that should triangulate to the thing after the symbol being defnd is not either of those.
2017:04:26 22:53:24 thheller ... please don't get hung up on my toy example. This particular example is "good enough" with spec.
2017:04:26 22:53:47 thheller Parameter declaration "1" should be a vector is the default error message without spec ...
2017:04:26 22:54:14 thheller which IMHO is better than the spec error (but much less accurate)
2017:04:26 23:10:52 thheller @alexmiller will the new spec projects/artifacts get their own JIRA projects or should issues still go to CLJ?
2017:04:26 23:33:47 alexmiller CLJ
2017:04:27 02:52:11 danielcompton I'm not even sure exactly what this would mean, but is it possible to 'render' a spec for display in for example a Swagger UI? I'm wanting to build a web UI for displaying the specs of commands that could be sent to the system, but I'm not sure if this is even possible, as Spec is so flexible
2017:04:27 05:07:50 ikitommi @danielcompton we have been working on the api-docs thing, just finished on the spec -> json schema and will do the -> openapi next. also have some tooling for the runtime conforming needed to support different wire-formats. But like Schema (and ring-swagger), spec is more powerful than the openapi spec, so some information will be lost. Might be a place to do full-clojure thing, with own “spec-ui”.
2017:04:27 05:09:17 ikitommi any schedule for cljs-port of the spec-alpha?
2017:04:27 05:09:33 danielcompton @ikitommi yeah, I don't specifically want Swagger, it was just the closest thing I could think of, a spec-ui would be what I was after
2017:04:27 05:16:20 ikitommi I think the spec-ui could be done easily, as the spec forms can be serialized. Also, something like the https://github.com/metosin/schema-viz but for spec. Both on todo-list, but with million other things.
2017:04:27 06:22:13 thheller so either the v itself or the (->sym v) that is currently used in the message for the ex-data call
2017:04:27 06:23:54 thheller I'm experimenting with some different error presentations and currently the var isn't accessible so can't show the root spec that failed (other than "parsing" the (.getMessage ex))
2017:04:27 06:34:23 thheller could either do this in the macroexpand-check or in the catch that wraps the error in Compiler.java
2017:04:27 08:29:09 jindrichm Hi -- How do you make a spec for a particular value? (partial = :value) predicate doesn't seem very idiomatic.
2017:04:27 09:09:31 dergutemoritz Unless your value is nil or false🙂
2017:04:27 09:09:40 dergutemoritz But then you have specific preds
2017:04:27 09:11:00 jindrichm Thanks! This should work fine and looks more idiomatic than partial.
2017:04:27 09:12:55 vandr0iy Hi clojurians!
How does one make up a spec for a huge nested map that may have different values for the same key but in different places?
like this:
How do I say that :foo is usually a string, but inside a certain context (b/baz in this case) it's a number, and in certain others (b/quux) it's a vector?
2017:04:27 09:22:42 dergutemoritz @vandr0iy If you're asking how you can make s/keys behave like that, the short answer is: it's not what it was designed for but rather it relies on namespaced keywords. However, you could define keywords of the same name in different namespaces representing your various contexts and piece the specs together accordingly with :req-un and :opt-un.
2017:04:27 10:34:44 mbjarland @vandr0iy errr....ok, they might help, but don't think they solve your different values for the same key problem, @dergutemoritz answer seems to be it
2017:04:27 11:33:06 ikitommi @vandr0iy @U4VDXB2TU: data-spec can do that, but I would use the normal s/keys if you need to reuse the attribute specs elsewhere. If it’s just a ~one-time thing, then something like this:
2017:04:27 11:48:34 vandr0iy in the end I just ended up doing specs for every single element that might be recognized as a pattern in its own namespace with the same name as it's called in the data structure I want to specify - but in its own namespace. For instance:
2017:04:27 11:54:25 thheller @alexmiller not sure, will try it. thx
2017:04:27 11:55:27 ikitommi thanks @alexmiller, looking forward to it.
2017:04:27 11:59:29 thheller @alexmiller yep should do, I can get the name of the spec via the meta:clojure.spec/name
2017:04:27 11:59:44 thheller which corresponds to the var
2017:04:27 12:16:04 alexmiller I guess I'm not sure why you want the var vs the name
2017:04:27 13:33:10 thheller for now just the name, maybe the var later. thinking that the var might contain more metadata, will see if I need that
2017:04:27 13:34:34 thheller probably not though
2017:04:28 12:05:02 christianromney hmm bear with me as i try to understand this. if we have a couple of invariants like: 1. we don't break APIs in non-alpha software and 2. if we do break an API, we pick a new name and 3. clojure.core (non-alpha) will depend on clojure.spec.alpha, then doesn't this imply that for a version of clojure to depend on a clojure.spec (and NOT depend on clojure.spec.alpha), it will have to change its namespace, say to something like clojure2.core? or have i missed something obvious?
2017:04:28 12:29:43 alexmiller clojure.core doesn't depend on clojure.spec.alpha
2017:04:28 13:03:58 christianromney thanks @alexmiller although I'm more confused now than before. i misinterpreted your comment above about the circular dependency. perhaps i should just go look at the artifacts 🙂
2017:04:28 13:18:39 christianromney hmm, @alexmiller could you elaborate a bit more? I'm just not understanding how to reconcile that with your previous explanation that "clojure will have a compile-time dependency on spec.alpha so you can always rely on it being available. So you don’t need to explicitly list the new library as a dependency." sorry if i'm just being a dunderhead...
2017:04:28 13:29:27 alexmiller my original statements are about artifacts. yours seemed to be about namespaces, so I answered in those terms. can you clarify which level you’re talking about?
2017:04:28 13:30:26 alexmiller Clojure 1.9 will ship with an alpha version of spec available - use it if you like (but beware that things in spec may change)
2017:04:28 13:30:54 alexmiller that’s the whole story
2017:04:28 13:35:34 mhuebert Is there a natural way to have the generator for a regular expression spec return a vector instead of a sequence?
2017:04:28 13:36:57 alexmiller currently, no
2017:04:28 13:37:31 alexmiller but it is something many people (including myself) have felt the need for and something I have discussed a couple times with Rich
2017:04:28 13:37:42 alexmiller and I think ultimately something will plug that gap
2017:04:28 13:46:05 mhuebert Ok, thanks. (The two use cases I have in mind are hiccup forms and function argslists)
2017:04:28 13:47:11 alexmiller yes, I’ve run into it in spec’ing the latter
2017:04:28 14:04:53 christianromney @alexmiller oh i see so you'll declare a dep on the artifact but never require it is that right?
2017:04:28 14:05:25 alexmiller mostly - there are a couple things in Clojure that use spec
2017:04:28 14:05:56 alexmiller one is the args check for fdef macro specs during macroexpansion (so in the compiler)
2017:04:28 14:06:25 alexmiller the other is the lookup and inclusion of specs in printed docs from clojure.repl/doc
2017:04:28 14:06:57 alexmiller both of those are kind of implicit rather than being part of the API though
2017:04:28 14:08:00 christianromney ok thanks, that gives me much more to chew on. 🙂 i don't see this necessarily impacting me, i just really want to grok the engineering you're doing 🙂
2017:04:29 06:47:29 fossifoo @ikitommi not sure, where to put this on slack, but i currently try to use spec-swagger from cljs/node and it seems like spec-tools depends on a lot of internals from clojure.spec and doesn't currently build on cljs. i get No such var: s/def-imp. can you (or anybody else here) confirm that?
2017:04:29 06:51:18 fossifoo eh, there's a cljs.spec.cljc and a cljs.spec.cljs? how does that work?
2017:04:29 11:40:30 alexmiller Platform-specific is used first, then falls back to cljc if not found
2017:04:30 05:50:38 lincpa Make readability better spec will expressed in the form of SQL DDL?
2017:04:30 11:14:49 fossifoo hmmm. since ther e is a def-impl in cljs.spec, i guess this should work then
2017:04:30 11:15:34 fossifoo and actually, i had to rename the imports to cljs.spec from clojure.spec although i think the compiler should find it with either name
2017:04:30 11:15:40 fossifoo so maybe something else is worng
2017:05:01 03:00:30 olivergeorge Just putting it out there: I love clojure.spec
2017:05:01 17:55:49 ikitommi @fossifoo spec-tools tests are run on travis with node too, so it should work, tested with cljs-version 1.9.518. But to enable specs to be created a runtime, data-specs uses the functional internal of clojure.spec (the ^:skip-wiki fns :() => just extracted ‘em into separate namespace - don’t have to use those. Spec-swagger needs a lot of attention, next on the todo-list, will start by converting to vanilla specs.
2017:05:01 17:57:04 ikitommi If I remember right, there will be public *functions* to create specs some time in the future.
2017:05:01 18:03:31 ikitommi is there a way to unform a conformed value of s/map-of?
2017:05:01 18:29:04 ikitommi thanks. is it still a plan to create a public layer of functions to create the specs (besides the macros)?
2017:05:01 18:30:05 ikitommi also, would be interested in having a functional way to register specs. the def-impl looks private as it has the :skip-wiki meta on it.
2017:05:01 18:30:53 ikitommi (and “Do not call this directly”…)
2017:05:01 18:43:56 fossifoo @ikitommi okay, just wanted to check the status. i want to go the other way around (use swagger json to generate testdata) and think i might go via spec. a generic spec will also be very helpful for parsing. changing it to standard spec would indeed be better for me. it's hard enough to understand one DSL
2017:05:01 19:02:22 alexmiller @ikitommi still tbd. As I've said in the past, you can get same benefits of functional API by unforming with spec specs
2017:05:01 22:55:14 danielcompton @thheller what was the form that gave that error? Here's what I got, I must be misunderstanding what you meant:
(let [a 1
b 2
c 3
d]
a)
clojure.lang.ExceptionInfo: Call to clojure.core/let did not conform to spec:
In: [0] val: () fails spec: :clojure.core.specs/bindings at: [:args :bindings :init-expr] predicate: any?, Insufficient input
:clojure.spec/args ([a 1 b 2 c 3 d] a)
2017:05:01 23:05:10 mobileink d needs a value.
2017:05:01 23:17:34 danielcompton yep, I know, I'm trying to reproduce the error @thheller got
2017:05:02 00:35:16 jatkins Hi all! Is anyone else getting errors like Call to clojure.core/ns did not conform to spec? I'm working on a clj/s project and I want to use spec with reframe. I've tried the latest version of clojure (`1.9.0-alpha16`) and I get the aforementioned error. However, when I move the dependency to 1.9.0-alpha10 the error does not occur. Any idea for the change? BTW This is my first attempt at using a clojure alpha version, and I have many dependencies that may be incompatible (I feel that this is unlikely in this case, though correct me if I'm wrong). If needed I can provide more information.
2017:05:02 00:41:47 alexmiller The namespaces for spec just changed in alpha16 and cljs and other deps are not compatible with it
2017:05:02 00:42:05 alexmiller You might want to fall back to alpha15
2017:05:02 00:43:49 jatkins I actually tried every alpha from 16 to 10, and 10 was the first one that worked.
2017:05:02 00:47:33 alexmiller What's the actual error you're getting?
2017:05:02 00:48:18 alexmiller Many libs had bad ns deps - most of those have been fixed and newer versions exist
2017:05:02 00:52:43 jatkins I put a gist up: https://gist.github.com/JJ-Atkinson/a1d19e48d9440c2ca9f0bed219c8201f. How can I tell where the error is originating from? I see that it likely is from (onelog/core.clj:1:1), but I don't know where that is, (probably in timbre though, I'll see if I can bump that version up).
2017:05:02 01:05:28 jatkins Onelog is part of ring apparently part of ring-middleware-logger. I'm currently using the latest of both. So, is the issue not with clojure itself, but with onelog? If that's the case, I'll see about a pull request/issue to them.
2017:05:02 01:27:29 jatkins So, after crawling through the error message more thoroughly, it appears that onelog is using the syntax (ns ... (:import (com.something SomeClass))). Spec changed something about ns with -alpha11, and it was already reported to them https://github.com/pjlegato/onelog/issues/3. Thanks for the help!
2017:05:02 02:40:36 drewverlee Apologies as this has probably been asked a dozen times, but does clojurescript current support generators?
2017:05:02 03:38:56 tbaldridge @drewverlee not that I know of, but generators tend to encourage a impure code. Most of what you would need generators for can be done with more functionally pure constructs like map, filter and for.
2017:05:02 06:34:05 curlyfry @alexmiller So if I want to use alpha16 and have cljs in my project I should wait for a compatibility patch or something similar?
2017:05:02 06:53:38 dergutemoritz @tbaldridge I think @drewverlee is referring to the test.check kind of generators, given that he's asking in here 🙂 @drewverlee test.check supports cljs at least, not sure if clojure.spec's wrapping of it works with cljs, though.
2017:05:02 11:07:09 lwhorton With the announcement of spec moving to its own lib, I see that spec.alpha is under org.clojure, but nothing like org.clojure/clojurescript.spec.alpha? Has it not moved yet?
2017:05:02 12:08:20 alexmiller Not yet
2017:05:02 12:43:05 lwhorton alrighty, thanks
2017:05:02 21:55:15 wei what’s a good example of adding validation on a macro?
2017:05:02 22:07:35 mobileink what do you want to validate?
2017:05:02 22:22:34 wei actually, might need to rethink this
2017:05:02 22:25:00 wei I have a wrap-error macro that takes a map of context data and a body.
I want to make sure that the macro user doesn’t forget to pass in the context map, because if so, the first line of the body wouldn’t get executed. but I’m not sure it’s possible to do that at compile time
2017:05:02 22:39:11 mobileink what context map?
2017:05:02 22:39:45 mobileink you mean you want to validate args?
2017:05:02 22:41:02 mobileink no way to do this without a macro?
2017:05:02 22:45:42 mobileink you want to use spec to validate args? add a clause to your let?
2017:05:02 23:07:51 wei sorry let me try a better explanation. I want people to use this macro like this:
(wrap-error {:some :data :a 1 :b 2} ...body...)
where the first argument is some metadata to associate with the error, if there is one. the macro has been used mistakenly like this:
(wrap-error ...body...)
without the first arg, resulting in incorrect behavior because the first element of body is interpreted as the metadata and not executed. was wondering if there was a way to use spec to prevent this at read-time. less about validating the metadata itself so much as the presence of it.
2017:05:02 23:09:08 wei might be hard to distinguish at read-time though. i wonder if this has to be a run-time check (`(s/valid? map? args)`)
2017:05:03 00:01:15 mobileink why do you need a macro? you're just wrapping try/catch, no?
2017:05:03 00:02:37 mobileink i'm not an expert, but the usual advice is to avoid macros.
2017:05:03 00:05:41 mobileink "wrap-error" strikes me as misleading. you're really just wrapping a fn, which may not have an error, no?
2017:05:03 01:57:17 cfleming The doc seems to imply that s/alt is ordered choice, i.e. the alternatives will be tested in order and the first match returned. Is that correct?
2017:05:03 02:00:29 tbaldridge @cfleming yes from the doc string: "Returns a regex op that returns a map entry containing the key of the
first matching pred and the corresponding value. "
is there a way to narrow error location down? (:dsc/patch is a compound spec, ~30 lines long)
2017:05:03 09:24:47 slipset I really wished spec was spec'ed. Just lost som time on why s/keys was valid on my invalid maps. Reason? I used :un-req instead of :req-un
2017:05:03 11:04:04 misha is there something built in for what I'am trying to achieve (generator for next-id, which never returns a duplicate)?
2017:05:03 11:55:08 misha how does one combine two map specs? (union)
2017:05:03 11:58:57 misha here I'd like to avoid custom reduce-through-hashmap predicate, and combine s/keys and s/map-of specs instead
2017:05:03 12:35:10 misha that one fails because s/and results in empty set of valid keys. s/merge will result in an empty set too.
it seems like I need a custom predicate after all:
2017:05:03 12:06:41 misha @souenzzo tried that right now. it also gives me
ExceptionInfo Couldn't satisfy such-that predicate after 100 tries. clojure.core/ex-info (core.clj:4725)
2017:05:03 12:07:42 souenzzo I've never used generators. =/
But I use tons of s/merge
2017:05:03 12:14:19 lwhorton has anyone been able to use generators with cljs? I’m not really sure how (I don’t even see gen defined in the cljs.spec.impl.gen ns, and my google foo is failing
2017:05:03 12:26:08 bhagany @lwhorton yes, but I don't remember off the top of my head what the differences with clj were...
2017:05:03 15:46:28 chillenious Hi. Trying out spec and running into a problem right away. Simply adding [clojure.spec.alpha :as s] to a :require section in my clojure file, and then trying to load that file in the repl (Leiningen) via use gives me this error:
java.lang.ClassCastException: clojure.spec.alpha$regex_spec_impl$reify__1340 cannot be cast to clojure.lang.IFn
When I use require instead of use, it seems fine. Anything I'm doing wrong/ known bug?
2017:05:03 16:09:09 chillenious actually, scrap that... I get this error with either use or require whenever I pass in the :reload-all flag
2017:05:03 16:24:26 seancorfield Yes, there’s an AOT-related issue at present with the version of spec.alpha that Clojure 1.9.0 Alpha 16 pulls in. If you explicitly depend on org.clojure/spec.alpha "0.1.108" the problem should go away.
2017:05:03 16:24:53 seancorfield It specifically seems to affect REPL workflows and anything that reloads code.
2017:05:03 16:28:55 chillenious thanks!
2017:05:03 19:08:15 slipset I’m writing specs for my om components, which basically are vararg functions of which I normally just care about the first arg.
2017:05:03 19:08:29 slipset So I end up with something like
2017:05:03 19:09:12 slipset the :rest (s/* any?) bit seems a bit silly.
2017:05:04 01:03:47 alexmiller Why?
2017:05:04 07:22:01 slipset Silly might be wrong, but tedious maybe?
2017:05:04 07:25:40 slipset what I’d like to be able to state is “I only care about the first argument and it should be a grid ”
2017:05:04 07:54:33 dergutemoritz @slipset That's what you're stating with that spec, though, isn't it? 😄 If you feel it's too noisy and/or you're going to use it in many places, you could define a macro that hides that noise.
2017:05:04 07:56:17 slipset @dergutemoritz true
2017:05:04 07:57:44 slipset apart from that, it seems like spec is giving me exactly what I want: the possibility to sprinkle “types” over my program when it’s “done”.
2017:05:04 21:41:58 joshjones Namespace myproj/myns.clj contains a function, which requires specs defined in myproj/specs.clj. I have the function fdef'd and it currently sits next to the function itself in myns. However, I'd like to move the function spec away from the function itself. So myns depends on specs, but the fdef'd function depends on knowing about the function itself in myns, so I can't move it to specs. Circular dependency. Where are you putting your spec'd functions (or in general, any of your specs) in your project?
2017:05:04 22:07:45 ghadi the specced namespace IMHO should not require the specs
2017:05:04 22:08:10 ghadi it was explicit design of clojure.spec that you can spec things you don't own
2017:05:04 22:08:48 misha
(s/fdef myns/my-fn ...)
without explicitly importing myns in specs, @joshjones
2017:05:04 22:18:09 mobileink misha: is that missing a colon?
2017:05:04 22:21:49 misha no, its a fully qualified fn name, like clojure.string/blank?
2017:05:04 22:24:32 mobileink misha: so you have to require that ns in your spec file? or otherwise define it there? that seems odd, but i'm a relative noob at spec.
2017:05:04 22:27:45 misha no, requireing needed. afair, spec just uses fn symbol to register spec, and does not call actual fn, so the actual require'ing is not needed
2017:05:04 22:08:51 ghadi ^
2017:05:04 22:12:00 joshjones many thanks to you both -- @ghadi , by "the specced namespace" you mean myns or specs in my example above?
2017:05:04 22:12:11 ghadi myns
2017:05:04 22:12:53 ghadi yourproj.specs should depend on yourproj.myns, not the other way around, like @misha 's example
2017:05:04 22:12:57 misha yeah, myns might not need specs imported unless it uses one for explicit validation as a workflow step
2017:05:04 22:13:26 joshjones it does -- (s/assert ::specs/myspec some-data)
2017:05:04 22:14:30 joshjones Many examples of using valid?, conform, etc also
2017:05:04 22:15:46 misha then either keep fdef's closer to functions, or use fully qualified fn names in fdefs w/o explicit imports
2017:05:04 22:16:44 mobileink you only need to require if you want to use an alias, is that correct?
2017:05:04 22:25:48 misha mobileink: I'd say: only if you need to call something from that ns, or import for side effects. alias works w/o explicit import (at least it worked for me just now)
2017:05:04 22:28:38 mobileink hmm, need to experiment more. i could swear i've used :foo.bar/baz without require or import, but theres a goid chance i misunderstood what i was doing. ;)
2017:05:04 22:30:07 misha 2 types of specs: for data – keyword (:foo/bar), for functions – symbol (foo/baz)
2017:05:04 22:31:12 mobileink i didn't think you could alias an ns without "making" it whether by require or sth else. i.e. just a symbol won't work. but i'm away from my machine, will try later.
2017:05:04 22:31:56 mobileink aha, my bad - have not yet worked my way up to fn specs. sorry!
2017:05:04 22:32:49 misha I am trying now, and I see no spec ns import required either: I can define specs for functions in foo.specs w/o importing foo.fns, and can use specs for assertions in foo.fns w/o importing foo.specs
2017:05:04 22:34:24 mobileink time for me to take another sip from the firehose. not so easy when your lips have already been ripped off!
2017:05:04 22:34:25 misha two namespaces, neither imports the other. instrumentation and validation work, because specs get registered in global registry
2017:05:04 22:35:23 misha cc @joshjones
2017:05:04 22:36:11 mobileink i think i assumed reg keys are always kws. bad programmer!
2017:05:04 22:37:50 misha I have spec guide open at all times while doing spec development, for quick reference https://clojure.org/guides/spec
2017:05:04 22:39:45 mobileink me too - just haven't needed to deal with fns yet. it's hard enough to figure out how to do what i need with maps!
2017:05:04 22:40:20 misha the only require use case I can think of now – is google closure compiler advanced optimization, where specs might get dead-code-eliminated. Haven't tried that yet though
2017:05:04 22:41:56 mobileink fwiw i just got my clj map specs working in cljs, using cljc code. works like a charm - but haven't done advanced opt yet. shivers
2017:05:04 22:45:55 joshjones thanks @misha, very helpful
2017:05:05 10:27:30 slipset I seem to remember that the fdef:ret is not enforced/checked under instrument. Is that correct, and if so, how does one go about checking that the return value of a fn conforms to the spec?
2017:05:05 13:18:33 slipset The reason I was wondering (and wanting this) is that in cljs, I run my dev-setup with (stest/instrument) which is super nice, but it would be nice to get the :ret bit as well while running in dev-mode
2017:05:05 14:15:34 joshjones @slipset The answer you found requires an additional binding in many cases, and will generally mess up the flow of the function. I'm in the same boat as you, looking for a way to automatically check the return, with the ability to disable it for production. Not having the ability to test :ret except through stest makes a function spec much less useful. something like this works at least on a small example:
2017:05:05 14:18:33 joshjones it can be turned on/off with s/check-asserts, so it may be a decent workaround to this limitation (it may be by design, but it is still a limitation)
2017:05:05 18:20:08 onetom please share your thoughts on it
2017:05:05 18:58:41 dnolen I just changed cljs.spec to cljs.spec.alpha now would be a good time to try master - would like to cut a release for this
2017:05:06 02:10:21 jimmy dnolen: yay 😄
2017:05:06 19:23:52 reefersleep When using spec/explain on a spec which contains spec/coll-of in cljs and passing in an invalid datastructure, I don't seem to be getting any information about the offending element of the coll, only that something is wrong within the coll. Example:
For contrast, here is the ouput for a Schema validation similar to the spec explain:
(schema/validate [schema/Str] ["hej" "yo" 9])
#error {:message "Value does not match schema: [nil nil (not (cljs$core$string? 9))]", :data {:type :schema.core/error, :schema [
Can I do something to make Spec give me more detailed output, similar to that of Schema?
In my actual use case, the spec/coll-of contains more complex maps with nested specs, and if one thing goes wrong somewhere in the coll-of, I'm left in the dark, fumbling for the specifics.
2017:05:06 19:23:52 reefersleep When using spec/explain on a spec which contains spec/coll-of in cljs and passing in an invalid datastructure, I don't seem to be getting any information about the offending element of the coll, only that something is wrong within the coll. Example:
For contrast, here is the ouput for a Schema validation similar to the spec explain:
(schema/validate [schema/Str] ["hej" "yo" 9])
#error {:message "Value does not match schema: [nil nil (not (cljs$core$string? 9))]", :data {:type :schema.core/error, :schema [
Can I do something to make Spec give me more detailed output, similar to that of Schema?
In my actual use case, the spec/coll-of contains more complex maps with nested specs, and if one thing goes wrong somewhere in the coll-of, I'm left in the dark, fumbling for the specifics.
2017:05:07 20:40:43 mobileink reefersleep: it tells you precisely that your val contains a non-string.
2017:05:07 20:42:37 mobileink reefersleep: do you have an example of being left in the dark?
2017:05:08 14:07:35 reefersleep @U0LGCREMU: here's a more elaborate example of what I mean in regards to Schema providing more information than Spec, leaving me desiring more from Spec.
Spec:
(spec/def ::id int?)
(spec/def ::name string?)
(spec/def ::person (s/keys :req-un [::id
::name]))
(spec/explain (spec/coll-of ::person []) [{:id 1 :name "john"} {:id 2 :name :heather}])
val: [{:id 1, :name "john"} {:id 2, :name :heather}] fails predicate: (coll-checker :sleepydo.db/person)
;; No information about _which_ element of the coll failed to validate, nor details about how it failed to validate
Schema:
(def Id schema/Int)
(def Name schema/Str)
(def Person {:id Id
:name Name})
(schema/validate [Person] [{:id 1 :name "john"} {:id 2 :name :heather}])
#error {:message "Value does not match schema: [nil {:name (not (cljs$core$string? :heather))}]", :data {:type :schema.core/error, :schema [{:id
2017:05:06 20:21:13 xiongtx Does s/with-gen take a thunk that returns a generator for laziness? Is that way it doesn’t just take a generator directly?
2017:05:07 19:08:37 reefersleep Is there an easy way to refer to a registered spec under a different name in a map spec?
2017:05:07 23:09:21 taylor @U0AQ3HP9U if I understand your question, I don't think there is and I also think that's by design
2017:05:07 23:11:47 alexmiller no, other than by using :req-un for a spec with the same name but different namespace in s/keys
2017:05:08 13:43:12 reefersleep Thank you @U3DAE8HMG and @U064X3EF3 for your succinct answers!
2017:05:07 22:12:26 caio is there an easy way of defining spec for something inside a ref?
2017:05:07 22:12:47 caio I'm trying to write a spec for funcool cats' either monad, but no luck
2017:05:07 22:15:12 caio I was able to write something to conform/unform, but got stuck when trying to write a proper generator
2017:05:07 23:11:54 alexmiller @caio not yet
2017:05:07 23:12:13 alexmiller possibly will be added in the future
2017:05:08 01:11:27 caio what about a generator for records? can that be done?
2017:05:08 01:42:12 caio also, I think my problem is a bit more generic than writing specs for refs. I was trying to write a spec for a type defined in cats. after seeing gen/fmap, looks doable
2017:05:08 01:50:50 gfredericks yes technically that will work
2017:05:08 01:51:28 gfredericks normal test.check usage is aimed at writing generators for data, not stateful objects
2017:05:08 01:51:55 gfredericks I'm not sure what will happen if spec starts having specs for reference types
2017:05:08 01:59:40 caio @gfredericks this is not stateful. IDeref was implemented as a syntatic sugar
2017:05:08 02:00:01 gfredericks @caio okay that's not bad then
2017:05:08 02:00:32 gfredericks @bhagany btw the bind+return thing there can be simplified to fmap
2017:05:08 02:00:58 bhagany @gfredericks thanks 🙂
2017:05:08 07:22:28 mpenet any reason why (s/assert x) compiles to x instead of nothing (like core/assert) when s/*compile-asserts* is false?
2017:05:08 08:12:34 Niclas Is there any way of specing async channels and the data that may be sent over them?
2017:05:08 08:30:18 misha @looveh wrap >! with validating fn? you can put anything on a channel, so any limitation would need to be enforced by client code, I think.
2017:05:08 08:32:03 Niclas @misha That’s one way to do it, but it would also be useful to be able to spec channels as function arguments
2017:05:08 08:32:14 Niclas Not just the actual get/put on channels
2017:05:08 08:32:58 misha wrapping >! would bail right away. wrapping <! would bail sometime later/never. Whichever would suite you best.
2017:05:08 08:34:03 misha wrap channel in a record, spec record, spec protocols it implements?
2017:05:08 08:35:21 misha *with multispec.
2017:05:08 08:35:49 Niclas Yeah that would work, feels a bit hacky though imo since channels are part of a core lib
2017:05:08 08:36:15 misha I'd imagine you need to add something to the channel-thing itself to be able to use different specs for different channels.
2017:05:08 08:37:17 misha so either spec entry/exit points, or wrap, and spec/pass around/expect wrapped channels.
2017:05:08 08:42:29 misha ¯\(ツ)/¯
2017:05:08 08:44:18 Niclas Hahah that’s amazing!
2017:05:08 13:36:00 tbaldridge @misha @looveh specs could be put on a filter transducer on the channel, or a call to map that throws. On an exception the exception would be handed to the channels' exception handler.
2017:05:08 14:06:00 misha @tbaldridge I think @looveh's intention was to "spec a function, so that wrong kind of channel passed as an argument would throw", not "make sure channel would not accept random stuff".
I like "spec as a filter" much more than macro above, thank you.
2017:05:08 14:07:28 misha @tbaldridge can that channel's filter be used as a dispatch value once channel is constructed? is it accessible from outside?
2017:05:08 14:09:43 tbaldridge @reefersleep what does explain-data show for the spec code?
2017:05:08 14:11:55 reefersleep @tbaldridge: Nothing more.
2017:05:08 14:12:06 reefersleep Note that I'm talking about cljs, not clj. 🙂
2017:05:08 14:16:19 misha @reefersleep why is there [] in (spec/coll-of ::person [])?
2017:05:08 14:17:28 misha for vector it is (spec/coll-of ::person :kind vector?), isn't it?
2017:05:08 14:20:07 misha clojure gives me
(s/explain-data (s/coll-of string? []) ["a" "b" 1])
CompilerException java.lang.IllegalArgumentException: No value supplied for key: []
and in cljs (js) I can imagine it fails silently somewhere, but produces compromised output anyway
2017:05:08 15:46:35 reefersleep I thought I should use :kind vector? as well, until I tried it.
(spec/explain-data (spec/coll-of ::person :kind vector?) [{:id 1 :name "john"} {:id 2 :name :heather}])
---- Could not Analyze <cljs form> line:1 column:20 ----
Wrong number of args (3) passed to: spec/coll-of
1 (spec/explain-data (spec/coll-of ::person :kind vector?) [{:id 1 :name "john"} {:id 2 :name :heather}])
^---
---- Analysis Error ----
Then I tried without any :kind-indication:
(spec/explain-data (spec/coll-of ::person) [{:id 1 :name "john"} {:id 2 :name :heather}])
---- Could not Analyze <cljs form> line:1 column:20 ----
Wrong number of args (1) passed to: spec/coll-of
1 (spec/explain-data (spec/coll-of ::person) [{:id 1 :name "john"} {:id 2 :name :heather}])
^---
---- Analysis Error ----
So I looked up the documentation:
(cljs.repl/doc spec/coll-of)
-------------------------
cljs.spec/coll-of
([pred init-coll])
Macro
Returns a spec for a collection of items satisfying pred. The generator will fill an empty init-coll.
2017:05:08 14:26:56 alexmiller if you remove that invalid trailing [], clojure says:
2017:05:08 15:50:33 reefersleep alexmiller: So I guess what I'm seeing is a feature of clj spec missing (as of now) from the cljs port. I kind of suspected this, as I thought I'd seen more useful output in guides/tutorials at the time spec came out.
2017:05:08 14:28:28 alexmiller which seems better than the schema message to me
2017:05:08 14:50:50 andrewboltachev Hello. When I have keys #{:a :b :c} and a spec ::foo, how do I get
2017:05:08 14:57:34 andrewboltachev sure, but I'll need to repeat for 4 or 5 different types of ::foo
2017:05:08 14:58:07 misha afaik, you have to alias desired keys to their desired values' specs
2017:05:08 14:59:55 misha or provide your own generator, which would make sure all the keys are present. (I'd just alias things, custom generators might screw up s/explain-data error address within the structure)
2017:05:08 15:00:49 andrewboltachev yep the custom generators are obviously "approach from the opposite side"
2017:05:08 15:01:14 andrewboltachev and well, honestly also having N keys as I do might mean semantically the sequence, not a map
2017:05:08 15:02:06 misha plenty of options there as well: s/coll-of, s/tuple
2017:05:08 15:02:33 andrewboltachev yep, would take a look at every function
2017:05:08 15:55:18 alexmiller I think you must be using an older version of ClojureScript - that’s not what I see with the latest
2017:05:08 15:55:44 alexmiller afaik, ClojureScript is basically at parity with the Clojure version wrt spec
2017:05:08 15:57:59 alexmiller I’m using ClojureScript 1.9.293 btw
2017:05:08 16:05:43 reefersleep I thought I was up to date, but apparently, I misread the version number... 1.9.93. 🙂
2017:05:08 16:07:21 reefersleep I'll try updating the version, I'm sure that'll make things better for me!
2017:05:08 16:07:24 reefersleep Thanks all!
2017:05:08 16:29:36 alexmiller what’s 200 commits between friends? :)
2017:05:08 16:37:01 reefersleep Now I'm having a different problem, not spec-related - I'm trying to destruct with (let [{:keys [a b] :as full} (fn-call something...)] ...
2017:05:08 16:41:06 reefersleep I've not changed the code for that destructuring or what goes on underneath between cljs version changes. However, the output of (fn-call ...), which is a map {:a "a's value" :b "b's value"}, - I can tell by printing at the end of my fn-call - turns into {[:a "a's-value] [:b "b's value"} upon destruction (I can tell by printing full), so I cannot destruct`a` or b.
2017:05:08 16:41:12 reefersleep So weird.
2017:05:08 16:51:00 reefersleep (asking in #clojurescript , too)
2017:05:08 16:55:03 caio would be nice if someone could validate if this is safe. I had to go through spec's code for doing this, and some stuff were not really clear to me
2017:05:08 16:59:33 caio also saw something in spec-tools that they do some checking on conform that made me think it may not be thread-safe. idk if that's the case
2017:05:08 17:27:19 ikitommi @caio about spec-tools - if the conform doesn’t use threads, I should be ok. But not optimal. would be nice to use just 3rd parameter in conform (http://dev.clojure.org/jira/browse/CLJ-2116)
2017:05:08 17:27:44 caio You can say it's a non conforming or between two choices If you want to push it (and probably piss someone 😆 )
2017:05:08 18:00:42 mobileink I’m getting a strange error that seems related to the switch to spec.alpha. In my code this works: (clojure.core/require [pagespace-sym] :reload). But if I change it to :reload-all I get the following error:
java.lang.ClassCastException: clojure.spec.alpha$regex_spec_impl$reify__1340 cannot be cast to clojure.lang.IFn
clojure.lang.Compiler$CompilerException: java.lang.ClassCastException: clojure.spec.alpha$regex_spec_impl$reify__1340 cannot be cast to clojure.lang.IFn, compiling:(clojure/tools/logging.clj:1:1)
2017:05:08 19:48:30 alexmiller This is a known issue with spec.alpha not being aot compiled
2017:05:08 19:49:31 alexmiller It's been fixed in a new build of spec.alpha - 0.1.108. We haven't released a new Clojure alpha yet that depends on it but you can specify that to override
2017:05:08 20:03:00 mobileink @alexmiller thanks!
2017:05:09 08:42:27 mpenet Would it make sense to have a ^:fdef (or ^:spec) meta on fn to do the :pre/:post automatically from a fdef ? Before you mention instrument, it's not the same (instrument triggers gen).
2017:05:09 08:45:37 mpenet that would avoid some boilerplate :
2017:05:09 08:51:09 mpenet the spec and where it's applied is still decoupled, the latter would basically expand into the former (almost, omited the potential :fn part of the spec)
2017:05:09 08:55:48 alexmiller no, not interested in that. first, this re-couples vars and function specs which are intentionally not coupled (and can be created in either order). second, we don’t generally want to enable pre/post checking of specs for performance reasons.
2017:05:09 08:55:58 mpenet (it doesn't have to use :pre/:post, would open doors to auto conforming via metadata hint etc)
2017:05:09 08:56:59 alexmiller I don’t think Rich has any interest in that
2017:05:09 08:58:18 mpenet alright, not surprised, I guess this might end up being done as part of a lib
2017:05:09 08:59:10 mpenet pre/post is toggleable (s/assert as well), but again this would/should be done without these 2 anyway
2017:05:09 09:04:17 mpenet @alexmiller about ordering I am not sure I follow, if that expands to s/assert calls it wouldn't matter/cause any issue
2017:05:09 09:10:01 alexmiller pre/post effects what ends up in the compiled code (even if toggled off) and thus has a runtime cost
2017:05:09 09:10:25 alexmiller s/assert can actually be omitted from compilation entirely using s/*compile-asserts* etc
2017:05:09 09:11:00 mpenet ok, so lets say it's implemented without pre/post 🙂, just with s/assert
2017:05:09 09:11:23 alexmiller re ordering, right now you create the specs before the functions or after the functions
2017:05:09 09:11:34 alexmiller by having defn rely on the spec, you’ve changed that
2017:05:09 09:11:56 alexmiller if you’re compiling
2017:05:09 09:13:01 mpenet it would be the same as the above , you can just def ::foo-like after as well
2017:05:09 09:13:07 mpenet or I am missing something
2017:05:09 09:13:19 alexmiller I don’t know, maybe that would work
2017:05:09 09:13:52 alexmiller aside: the ::foo-like there would be `foo
2017:05:09 09:14:20 mpenet I don't follow
2017:05:09 09:14:31 alexmiller function specs are named by symbol, not keyword
2017:05:09 09:14:41 mpenet right!
2017:05:09 09:14:43 mpenet missed that one
2017:05:09 09:15:46 alexmiller generally, we don’t think you should be doing this though
2017:05:09 09:17:54 mpenet that would be handy to be able to reuse specs defined for instrument on live (staging/test) systems without paying the gen cost when that matters
2017:05:09 09:18:33 alexmiller there is no “gen cost”?
2017:05:09 09:18:55 mpenet hmm if I recall instrument does trigger gen in some cases
2017:05:09 09:19:09 alexmiller the only case like this is with fspec
2017:05:09 09:20:11 alexmiller if that’s a problem for you, then don’t use fspec
2017:05:09 09:20:22 alexmiller and just spec as ifn?
2017:05:09 09:21:17 mpenet well ... first class functions are quite common in clj code, but yes that's an option
2017:05:09 11:23:27 lambder hello all
I got the following problem:
2017:05:09 11:27:13 dergutemoritz @lambder That recursion will never terminate because (s/? ::s) may match nothing so once the two "abc" strings are matched, it will recur forever on the remaining sequence []
2017:05:09 11:32:01 lambder @dergutemoritz how do I much the following grammar then?
S -> T R | R
T -> :tag
R -> :foo | :foo R
2017:05:09 11:41:25 dergutemoritz @lambder That | in there is probably s/or
2017:05:09 11:41:51 dergutemoritz Err sorry, it's s/alt in regex
2017:05:09 11:42:27 dergutemoritz Everything else is s/cat
2017:05:09 11:42:46 dergutemoritz Just an educated guess, of course, not sure what grammar notation you're using exactly there
2017:05:09 11:43:11 lambder BNF
2017:05:09 11:43:23 dergutemoritz OK then yeah, that should work#
2017:05:09 11:43:53 dergutemoritz Hm well it's not really BNF though, that doesn't use -> at least
2017:05:09 11:44:11 dergutemoritz Don't have an SO account
2017:05:09 11:45:25 lambder ok
2017:05:09 11:46:14 lambder @dergutemoritz many thanks
2017:05:09 11:46:40 dergutemoritz YW, hope it works out!
2017:05:09 12:05:28 vandr0iy suppose I got a spec, (s/def ::foo (s/keys :req-un [::a ::b] :opt-un [::c ::d])) and a data structure, (def bar {:a 1 :c 3 :d 4 :e 5}). How do I remove from the bar all the key-value pairs not explicitly mentioned in the spec and fill in with nils the data that's missing (so, :b nil and remove :e)?
2017:05:09 12:24:46 alexmiller spec doesn't do that
2017:05:09 12:46:07 lambder is it possible to preserve attached meta to structures which are spec conformed?
2017:05:09 15:51:26 lambder Is it possible to ‘tap’ into a protocol method. I’d like to have ‘around’ wrapper for all such method calls.
e.g there is a protocol clojure.spec.Spec which has conform* method defined.
Some of the conform* implementations call conform* recursively. I’d like to wrap my logic around each such call.
2017:05:09 15:51:56 lambder I want to inject some code allowing to preserve meta of the parsed data-structures
2017:05:09 15:53:29 lambder but the call to conform is sometimes omitted as conform* is called instead
2017:05:09 16:01:16 alexmiller in short, no and anything touching these impl details stands a good chance of being broken later
2017:05:09 16:02:35 lambder @alexmiller yes, i think so too. Any idea why conform* implementations aren’t calling conform? performance?
2017:05:09 16:42:26 dergutemoritz @lambder Err I just realized that the grammar you pasted is not equivalent to what you were trying to express in spec - according to that, :tag is only allowed as the first element of the whole sequence and then only :foos are allowed (because R recurs on itself, not on S). Is that what you intended?
2017:05:09 16:43:12 lambder not really. the grammar was simplification of what i needed
2017:05:09 16:48:56 dergutemoritz @lambder AFAIUI that's more complicated than necessary. Doesn't something like (s/def ::g (s/+ (s/cat :tag (s/? ::tag) :val (s/alt :number ::n :string ::s)))) capture what you want to do much neater + give you a more convenient result structure to work with?
2017:05:09 16:49:40 lambder possibly
2017:05:09 16:49:52 dergutemoritz Might be misinterpreting your use case, of course. Anyhow, gotta run. Cheers
2017:05:09 16:59:37 lambder cheers
2017:05:09 21:18:38 danielcompton Is there a way to define newtype's in spec? e.g. specialise a Long to an AccountId or a ProductId. I'm pretty sure there's not, but thought I'd ask in case.
2017:05:09 22:16:58 ghadi say wha?
2017:05:09 22:17:29 ghadi @danielcompton seems like the spec name itself is that information
2017:05:09 22:19:31 danielcompton Let's say I have a function that gets a product (get-product product-id). If the product-id and account-id are both Longs, nothing stops me from passing it (get-product account-id)
2017:05:09 22:52:47 caio I solve this by creating :account/minimal as (s/keys :req [:account/id] :opt [ all the other stuff]) when really necessary
2017:05:09 22:53:41 danielcompton but if you ever need to extract and pass around :account/id by itself, then you can't guarantee that that's what the functions are taking?
2017:05:09 23:40:23 caio I don't think so. specs are about contracts, not strict types
2017:05:09 23:42:37 caio if you want to be this strict, define :account/id as (s/tuple [#(= % :id/account) int?]) or something like that (maybe multi-spec?) and adapt inside the fns that use the id
2017:05:10 03:50:03 alexmiller in short, no you can’t do this
2017:05:10 05:54:38 olivergeorge @gfredericks came across your defn+spec gist (dated 11m ago). Did it ever evolve? I know I'm not meant to want it but seems like a handy way to get type info into my code.
https://gist.github.com/gfredericks/e4a7eafe5dcf1f4feb21ebbc04b6f302
2017:05:10 13:32:19 gfredericks I don't think so; but if there's nothing weird about it I could release it in schpec if that would help
2017:05:10 13:40:29 olivergeorge Thanks. I'm interested for use with CLJS.
2017:05:10 16:32:17 gfredericks @olivergeorge k I'll let you know when I get that pushed out
2017:05:10 19:23:13 john Is there a good example of a project with good "test coverage" that leverages spec?
2017:05:11 20:19:26 xiongtx rmuslimov: Seems like you’re missing a few colons in s/keys.
2017:05:11 20:21:07 rmuslimov yep, fixed. but question remain the same. Why spec is longer than data? is it supposed to be like that? Or it’s my un-optimized solution?
2017:05:11 21:13:01 alexmiller it’s supposed to be like that.
2017:05:11 21:27:46 john Would it be dangerous for a user to create a macro for that? like (sdef-each seq-of-keys string?)
2017:05:11 21:30:54 alexmiller no, macro as you like
2017:05:11 21:32:21 john rgr. so, @U0AR40HJ6, it's possible, bit it's definitely not a newbie/beginner level answer, if you're new to the language in general.
2017:05:11 21:33:06 rmuslimov @U050PJ2EU I can do a macro, was just wondering if there exist any proper way to that. thanks!
2017:05:11 21:34:54 john No prob. Might want to consider using the long form though, from a user/documentation perspective.
2017:05:11 21:36:20 john If core comes out with such a function in the future, then you can rely on your downstream users to know what you're talking about (if that's a factor for you)
2017:05:11 21:36:37 alexmiller not going to add that to core
2017:05:11 21:37:44 john right, so if you are making a library you expect to be consumed by others, remember that spec is also intended to be used for documentation.
2017:05:11 21:38:55 alexmiller if you’re just wrapping calls to s/def, I don’t see how that matters
2017:05:11 21:39:20 alexmiller there will be no difference between doing it explicitly or doing it in a macro?
2017:05:11 21:39:46 alexmiller the macro would presumably just unroll to the same set of calls
2017:05:11 21:40:05 john @U064X3EF3 so any documentation functions that operate on sources will probably operate post macro-expansion?
2017:05:11 21:40:33 john (Or whatever documentation features are expected to work with spec in the future)
2017:05:11 21:40:37 alexmiller doc functions don’t use the sources for specs
2017:05:11 21:40:43 john rgr
2017:05:11 21:40:50 alexmiller docs look up the specs in the registry
2017:05:11 21:40:51 john macro away then 🙂
2017:05:11 21:47:43 caio Why not a vec ::emails with 5 string elements? 🤔
2017:05:11 21:08:31 mobileink not that i know of. you have an idea?
2017:05:11 21:09:41 mobileink e.g. (s/def* string? [email1 email2 ... ]),
2017:05:12 00:48:58 tbaldridge @rmuslimov perhaps structure it as a list of strings? That approach is simpler for the programmer, and easier to maintain
2017:05:12 00:52:46 bbrinck If my code accepts JS values at the boundary of the system, what’s the preferred way to use spec to validate it? AIUI, s/keys only works with keywords. Is it common practice to convert all string keys to keyword keys when converting to CLJS in order to use spec? e.g. (js->clj input :keywordize-keys true)
2017:05:12 00:59:15 rmuslimov @tbaldridge yes it is dead simple, however spec is longer that data it describes. I was wondering if I’m missing some nice shortcut macro for that
2017:05:12 03:04:37 tbaldridge @rmuslimov no, I'm saying that this is the better approach:
2017:05:12 03:06:06 rmuslimov @tbaldridge it wasn’t real world example, I just created the most trivial example of the problem. Real world example would be something like:
2017:05:12 03:27:28 olivergeorge @gfredericks Thinking more about defn+spec made me explore why i wanted it and alternative ways to get the the same result.
2017:05:12 03:28:02 olivergeorge One key thing spec gives me is protection from "garbage in garbage out" errors while I'm hacking up new features. Colocating the spec with the defn is helpful during this phase because I'm refactoring heavily as I go - if the s/fdef specs were in another file they're more likely to get stale.
2017:05:12 03:28:05 olivergeorge I can do this by adding s/assert statements in my defn.
2017:05:12 03:28:37 olivergeorge Later when code stabilises I see the logic of specs living in a different namespace so they are out of the way but still available to reference.
2017:05:12 03:28:48 olivergeorge As an aside, IDE support to allow hiding of spec statements would make that less important. #cursive
2017:05:12 03:29:17 olivergeorge But I appreciate the idea that separate namespaces for specs also allow for different specs to be applied to the same code depending on need.
2017:05:12 03:30:07 olivergeorge @gfredericks defn+spec would benefit from the s/assert flag so that it can have no impact on prod performance.
2017:05:12 03:33:26 tbaldridge @rmuslimov that's covered in some of the spec docs
2017:05:12 04:47:18 rmuslimov I did actually, as far as I can see: only s/keys is given for building spec for maps. I cannot do something close to way I proposed. Developer should always define explicitly keys like (s/def ::total ::money) and then build spec for map with s/keys.
2017:05:12 12:52:10 alexmiller It's quite intentional that information maps are built as sets of attributes. This is a key design principle in spec - the semantics belong to the attribute, not to the map.
2017:05:13 02:39:01 danielcompton Will there also be a way to provide a spec along with a function? e.g. (s/defn ...) or (defn my-fn [args] {:args (s/cat ...)} ... or something similar?
2017:05:13 02:39:14 danielcompton If I do want to define my functions with my spec?
2017:05:13 02:47:47 lincpa I think proactive data standardization work is better than passive data validation. Industrial assembly line recommended using standardized materials.
2017:05:13 02:51:30 lincpa Standardize input and output data at the beginning and end of data processing
2017:05:13 02:57:17 lincpa Proactively standardize non-standard data, and if fail do exception handling.
2017:05:13 05:00:25 alexmiller @danielcompton no, there will not be a way to combine function definition with spec
2017:05:13 09:26:04 misha are there any builtin predicates for queues in clj/cljs?
2017:05:13 15:02:36 john @misha What do you mean? like qeueu? Don't believe so. If you mean query for type, you could use (instance? cljs.core/PersistentQueue #queue [])
2017:05:13 15:32:53 alexmiller @misha no
2017:05:13 15:32:55 misha @john yes, sort of, with a generator included, please
2017:05:13 15:34:07 alexmiller You might be able to spec it with coll-of and some of the kind and into options, but I haven't tried
2017:05:13 15:35:07 misha @alexmiller, what is a correct way to define spec for dynamic function (like reducing function)?
I end up with following, is that legal?
(s/def ::reducing-fn
(s/fspec :args (s/cat ...))
2017:05:13 15:35:41 alexmiller Sure?
2017:05:13 15:35:55 alexmiller What is dynamic about that?
2017:05:13 15:38:47 misha AFAIR, s/fdef accepts qualified fn symbol, with intention to use it on a function with that exact name.
There is an f(g) where g - is a function. I need to: 1) spec g as an argument, 2) re-use that spec.
(s/def :foo (s/fspec ...)) seems to work, but I expect it to bite me in some way later.
2017:05:13 16:18:16 misha "how to get named spec for anonymous fn", there.
2017:05:13 18:31:05 alexmiller That should work
2017:05:14 09:24:25 thedavidmeister anyone else seen java.lang.RuntimeException: Unable to resolve symbol: qualified-keyword? in this context, compiling:(clojure/spec/gen/alpha.clj:131:4) with clojure version 1.9.0-alpha16?
2017:05:14 13:57:15 weavejester What’s the value of *clojure-version*, @thedavidmeister?
2017:05:14 18:41:11 alexmiller @thedavidmeister no but sounds like you are using spec.alpha with Clojure < 1.9
2017:05:14 23:09:47 thedavidmeister @alexmiller @weavejester yeah that was it, boot had its own version of clojure messing with things
2017:05:14 23:09:48 thedavidmeister thanks
2017:05:15 09:26:49 danielneal I'm trying to take a reagent vector, conform it, make a modification and unform it back.
2017:05:15 09:40:40 danielneal is that appropriate/idiomatic do you know?
2017:05:15 09:41:16 dergutemoritz I guess that's an innocent enough use of s/conformer, yeah 🙂
2017:05:15 09:41:36 dergutemoritz Maybe we should call this kind of use "vegan" as opposed to the "meat grinder" pattern
2017:05:15 09:44:16 danielneal 😂
2017:05:15 09:45:34 dergutemoritz "No animals were harmed in this use of s/conformer"
2017:05:15 14:08:33 alexmiller the need for regex ops in a vector is a known issue - it’s pretty challenging right now to get conforming, validation, unforming, generation, and describing to work properly together
2017:05:15 21:57:43 zane What's the right way to attach a generator to a value produced by s/conformer?
2017:05:15 21:58:25 alexmiller s/with-gen
2017:05:15 22:00:30 zane Right, but how?
2017:05:15 22:00:35 zane Does it need to be attached to a keyword?
2017:05:15 23:56:57 alexmiller (s/def ::example #{:some :specific :values "some" "specific" " "values"}) would work :)
2017:05:15 23:57:26 alexmiller What do you actually need to spec?
2017:05:16 00:30:27 zane That the incoming values will be in that set when conformed, and that strings will be conformed to keywords.
2017:05:16 00:31:03 zane I'm trying to make my example more DRY.
2017:05:16 00:45:45 alexmiller (def vs #{:some :specific :values})
(s/def ::example (into vs (map name) vs))
2017:05:16 00:48:33 alexmiller That doesn't conform to keywords but transformation is not really the point of spec
2017:05:16 06:33:59 odinodin does anyone know of any efforts that present clojure spec validation errors in a nice way? I’m looking for ideas and inspiration for a web based error message component I’m working on
2017:05:16 07:07:03 odinodin @rmuslimov thanks. I was thinking more about how to present raw clojure.spec validation errors in a nicer way during development
2017:05:16 07:09:24 rmuslimov ah ok, interesting. Not sure how standard can be improved, may be you already have any ideas - how it may look like?
2017:05:16 07:16:16 odinodin Just started digging, not sure where it will lead yet
2017:05:16 11:10:20 urbank why has cljs.spec been renamed to cljs.spec.alpha?
2017:05:16 13:14:47 curlyfry Any news on https://dev.clojure.org/jira/browse/CLJ-2123 or something similar? Having some issues right now with name clashes on specs (that we still want in the same file). We're specing a re-frame db and want to keep the specs for a specific page in the same file. Often parts of specs for input field and similar end up having the same names as other parts of the state. The solution right now is either to create an artificial and verbose namespaced keyword (:my-page.input.domain/id) or to add a new file just for the clashing specs.
2017:05:16 13:57:28 stathissideris @curlyfry in a similar vein: when you refactor clashing keywords to have different namespaces, does the rename mean that you get rippling changes throughout your code?
2017:05:16 14:23:07 pseud I've got a map where all entries should follow some template (e.g. (s/map-of string? number?) except one optional key, let's call it :foo.
I tried something like s/merge, but it seems like it was designed with multiple s/keys specs in mind.
(s/merge (s/map-of string? number?) (s/keys :opt-un [::foo])).
I can of course write my own predicate function to split the map in two and check against each spec with s/valid?, but I'll lose a lot of contextual information in case of failures.
Is there no better way ?
2017:05:16 15:43:04 alexmiller @curlyfry no update
2017:05:16 15:44:13 alexmiller yours is a bit simpler than the one described there
2017:05:16 16:59:15 pseud @alexmiller cool, a bit too tired to digest, but bookmarked.
2017:05:16 18:35:42 grierson Hey, does (s/exercise) work with CLJS?
2017:05:16 18:54:48 alexmiller afaik
2017:05:16 19:06:25 curlyfry @grierson Yup!
2017:05:16 19:19:17 grierson do I need to import anything else apart from cljs.spec?
2017:05:16 19:28:06 grierson Fixed it. Need to import [clojure.test.check.generators]. I was only using Spacemacs inline eval I wasn’t looking at the REPL.
2017:05:17 14:29:04 chillenious is there a way to in-line this:
(s/def ::a integer?)
(s/keys :req-un [::a])
so that I can state I expect ::a in the map that should be checked against integer? without having to specify ::a first?
2017:05:17 14:30:55 luxbock @chillenious no, you could write a macro to do something like (my-keys :req-un [(::a integer?)]) that expands to it
2017:05:17 14:31:11 chillenious ok, thx
2017:05:17 14:31:51 moxaj @chillenious that's by design, see https://clojure.org/about/spec - Map specs should be of keysets only
2017:05:17 14:33:02 chillenious Yeah, I was afraid you were going to say that 🙂 I can't be the first one who thinks that that's a PITA though. I very often have the situation where I'd like to use a different key from the 'type'.
2017:05:17 15:23:38 chillenious That looks great, will try it out later this week.
2017:05:17 15:27:07 wilkerlucio @chillenious I encourage you to avoid this, there are many reasons for spec to encourage doing things in the way it does, to encourage you to have value definitions and just make compositions out of it, if you start thinking on your data as records you gonna lose a lot of the benefits on the long run, I recommend watching this presentation from Rich, where he talks a lot about the grand ideas and why things are the way they are: https://vimeo.com/195711510
2017:05:17 15:28:59 chillenious I'll look at it, thanks. I'm using this as part of test code where I want some validation and documentation as to what data structures can/ should be used. So it's as important to me that it is easy to read and terse, and doesn't have to be a perfect water tight solution for all things spec 🙂
2017:05:17 15:19:44 wilkerlucio hello 🙂
2017:05:17 15:20:05 wilkerlucio I noticed when a spec is created with (s/with-gen) it loses track of the original form:
2017:05:17 19:07:40 gphilipp There was a question last year about speccing a binary tree:
(def BinaryTree
(maybe ;; any empty binary tree is represented by nil
{:value long
:left (recursive #‘BinaryTree)
:right (recursive #‘BinaryTree)}))
2017:05:17 19:08:32 gphilipp @alexmiller: you told Andrey Grin :“Yes, you can create recursive definitions by registering a spec that refers to itself via registered name (a namespaced keyword).“. I can’t find a way to do it, could please post a snippet of code ?
2017:05:17 19:32:35 wilkerlucio @alexmiller I was having this idea yesterday, about spec hierarchy tracking, currently when you define a spec from another (eg: (s/def ::something ::other-thing)) the resolving happens on definition time, I understand that is preferable for performance reasons, but doing that makes us lose track of the hierarchy. I was implementing a coerce engine on top of spec, and I miss having this hierarchy information, so I was thinking that we could have Clojure hierarchy for the specs, so when defining those we do a derive on those keywords, so we would have this information aside available for use, do you think this is a reasonable feature to have on spec?
2017:05:17 19:36:05 alexmiller I don’t know about all that. there are some open tickets about the resolution aspects that I expect will be addressed.
2017:05:17 19:36:28 alexmiller it is not the intent that you lose that tracking now
2017:05:17 19:37:02 wilkerlucio for example, given the following specs:
2017:05:17 19:37:17 wilkerlucio
(s/def ::a int?)
(s/def ::b ::a)
(s/def ::c ::b)
2017:05:17 19:37:40 wilkerlucio is there a way, starting from ::c to know that it is derived from ::b and ::a?
2017:05:17 19:38:31 alexmiller (s/get-spec ::c)
2017:05:17 19:38:48 alexmiller should tell you ::b
2017:05:17 19:41:40 wilkerlucio interesting, I didn't knew that get-spec would do that, and testing with the previous example works
2017:05:17 19:42:23 wilkerlucio glad to hear that my assumptions were wrong on this
2017:05:17 19:42:33 wilkerlucio thanks Alex
2017:05:17 22:16:39 lvh how do I make a spec refer to a spec in another ns? I did (s/fdef ::name ::storage/name) but I just get No value supplied for key: ::storage/name
2017:05:17 22:23:18 lvh never mind, I’m an idiot and meant s/def
2017:05:17 22:38:18 gphilipp Thx @misha. Would it be possible to have a fn to create that kind of spec with the 'number?' predicate being passed as a parameter to this fn ?
2017:05:18 07:19:46 misha @gphilipp I'd think macro is doable, if you really need to have several BTs. However I'd start with (s/or ...) with all the variants you want in it, and see if that's enough
2017:05:18 14:14:41 wilkerlucio Hello, I'm trying to use the overrides argument on a generator, but I'm not sure how I supposed to use it
2017:05:18 14:14:45 wilkerlucio I'm trying like this:
2017:05:18 14:15:42 wilkerlucio but that doesn't work, does someone have an example on how to use it?
2017:05:18 14:25:28 alexmiller that looks roughly correct, maybe share more of the example?
2017:05:18 18:07:23 wilkerlucio @alexmiller a more complete example here:
2017:05:18 18:08:35 wilkerlucio sorry the noise now, I was looking at the wrong number, A is always even here, saying it's correct, I'll check my other example again
2017:05:18 18:12:52 wilkerlucio @alexmiller ok, I think I might have found a bug
2017:05:18 18:13:15 wilkerlucio it works fine when the spec is not defined in terms of other spec
2017:05:18 18:13:22 wilkerlucio here is an example where the override doesn't work
2017:05:18 18:15:05 wilkerlucio if ::a was defined as int? it works, but when making it reference other spec, the override version is not used
2017:05:18 21:41:56 alexmiller Oh, yes there is a ticket for this
2017:05:19 00:45:46 cfleming I’m playing around with the specs for spec from CLJ-2112, and I’m having some problems with it.
2017:05:19 00:46:07 cfleming
(s/conform ::spec (s/form 'clojure.core/let))
IllegalArgumentException Don't know how to create ISeq from: clojure.lang.Keyword clojure.lang.RT.seqFrom (RT.java:547)
2017:05:19 00:46:51 cfleming When I try to investigate further, things get strange:
2017:05:19 00:47:57 cfleming Am I missing something obvious?
2017:05:19 00:49:55 cfleming I understand that the specs there are half-baked, but I didn’t expect to get exceptions back.
2017:05:19 00:50:57 cfleming This is with alpha16, in case that matters.
2017:05:19 03:01:03 cfleming Hmm, clearly I should be quoting the form I’m passing in, but then I just get back to the IllegalArgumentException
2017:05:19 03:12:02 cfleming Actually, this might be CLJ-2152
2017:05:19 04:11:43 alexmiller at the beginning when you do this: (s/form 'clojure.core/let) - that returns the fspec for let, not sure if you wanted that or the args spec which would be (s/form (:args (s/get-spec 'clojure.core/let)))
2017:05:19 04:13:03 alexmiller when you do (s/conform ::spec (clojure.spec.alpha/cat :name ... )), you’ll need to quote there as the spec is expecting a sequential thing
2017:05:19 04:13:48 cfleming Right, but when I quote then I get the exception.
2017:05:19 04:14:19 cfleming One sec, I got sidetracked, starting my REPL again…
2017:05:19 04:14:39 alexmiller I’m actually heading to bed but could load it up tomorrow and look at it
2017:05:19 04:15:26 cfleming
(s/conform ::spec (s/form (:args (s/get-spec 'clojure.core/let))))
IllegalArgumentException Don't know how to create ISeq from: clojure.lang.Keyword clojure.lang.RT.seqFrom (RT.java:547)
2017:05:19 04:15:39 alexmiller can you (pst *e) ?
2017:05:19 04:17:49 cfleming
java.lang.IllegalArgumentException: Don't know how to create ISeq from: clojure.lang.Keyword
at clojure.lang.RT.seqFrom(RT.java:547)
at clojure.lang.RT.seq(RT.java:527)
at clojure.lang.RT.first(RT.java:683)
at clojure.core$first__6389.invokeStatic(core.clj:55)
at clojure.core$first__6389.invoke(core.clj:55)
at clojure.spec.alpha$multi_spec_impl$predx__912.invoke(alpha.clj:904)
at clojure.spec.alpha$multi_spec_impl$reify__919.conform_STAR_(alpha.clj:916)
at clojure.spec.alpha$or_spec_impl$fn__958.invoke(alpha.clj:1035)
at clojure.spec.alpha$or_spec_impl$reify__963.conform_STAR_(alpha.clj:1057)
at clojure.spec.alpha$conform.invokeStatic(alpha.clj:150)
at clojure.spec.alpha$conform.invoke(alpha.clj:146)
at clojure.spec.alpha$dt.invokeStatic(alpha.clj:748)
at clojure.spec.alpha$dt.invoke(alpha.clj:743)
at clojure.spec.alpha$dt.invokeStatic(alpha.clj:744)
at clojure.spec.alpha$dt.invoke(alpha.clj:743)
at clojure.spec.alpha$deriv.invokeStatic(alpha.clj:1475)
at clojure.spec.alpha$deriv.invoke(alpha.clj:1469)
at clojure.spec.alpha$deriv.invokeStatic(alpha.clj:1483)
at clojure.spec.alpha$deriv.invoke(alpha.clj:1469)
at clojure.spec.alpha$deriv.invokeStatic(alpha.clj:1486)
at clojure.spec.alpha$deriv.invoke(alpha.clj:1469)
at clojure.spec.alpha$deriv.invokeStatic(alpha.clj:1483)
at clojure.spec.alpha$deriv.invoke(alpha.clj:1469)
at clojure.spec.alpha$re_conform.invokeStatic(alpha.clj:1610)
at clojure.spec.alpha$re_conform.invoke(alpha.clj:1601)
at clojure.spec.alpha$regex_spec_impl$reify__1340.conform_STAR_(alpha.clj:1651)
at clojure.spec.alpha$conform.invokeStatic(alpha.clj:150)
at clojure.spec.alpha$conform.invoke(alpha.clj:146)
at clojure.spec.alpha$dt.invokeStatic(alpha.clj:748)
at clojure.spec.alpha$dt.invoke(alpha.clj:743)
at clojure.spec.alpha$dt.invokeStatic(alpha.clj:744)
at clojure.spec.alpha$dt.invoke(alpha.clj:743)
at clojure.spec.alpha$multi_spec_impl$reify__919.conform_STAR_(alpha.clj:917)
at clojure.spec.alpha$or_spec_impl$fn__958.invoke(alpha.clj:1035)
at clojure.spec.alpha$or_spec_impl$reify__963.conform_STAR_(alpha.clj:1057)
at clojure.spec.alpha$conform.invokeStatic(alpha.clj:150)
at clojure.spec.alpha$conform.invoke(alpha.clj:146)
at clojure.spec.specs$eval3133.invokeStatic(form-init2127301951087080660.clj:1)
at clojure.spec.specs$eval3133.invoke(form-init2127301951087080660.clj:1)
at clojure.lang.Compiler.eval(Compiler.java:6977)
2017:05:19 04:19:06 cfleming I suspect this is related to CLJ-2152, Metosin in a blog post said that their spec walker/visitor didn’t work with s/& or s/keys* because of it.
2017:05:19 04:19:20 cfleming But I may be off there.
2017:05:19 04:19:56 cfleming From your test cases in that patch, I get this:
(s/conform ::spec (s/form (s/keys* :req [::foo])))
IllegalArgumentException Don't know how to create ISeq from: clojure.lang.Keyword clojure.lang.RT.seqFrom (RT.java:547)
2017:05:19 04:20:40 alexmiller that code had a number of broken cases due to form - we’ve fixed most of them since but there are a few known problems still
2017:05:19 04:20:49 alexmiller s/&, s/keys* in particular
2017:05:19 04:21:37 alexmiller I guess some of the nested specs in let bindings could be getting tripped up
2017:05:19 04:21:45 cfleming And I guess s/cat uses s/keys* under the hood?
2017:05:19 04:22:06 cfleming I’ll check that.
2017:05:19 04:22:27 alexmiller nah
2017:05:19 04:23:03 alexmiller but the specs for s/cat could use it
2017:05:19 04:23:58 alexmiller anyhow, that ticket is incomplete because it’s a wip and there are many things that don’t work with it yet so it doesn’t surprise me that things don’t work
2017:05:19 04:24:19 alexmiller rather than just having it sit on my hard drive I figured it was better to be a wip on a ticket
2017:05:19 04:24:43 cfleming Sure, is there a workaround I could use? It doesn’t seem like a problem with the specs themselves, but more likely with form
2017:05:19 04:25:08 alexmiller I don’t think it is - it looks like (s/form (s/keys* :req [::foo])) returns the right thing
2017:05:19 04:25:16 alexmiller sorry, grabbed wrong code there
2017:05:19 04:25:36 cfleming I was going to say - pretty sure an exception isn’t the right thing 🙂
2017:05:19 04:26:01 alexmiller I get (clojure.spec.alpha/cat :bindings :clojure.core.specs.alpha/bindings :body (clojure.spec.alpha/* clojure.core/any?)) for that, which seems good
2017:05:19 04:26:28 alexmiller the spec specs should only need to have working s/cat and s/* for that to conform
2017:05:19 04:27:12 cfleming I get the same, but the conform fails with that exception.
2017:05:19 04:28:22 alexmiller well, I’m too wiped to take it any further atm
2017:05:19 04:28:29 alexmiller will take a glance tomorrow
2017:05:19 04:28:35 cfleming It seems like s/& is the root cause - if I macroexpand (s/keys* :req [::foo]) I get:
2017:05:19 04:31:49 alexmiller I probably just didn’t test that case
2017:05:19 04:32:01 alexmiller I was spending most of my time fixing s/form bugs :)
2017:05:19 04:32:10 alexmiller anyhow, good night…
2017:05:19 04:32:12 alexmiller for real!
2017:05:19 04:32:35 cfleming Looks good - thanks!
2017:05:19 07:49:31 seantempesta Probably a dumb question, but when I use s/describe I’m getting back a list in the form of (keys :req [:person/first-name :person/last-name] :opt [:person/contact]). Is there an easy way to convert this to a map? Is it hacky to use n’th to access the required and optional fields?
2017:05:19 12:37:29 pseud @seantempesta I really wouldn't worry about that. If you write a single function to handle the extraction of what you want into a data structure suitable to your needs you only have a single function to refactor should the world change...
2017:05:19 12:37:41 alexmiller Well the prior conversation here is about specs for specs which allow you to conform a spec form into a map
2017:05:19 12:37:47 alexmiller But still a wip
2017:05:19 19:06:52 arohner this looks like a bug to me:
2017:05:19 19:07:37 arohner 1) define an s/keys, without defining :bogus/bogus. There’s no warning/error that :bogus/bogus is undefined, and the spec conforms
2017:05:19 20:41:05 alexmiller agreed, file a jira
2017:05:19 20:42:12 alexmiller although as I try other things, this is also the behavior with :req
2017:05:19 20:42:14 misha how is this bug, when key specs are opt in?
2017:05:19 20:43:06 misha and s/keys only specifies keys set. why would it complain?
2017:05:19 20:43:18 alexmiller “The validator will ensure the :req keys are present.”
2017:05:19 20:44:00 alexmiller from the s/keys docstring
2017:05:19 20:44:25 misha but :bogus is present
2017:05:19 20:44:47 alexmiller I’m agreeing with you
2017:05:19 20:45:27 alexmiller “keys … are required, and will be validated and generated by specs (if they exist)”
2017:05:19 20:45:44 alexmiller so, I’ve changed my mind - I think it’s doing what it says will do
2017:05:19 20:45:56 misha
2017:05:19 20:48:28 misha for a minute I was questioning 2/3 of specs I wrote
2017:05:19 23:34:19 jfntn What would be a good way of and-ing multiple specs without the conforming behavior of s/and?
2017:05:19 23:58:15 seantempesta Good point. Thanks!
2017:05:20 19:05:15 borkdude Is it possible to express the following in spec: f accepts a function g from string? to int? and it is also possible to generate random instances of g?
2017:05:20 19:07:11 borkdude I wondered about this after playing with Quickcheck in Haskell, which can do this
2017:05:20 19:47:58 borkdude @luxbock Looks good. How about the generational aspect?
2017:05:20 20:40:22 alexmiller Gen of fspec is a function that uses its ret generator
2017:05:20 20:46:56 stathissideris How would I register (and then use) specs that were generated dynamically? I think I’m at the stage where I need to start property testing spec-provider. To do this I need to generate random data (possible), infer specs for them (that’s what the lib does) and then validate that all the random data are valid for those inferred specs.
2017:05:20 22:22:45 alexmiller Well you can either call s/def and register your specs or you can just call conform on the specs directly without registering, which works for a lot of stuff
2017:05:21 08:47:16 stathissideris @alexmiller thanks. But I’m generating the forms of the specs, so I guess I’d have to eval them in any case, right?
2017:05:21 12:49:34 alexmiller Yes
2017:05:21 18:44:02 stathissideris @alexmiller great, makes sense, thanks
2017:05:22 19:07:54 mobileink i have a bunch of map specs, each respresenting a "resource type". i want to present them on a web page. is s/form the way to go? is there a library that makes it easy to get and send a json/edn respresentation of a map spec from the registry?
2017:05:22 21:30:09 alexmiller s/form will be fully resolved, s/describe will be more terse
2017:05:22 21:30:39 alexmiller Stuff in there should be edn afaik?
2017:05:23 04:16:18 danielcompton What is the right way to spec functions that can throw exceptions? It seems like I need to allow :ret to be nil to handle exceptions here
2017:05:23 16:11:51 alexmiller If there is an exception there is no return value
2017:05:23 16:12:14 alexmiller Exceptions are not covered by spec
2017:05:23 16:12:36 alexmiller Because they are ... exceptional cases
2017:05:23 17:45:26 deplect hai all, how do we handle ordering with spec?
2017:05:23 17:57:44 alexmiller you’ll need to explain more, not sure what you mean
2017:05:23 18:06:31 caio (s/and sorted? ...)?
2017:05:23 18:07:28 caio (not the clojure.core/sorted?)
2017:05:23 18:26:44 andrewmcveigh Is this expected?
2017:05:23 18:34:24 andrewmcveigh I just wonder if this is a bug, or if the s/every macro is meant to check the 'n symbol rather than what it evals to.
2017:05:23 19:19:03 alexmiller I think I’d call it a bug, given the docstring states that nil is the default which seems to imply you could pass it
2017:05:23 19:19:07 alexmiller feel free to file a jira
2017:05:23 19:53:49 andrewmcveigh Ok, will do. Thanks
2017:05:24 15:28:20 weavejester Should instrument report the name of the function that failed in the ex-data?
2017:05:24 15:29:03 weavejester It reports the function name in the exception message, but it’s missing from the exception’s data.
2017:05:24 15:37:07 alexmiller most likely yes
2017:05:24 15:37:16 alexmiller example would help, but feel free to file something about itk
2017:05:24 15:43:03 alexmiller @weavejester just looked at an example - definitely yes
2017:05:24 15:43:14 weavejester Ah, I was just about to post one 🙂
2017:05:24 15:43:30 weavejester Shall I add a JIRA report, @alexmiller ?
2017:05:24 15:47:08 alexmiller that would be great
2017:05:24 15:47:19 alexmiller if you patch it, I will screen it too :)
2017:05:24 15:48:11 alexmiller btw, enjoyed your defn podcast episode
2017:05:24 16:06:19 alexmiller I assume this is obvious, but patch will be against the spec.alpha lib, not core clojure repo
2017:05:24 16:18:41 weavejester Sorry, I couldn’t find the spec.alpha lib in the project list when I created the issue.
2017:05:24 16:39:48 stathissideris is there any support for string keys in s/keys? or any plan to support them later?
2017:05:24 17:44:38 ghadi no @stathissideris -- s/keys needs things that are globally registered (keywords/specs)
2017:05:24 17:52:57 stathissideris @ghadi thanks
2017:05:24 18:05:53 alexmiller @weavejester you filed it in the right place - spec issues are being managed in CLJ, just meant that the patch would be on a different repo
2017:05:24 18:06:02 weavejester Ah, cool.
2017:05:24 18:07:05 jfntn What’s an idiomatic way to spec args for a multi-arity function where the longer arities only add new arguments to the shorter ones? Say the full args vector is [a b c ...] but you have aritites with defaults that only take [a], or [a b] etc?
2017:05:24 18:12:16 stathissideris @jfntn I think you either do (s/or :arity1 (s/cat ..) :arity2 (s/cat ...) :arity3 (s/cat)) or (s/cat x y (s/? w z))
2017:05:24 18:12:27 stathissideris not sure which is more idiomatic
2017:05:24 18:12:56 stathissideris actually it’s not s/?
2017:05:24 20:57:56 mattly yeah, the individual specs I haven't had problems with, what I need is to validate that a value in a map conforms to one of possibly 100 specs
2017:05:24 20:59:38 donaldball Well, a way you could do that is to use eval to build the s/or form you want
2017:05:24 21:00:52 mattly hm, thanks
2017:05:24 21:11:19 arohner is there a way to get the default generator, so I can update it?
2017:05:24 21:27:28 alexmiller Yeah, having a shorthand for this case is something I think would be useful
2017:05:24 21:27:42 alexmiller It comes up regularly
2017:05:25 22:21:33 jfntn Are recursive specs not supported in Clojurescript? I have (s/def ::parent ::self) (s/def ::self ...) failing at the first line.
2017:05:25 23:53:19 donaldball Has anyone written a macro to combine defn and s/fdef yet?
2017:05:26 00:15:07 alexmiller @donaldball I think maybe @gfredericks did?
2017:05:26 00:15:19 alexmiller @jfntn not sure
2017:05:26 00:16:26 jfntn @alexmiller the workaround was to move the self spec before the parent
2017:05:26 00:17:42 alexmiller hmm, fails on Clojure too - I think this is related to a couple other existing tickets where s/def chases the spec too early
2017:05:26 00:43:16 gfredericks I wrote a defn macro that dispatches on specs matching; it didn't make an fdef
2017:05:26 16:04:20 wilkerlucio hello people, I wanna share with you a snippet that I wrote here to find bad generators on your project, it's helping me a lot (specially when you have a s/keys filled with items and don't know which of then is not generating)
2017:05:26 17:37:45 bbss let me check that, thanks
2017:05:26 17:38:33 ghadi reloading things will redefine some classes. Those classes will have the same name as existing classes, but will be != from the JVM's perspective
2017:05:26 17:38:56 bbss makes sense
2017:05:26 17:39:09 ghadi thus instances of RoseTree(1) are not instances of RoseTree(2)
2017:05:26 17:41:50 hiredman hmmm?
2017:05:26 17:43:12 hiredman aot drowns kittens in a sack in the river (the kittens are a metaphor for your hopes and deams)
2017:05:26 17:47:55 bbss hmm, now I'm running into issues with clj-time.format not being found.
2017:05:26 17:48:26 bbss Which the .coerce namespace depends on, which is weird because when I used the function before I restarted it worked.
2017:05:26 17:49:05 bbss It's really the only kitten that sometimes needs to claw itself to fresh air once in a while.
2017:05:26 17:49:44 bbss (metaphor for the only thing that sometimes gets to me with cljx)
2017:05:26 18:00:28 ghadi you're using cljx?
2017:05:26 18:00:34 ghadi @bbss
2017:05:26 18:01:10 bbss @ghadi is that not short for clojure(script) ?
2017:05:26 18:01:35 bbss Oh right, it's the pre-conditional reader library
2017:05:26 18:01:38 ghadi there was an earlier library called cljx that did something similar to cljc
2017:05:26 18:01:40 bbss sorry, it's cljc
2017:05:26 18:02:27 bbss I meant to say, the only thing that I find really annoying at times is dependency issues like these. It takes by far the most of my time out of all time-taking things that aren't normal debugging/coding.
2017:05:26 18:42:27 bbss lein clean and lein deps solved it
2017:05:26 18:42:51 bbss thanks for saving my kittens 🙂
2017:05:26 22:22:37 seancorfield For folks doing programmatic decoding of explain-data to produce nice messages, it seems that required keys now (in 0.1.123) produce a :pred of (clojure.core/fn [%] (contains? % :the-key)) whereas they used to produce (contains? % :the-key) — this broke a lot of our decoding logic so I figured I’d mention it here in case it helps other debug similar breakage…
2017:05:26 22:34:28 arohner given an (s/keys :opt-un [::foo]), what is the best way to generate a map with foo always in it? I occasionally see test failures in CI Couldn't satisfy such-that predicate after 10 tries, when I do (gen/such-that :foo (s/gen ::the-map))
2017:05:27 00:37:54 athos Happy to hear a new version of spec.alpha has been released!! I've been waiting for CLJ-2059 and CLJ-2085 to be merged 😆
2017:05:27 01:04:32 seancorfield Yes, 0.1.123 is the correct version.
2017:05:27 01:04:52 seancorfield Alex explained there was a build snafu and they had to re-release 0.1.109 under a new version number.
2017:05:27 01:05:11 seancorfield So it’s technically both 0.1.109 and 0.1.123 🙂
2017:05:27 01:14:54 athos I see 🤔
2017:05:27 01:27:05 alexmiller Oh, I will update the changes file - I forgot to do that
2017:05:27 01:30:33 alexmiller Fixed!
2017:05:27 02:02:51 gfredericks @arohner what about generating it from a spec where that key is required?
2017:05:27 02:22:45 arohner @gfredericks that’s not a terrible idea. I was hesitant because ‘in the real world’, the key is optional, and I only needed it required for a specific test case
2017:05:27 02:23:05 arohner I guess that’s fine. (s/def ::foo...)(s/def ::foo-with-bar...)
2017:05:27 02:23:39 gfredericks @arohner these are unit tests?
2017:05:27 02:23:43 arohner yes
2017:05:27 02:23:53 arohner ::foo is used in production, ::foo-with-bar would only be used in tests
2017:05:28 14:44:28 bbss I'm developing web stuff in cljc with spec and loving it. It's so great to start out with some specs and to generate sample data and build out your front-end from there.
2017:05:28 14:45:05 bbss they can double as react-props with {pre [(valid? spec state)]}
2017:05:28 14:48:30 bbss one thing I'm hitting now is I'd like to generate ids so I can give those as unique react keys, which is necessary when react displays lists. Since I will eventually replace the sample data ids with :db/id from datascript, I figured I can add uuid as spec pred and have that generate unique id's. But that doesn't match what datascript generates for ids.
2017:05:28 14:49:53 bbss those are just ints. Is there a way to generate unique ints?
2017:05:28 15:14:14 bbss I think I'll have to write my own generator for that
2017:05:28 15:15:08 bbss I've tried with s/or, which would work for the validation but it also generates random (non unique) ints. As it should of course 🙂
2017:05:29 12:38:10 tianshu why spec become spec.alpha?
2017:05:29 13:18:20 robert-stuttaford @doglooksgood there’s a post about it in the google group
2017:05:29 14:13:29 tianshu but is there any problems with current version of clojure.spec? any feature is missing or critical bug?
2017:05:29 14:15:48 mpenet it depends
2017:05:29 14:17:00 mpenet I don't think there are "critical" bugs, but there are a few bugs left to fix. missing features: certainly, but that's quite subjective, depends on what you need
2017:05:29 17:07:06 bbss @wilkerlucio thank you, missed that from the docs!
2017:05:30 12:18:45 gardnervickers I have a (s/cat ...) spec form that I am using for parsing a seq with s/conform. I want to make this extensible through a multi-spec. Is there a regex op that can help me here? Perhaps using the dispatch key on the multi-spec to tag conformed values?
2017:05:30 13:27:56 misha is it ok to actually rely on order of s/or sub-specs?
2017:05:30 16:19:25 don.dwoske @stuartsierra I would love to see the clojure.spec definitions used in your recent article : http://blog.cognitect.com/blog/2017/4/6/developing-the-language-of-the-domain We are going through a very similar process now. Your clojure spec and tools, web app to visually explore the models, et. al. are things I wish I had right now. The business analysts are trying to use Avro specifications to describe the domain model, and are immediately hitting limitations of what can be expressed. I am developing a clojure DSL similar to the one in your article to describe our domain, which I can propose as an alternative. Getting my hands on anything you can share would give me a great jumpstart.
2017:05:30 17:34:41 stuartsierra @don.dwoske I'm sorry, I do not have permission to share any of that work beyond the examples in the article.
2017:05:30 17:36:10 don.dwoske @stuartsierra Sorry to hear that. I enjoyed the article nonetheless, it gave me some good ideas.
2017:05:30 17:38:38 stuartsierra thanks, glad it helped
2017:05:31 13:43:46 misha is there a way to highjack a "bottom" component of composed spec to avoid manually writing 2 sets of specs?
use case would be: a) spec for datomic entity with both temp and actual ids in refs; b) and the "same" spec, but with actual ids only?
(s/def :datomic/temp-id neg-int?)
(s/def :datomic/actual-id pos-int?)
(s/def :datomic/id (s/or
:actual-id :datomic/actual-id
:temp-id :datomic/temp-id))
(s/def :datomic/actual-ref (s/map-of #{:db/id} :datomic/actual-id))
(s/def :datomic/ref (s/map-of #{:db/id} :datomic/id))
(s/def :foo/bar :datomic/actual-ref) ;; actual goal is to replace `actual-ref` with `ref` here, and still be able to use both `:entity/foo` specs in different situations
(s/def :entity/foo (s/keys :req [:foo/bar]))
;; is there a way to re-use :entity/foo, and avoid writing following:
(s/def :foo/bar* :datomic/ref)
;^^^ I don't even know how to keep attribute name, but change underlying spec
(s/def :entity/tepm-foo (s/keys :req [:foo/bar*]))
2017:05:31 14:00:41 danielneal Maybe you could use macros to generate both sorts of specs according to a macros convention
2017:05:31 14:05:50 misha @danieleneal the fact, that you can't have same spec-name (:foo/bar above) with a 2 different specs under it at the same time, makes me think I need to somehow hot-swap the bottom (:datomic/ref <--> :datomic/actual-ref) spec definition on-demand. Which, given global nature of registry, might not behave/scale well in e.g. treaded env.
2017:05:31 14:07:58 danielneal ah so you don't want :foo/bar* to exist ever, even if it is autogenerated
2017:05:31 14:10:03 misha I'd want at least something like
(let [actual (with :entity/temp-foo {:datomic/ref :datomic/actual-ref})] ;; swaping `ref` with `actual-ref` inside :entity/temp-foo
(s/valid? actual {...}))
2017:05:31 14:14:40 danielneal ah ok, I don't know how to do that
2017:05:31 14:15:38 misha @danieleneal I don't know yet. But at least one of the problem with duplicating entire "tree" of :entity/tepm-foo here, is you can't have 2 different specs for the same attribute at the same time (AFAIK), so you are forced to do s/or, and then, in app code, go through conform data and see which or branch attribute conformed to.
So you can't just use 2 simple (s/valid? :entity/maybe-temp-foo entity) and (s/valid? :entity/only-actual-foo entity)calls in different places.
You need to conform, and search for those or branches with custom walkers.
2017:05:31 14:16:14 misha + this is a trivial example, spec tree might be tens of levels deep (imagine nested datomic pull result, where some nodes are pulled, and some are just {:db/id 9999})
2017:05:31 14:19:26 misha but maybe (re-)defining specs on the fly on-demand is ok. Is it, @alexmiller?
2017:05:31 14:35:35 alexmiller I don't think it's a good idea
2017:05:31 14:36:01 alexmiller If you want non-conforming s/or, then wrap s/nonconforming around it
2017:05:31 14:36:17 alexmiller Or maybe it's s/non-conforming, can't remember
2017:05:31 14:37:22 alexmiller The best way to write a spec is to make true statements about your data
2017:05:31 15:16:10 misha @alexmiller how would you go about speccing same entity for validating against these at the same time:
a) being actual entity (containing no datomic temp ids), and
b) being to-be-transacted entity (maybe containing some temp ids)?
2017:05:31 19:31:06 seancorfield @misha This feels to me like you might be over-specifying your data. If the difference between actual-ref and ref doesn’t matter in most cases, then don’t specify it at all. When and where it actually matters, have a separate spec that can validate you have the one you want.
2017:05:31 19:31:35 seancorfield Deeply nested specs feel like an anti-pattern to me (based on my usage of spec so far in production).
2017:05:31 19:34:25 seancorfield When we first got started with spec, I found myself trying to declare every aspect of every known key in a data structure and soon realized that got in the way of writing generic functions across variants of that data. It took a while to settle into a flow of only specifying what was important for the places where specs were being used to help drive the system.
2017:05:31 19:38:54 dpsutton like unit tests: don't shoot for 100% coverage, shoot for important constraints?
2017:05:31 21:21:54 misha @seancorfield @dpsutton yeah, I am sure such questions are a part of learning and calibration process, on the other hand, there might have been a "function for that" after all
2017:05:31 21:25:43 misha Many of existing things still surprise me, and I'd like to avoid prematurely limit myself in approaches/tools – we are in lisp world after all! :)
2017:05:31 21:37:35 jjttjj if I have a set of possible usernames ["fred" "tom" "mary"] and a set of possible mail host domains ["" ""], how do I combine them into a spec generator to make strings like <mailto:/cdn-cgi/l/email-protection|/cdn-cgi/l/email-protection>
2017:05:31 23:59:49 alexmiller @jjttjj Use generators of s/tuple to combine other generators into random combinations, then use s/fmap to apply a function to the combinations
2017:06:01 00:03:35 alexmiller @misha I’d echo Sean’s advice -either don’t spec the ids, or spec them with an s/or (validation and gen will work better then), or combine an entity spec with an additional id spec if needed in some places, etc.
2017:06:01 04:17:25 jjttjj thanks all
2017:06:01 06:28:54 stbgz hey all I was wondering if there is a tool around that would take a swagger spec and transform it to a clojure spec
2017:06:01 06:29:29 stbgz I am interested in generating test for api specified in swagger
2017:06:01 08:34:33 misha @alexmiller is your f/map approach superior in some way to the following? Or is it just a matter of taste?
2017:06:01 09:53:21 grierson @misha ❤️ I didn't need the (spec/gen)
2017:06:01 09:54:06 grierson I was trying to generate a generator :@
2017:06:01 09:54:16 andrewmcveigh 🙂
2017:06:01 09:54:20 andrewmcveigh why not?
2017:06:01 13:15:12 ikitommi @stbgz have been waiting for someone to do the json schema -> spec converter too. Would also enable things like server code-gen from swagger-spec to a clojure(.spec) web server. Might help the enterprise adoption.
2017:06:01 13:16:35 ikitommi there are spec-tools & speculate which do the spec -> json schema. and spec-swagger for spec -> swagger (only 80% done)
2017:06:01 16:11:28 danielneal is there a recursive version s/describe which returns definitions of any specs referenced in the top level spec? Kinda like a macroexpand-all?
2017:06:01 17:59:09 royalaid Because I want to try and add specs for aleph responses
2017:06:01 18:00:54 royalaid Part of the problem is because deferred are used encapsulate async code and unrealized values the only way I can think of to check the spec is to use a wrapper function that unwraps the value and checks the spec
2017:06:01 19:33:04 wilkerlucio @grierson recommendation: use s as the alias instead of spec, I say that because it's a standard name for everybody, going away from well defined conventions is not a good ideia
2017:06:01 19:43:21 grierson @wilkerlucio Thank you, I will check it out.
2017:06:01 19:46:01 wilkerlucio you'r welcome 🙂
2017:06:01 19:48:20 bfabry @grierson that article has the right idea but we make exceptions for very-widely-used libraries. in those cases there's often a shorthand convention rather than following the rule
2017:06:01 19:50:48 alexmiller @misha one benefit of my approach is that I use only generators built from clojure.spec.gen.alpha, which uses dynamic loading, which means that you don’t need test.check at runtime to load the namespace. other than that, personal preference.
2017:06:01 19:52:18 alexmiller @danieleneal no, there is not a deep-describe, but it’s something we’ve talked about adding. I think having spec specs (CLJ-2112) would be a big help for this to generically walk specs to find sub specs, so I’ve kind of filed it behind that one. Another alternative would be to actually support this as some kind of recursive op built into the protocol.
2017:06:02 07:42:40 danielneal @alexmiller great! Yeah spec'd specs would be exactly the thing for that - would make it much easier to write a deep describe
2017:06:02 22:23:39 xiongtx Is spec supposed to treat undefined keys as “any”?
2017:06:02 23:31:29 english @xiongtx for s/keys, yes. Presence of the key will always be checked when using req and req-un, but conformance is checked only if specs are registered with the same keys
2017:06:02 23:50:48 xiongtx Is that a design decision or just an implementation detail?
2017:06:02 23:51:37 xiongtx B/c that seems like it opens the possibility of forgetting to define a key spec and s/valid? still passing in tests
2017:06:03 07:41:42 misha @xiongtx if you s/exercise it will complain about "no spec for key".
2017:06:03 07:43:33 misha It is design decision, so you could opt in to writing detailed spec, instead of being forced to.
2017:06:03 20:50:08 alexmiller @tclamb seems like a bug, but would be curious to see the actual value causing the error
2017:06:03 20:57:10 tclamb @alexmiller s/exercise is generating maps with clojure.lang.LazySeq keys, like {(lazy-seq '(:r)) :r}
2017:06:03 20:58:07 tclamb that passes s/valid? and s/conform
2017:06:03 20:58:54 tclamb I didn’t expect this to be true as well: (identical? (s/conform spec x) x)
2017:06:03 20:59:49 tclamb but I think that’s an artifact of the map value spec being keyword? instead of e.g. s/cat
2017:06:03 21:04:09 tclamb I ran into this speccing a map-of point to tile for the board in a game like scrabble
2017:06:03 21:55:31 stbgz hey all I am trying to wirite a spec that would satisfy some json data in which the keys have a shape but are not defined eg
2017:06:03 22:25:18 stbgz I am modeling the well-know part of the spec with s/keys and the other with s/map-of however s/merge doesn’t seem to be working given that the “optional x...” could be 0, does merge understand optional?
2017:06:03 22:28:26 tclamb the docs sound like s/merge only works with s/keys, not s/map-of
2017:06:03 22:36:57 stbgz hmm this is harder than I though
2017:06:03 22:37:21 stbgz I think I have to drop-down to everkv
2017:06:04 13:10:43 alexmiller merge works with any map spec
2017:06:04 13:11:36 alexmiller But you do probably need to use every-kv to spec tuples if you're not keywordizing the map keys
2017:06:04 13:13:38 alexmiller @tclamb seems like a bug if you want to file a jira. The identical thing is expected - spec tries to avoid modification if possible for perf.
2017:06:05 19:13:04 wilkerlucio @alexmiller hey Alex, can you tell if there is any progress going on about the namespace aliasing for inexistent namespaces? I remember I read something around this in past, to facilitate having alias used just for namespace resolution of keywords
2017:06:05 19:13:11 wilkerlucio currently I have a bunch of those on my code:
2017:06:05 19:14:41 wilkerlucio that would work same way as require, but without actually requiring that namespace to exist
2017:06:05 19:15:14 wilkerlucio given the reason for that namespace is just organization sake
2017:06:05 20:57:33 alexmiller No update right now, sorry
2017:06:05 20:58:04 alexmiller But I believe Rich has some idea that he has not shared with me
2017:06:05 20:58:41 alexmiller In general I think there is a strong preference not to make ns any more complicated
2017:06:06 00:20:37 mfikes I was surprised by the interaction of fspec and instrument. I was expecting instrument to only check for spec'd function arg conformance, but it seems to additionally result in "exercising" passed functions. A gist with a concrete minimal example of what caught me off guard, where, in the last two forms evaluated in the REPL session, you can see arg conformance checking, followed by the function I'm passing being exercised: https://gist.github.com/mfikes/3ac5ca668b299e104490059ad0ccfcca
2017:06:06 00:24:28 mfikes Perhaps ::number-sink should not be defined using fspec, but instead using ifn?, as in
(s/def ::number-sink ifn?)
2017:06:06 01:24:27 alexmiller @mfikes the idea here is that if you spec a function arg, the only way to determine that the function you actually passed conforms is to exercise it. I have certainly used ifn? as an alternate when that is not appropriate.
2017:06:06 08:46:46 alexmiller it’s possible to do BOTH too by declaring the fspec with the detailed spec, then providing a simpler override spec when you call instrument
2017:06:06 11:03:35 deg It seems that (sort (gen/sample (s/gen (s/inst-in #inst "1800-01-01" #inst "2199-12-31")) 100)) has a very strong bias to generate dates within milliseconds of Jan 1, 1970.
2017:06:06 13:08:19 wilkerlucio thanks for the clarifications Alex
2017:06:06 13:33:53 gfredericks @deg would you want a uniform distribution between the two dates you gave?
2017:06:06 13:48:11 deg @gfredericks Yes. (Definitely for date instances. Probably also for numbers where I've specified a range. OTOH, for unbounded numbers, I agree that clustering around zero is probably best)
2017:06:06 14:01:58 gfredericks It's tricky because test.check wants to be able to generate "simpler" instances when asked
2017:06:06 14:02:21 gfredericks So you're describing a generator that doesn't attempt to do that
2017:06:06 14:33:21 deg I hear ya. But, I don't believe that Jan 1, 1970 is an appropriate simplification nucleus for dates. The fact that it happens to have an internal rep of zero is not interesting for nearly all uses or tests of dates. Clustering around now is much more likely to catch interesting simple cases.
2017:06:06 14:34:35 danielneal is there something like instrument that also checks the :ret value of functions? Just for catching the bad functions I write as close as possible to the problem
2017:06:06 14:35:27 deg And, even for numbers, if I've specified a range, than interesting simple cases are likely to be the two end-points of the range and the mid-point.
2017:06:06 14:39:05 zane Writing your own generators that have that behavior ought to be pretty straightforward.
2017:06:06 14:42:22 deg Sure, but I think this should also be the default behavior.
2017:06:06 15:20:23 gfredericks @deg agreed; the unfortunate thing about "now" is it makes the generator nondeterministic
2017:06:06 15:48:32 deg @gfredericks But, aren't generators already non-deterministic? I already can't assume that I'll get the same values each time. This change just means that I can't assume the same probability distribution either.
2017:06:06 16:00:23 bronsa I might be wrong but I think test.check uses a PNRG for its generators
2017:06:06 16:00:27 bronsa which is deterministic
2017:06:06 16:01:01 bronsa that's how you get reproducibility through the test seed
2017:06:06 16:15:16 deg Ah, ok. Then I see the problem.
In any event, the number range case can still be improved. And, for dates, it might make sense to center around 2015 or 2020, rather than 1970.
To pick one (admittedly contrived) reason: it would be crazed for a test-generator tool written today to only test dates before the y2k problem!!!
2017:06:06 16:23:32 deg Different question.... I just realized that s/tuple needs a vector, and won't validate a list. Why? And, what is an idiomatic way to test for, say, a list of precisely two ints?
2017:06:06 16:25:41 deg Actually, let me rephrase: what is an idiomatic way to test for a list of one int followed by one string?
2017:06:06 16:28:44 gfredericks @bronsa correct
2017:06:06 17:26:56 alexmiller @deg a tuple is inherently positional (indexed) whereas a list is not (it can only be traversed from the beginning).
2017:06:06 17:27:12 alexmiller but you can use (s/cat :i int? :s string?)
2017:06:06 17:27:37 alexmiller regex op specs can be matched against sequential stuff (lists, vectors, seqs, etc)
2017:06:06 17:30:20 alexmiller @deg there is no set of random dates that will be sensical for anyone. if you want to influence it, you should provide a generator that does so (via s/inst-in). inst-in admittedly has a poor generator due to its non-uniform distribution. That is a bug (my fault) and should be fixed.
2017:06:06 17:49:28 deg @alexmiller Understood. That Jira case nicely sums up all my concerns about inst-in.
I think s/int-in should also be generated uniformly.
2017:06:06 17:49:46 alexmiller in my head I thought it did! but it does not.
2017:06:06 17:49:53 alexmiller will fix that as well
2017:06:06 17:49:57 deg thx
2017:06:06 17:50:21 alexmiller and I wrote it!
2017:06:06 17:50:36 deg I'm just starting with spec (looked at it a few months ago, but only seriously these past couple of hours). So far, I like the direction a lot. Great work!
2017:06:06 18:02:01 alexmiller ok, patch attached that fixes both, hopefully will get a look at some point
2017:06:06 23:14:19 drewverlee can a spec be-used as a replacement for de-structuring a function? I know can spec/conform inside your function, but it would be nice if could refer to the symbols directly rather then the map. I suppose even with desctructring you need to refer to the keys…
2017:06:07 00:07:38 alexmiller yes, although due to the performance, I would only recommend doing this in macros (where it happens at compile time)
2017:06:08 02:04:01 bbrinck I find it a little confusing because my original data doesn’t contain ["foo" "bar"] anywhere
2017:06:08 02:25:01 alexmiller regex ops match seqable things. maps are seqable and return 2-tuples of k and v
2017:06:08 02:25:20 alexmiller
user=> (first {"foo" "bar"})
["foo" "bar"]
2017:06:08 02:25:49 alexmiller ^ your data does contain [“foo” “bar”] when viewed as a seq
2017:06:08 02:29:03 bbrinck Correct me if I’m wrong, but maps are intended to be unordered, right? And regexp ops are about specifying an order? When might I use a regex to validate a map?
2017:06:08 02:30:15 bbrinck I suppose if I have a map with zero or one pair, it is ordered and I could use a regex sequence to describe it.
2017:06:08 02:39:56 bbrinck For context, I’m trying to write a pretty-printer that uses explain-data. One part of the pretty printer is a way to highlight where the invalid data “lives” in a larger tree of data. This is tricky in this case (and maybe others) because the :val and :in no longer make sense in the context of the original data, but it’s not clear to me how to unwind this transformation or even figure out it occurred.
2017:06:08 04:24:55 alexmiller I’m not saying you should use a regex op to spec a map, just explaining how spec interprets that :)
2017:06:08 04:27:25 alexmiller In: [0] says it’s the 0th element of the collection that is the problem. in a map the elements are kv pairs and its this kv pair itself that is indicating the problem (not the particular key or value). what other “in” value would help find it better?
2017:06:08 04:29:49 alexmiller I guess you could constrain the type of the collection that works with regex op specs more to only be sequential?. Then you would get a predicate error at the map level.
2017:06:08 04:30:51 alexmiller Something like: In: [] val {“foo” “bar”} fails spec :… at [] predicate: sequential?
2017:06:08 04:33:01 alexmiller oh, the other thing is that in the latest version of Clojure spec (not sure if cljs is synced up but I think so), you get an extra key :value in the explain-data that is the original root value
2017:06:08 04:34:22 alexmiller ^^ @bbrinck
2017:06:08 09:31:29 pithyless
(spec-tools/explain-data (data-spec/spec ::foo {:foo string?}) {:foo 42})
Unhandled java.lang.IllegalArgumentException
Don't know how to create ISeq from: clojure.lang.Keyword
2017:06:08 09:32:11 pithyless ^ spec-tools seems really useful, but the error messages are useless. Am I missing something? Was this a regression introduced in alpha16?
2017:06:08 09:35:21 pithyless ^ maybe @ikitommi can offer some advice?
2017:06:08 09:48:15 pithyless ok, so it works if I use the keyword directly instead.
2017:06:08 14:42:28 bbrinck @alexmiller Thanks for the info!
> I guess you could constrain the type of the collection that works with regex op specs more to only be sequential?
That’s a promising idea. In my own specs, I’ve tried your suggestion and wrapped my cat specs with (s/and sequential? ,,,) and that solves the issue.
2017:06:08 14:42:35 bbrinck I wonder what the tradeoffs would be for enforcing this at the spec level? AIUI, we already do enforce predicates in some cases namely:
There may very well be downsides to requiring the value be sequential?, but a few benefits I can see are:
1. IMHO, the error about not satisfying sequential is clearer to me than an error about a key/value structure.
2. You can get into the case where two maps are equal, but one matches a spec while another does not.
https://gist.github.com/bhb/b617354b22cd0508d14ccbab4e6f5585
Admittedly, this is a carefully constructed spec designed to exploit this issue. I’m not sure it’s a practical concern. But the property of “equal values are equally valid for a given spec” would seem like a nice invariant to maintain.
2017:06:08 14:42:48 bbrinck I’m happy to open a JIRA for this, if it is helpful 🙂
2017:06:08 14:43:03 alexmiller @bbrinck I was suggesting we could build the sequential? check into the regex op spec itself
2017:06:08 14:43:13 bbrinck Ah, agreed 🙂
2017:06:08 14:43:23 alexmiller jira is fine
2017:06:08 14:43:30 bbrinck OK, will do
2017:06:08 14:44:25 alexmiller maps (and sets) are unordered and thus at odds with the value prop of the regex op specs which are about sequentially ordered collections
2017:06:08 14:44:49 alexmiller so it seems like a good idea to catch that up front rather than accidentally allow something that creates a weird error
2017:06:08 14:45:13 alexmiller if you really wanted it, you could always seq a map or set before using it
2017:06:08 15:04:35 bbrinck @alexmiller Do you find it helpful for me to link to this conversation in the JIRA ticket?
2017:06:08 15:55:54 seancorfield "Do you find it helpful for me to link to this conversation in the JIRA ticket?" -- this conversation will be gone in a few days due to the 10,000 message limit in Slack's free tier so I'm not sure you can link to it directly.
2017:06:08 16:39:53 bbrinck Oh, good point.
2017:06:08 16:40:09 bbrinck OK, well hopefully I captured the relevant bits from the conversation in the JIRA ticket.
2017:06:08 18:35:56 bmabey Does anyone know of existing specs for Avro schemas? I haven't found any on github yet but I figured someone must have done this already.
2017:06:08 18:37:22 bmabey I'm starting to create some now and it is pretty straight forward but I'm not sure how to make a spec conditional on other values in a map. In this case the ::default spec for an avro schema is dependent on the ::type. https://avro.apache.org/docs/1.8.1/spec.html
2017:06:09 11:51:20 lambder is there a way to have :gen-max with s/+ ?
2017:06:09 11:52:32 lambder e.g. to limit the size of the collection generated by s/+ spec
2017:06:09 11:59:56 alexmiller currently no, although I think that would be a good enhancement
2017:06:09 12:29:15 lambder thanks @alexmiller
2017:06:09 12:32:29 alexmiller @linuss line 7 - should be s/and ?
2017:06:09 12:33:52 linuss Hahahaha, wow, thanks!
2017:06:09 12:34:00 linuss that's a waste of a morning...
2017:06:09 12:34:16 alexmiller and not a function?
2017:06:09 12:35:17 linuss yeah, I must have dozed off while writing that
2017:06:09 15:17:20 stbgz Hey all, I haven been having problems trying to write a spec as follows.
I have a map that looks like this
{
:key1 "val1"
:key2 "
In essence the map is a mix of well know keys(:key1, :key2, :key3) and and optional set of patterned keys (:x-…). I can easily model either part: (s/keys for the first part and s/map-of for the second part) but I am having a hard time expressing both at the same time. Any ideas on how I can do it?
2017:06:09 16:37:12 alexmiller this is sometimes called a “hybrid map”
2017:06:09 16:37:43 alexmiller you can’t use s/map-of
2017:06:09 16:38:06 alexmiller but you can use an s/merge of s/keys and an s/every-kv that specs the map entry tuples
2017:06:09 16:39:02 alexmiller search for “hybrid” in there to find that part
2017:06:09 17:22:04 ikitommi @pithyless hi, and thanks for the report - there was a bug in explain, should be fixed in [metosin/spec-tools "0.2.1-SNAPSHOT"] (which now uses the latest clj, cljs & spec):
2017:06:09 18:15:27 pithyless @ikitommi - Perfect! I can confirm it fixes my issue. Thanks for the lightning-fast response and apologies for not going through GH issues. I figured I was just doing something wrong 🙂
2017:06:09 19:13:15 don.dwoske I'd like to pass parameters into a predicate function along with the value being checked ... a simple example might be a predicate to validate a integer with a min and max value... i.e. validate x is between 1-10. What are common design patterns for this? First thought is to write a function that returns a closure ... any real-world, slick examples of such a thing?
2017:06:09 19:26:05 joshkh is it possible to have something like (s/coll-of :some.other/spec)?
2017:06:09 19:29:40 ghadi @don.dwoske there's already some builtins like int-in-range?
2017:06:10 14:21:26 stbgz thanks for the help @alexmiller
2017:06:10 16:32:42 mikecarter if i have a person comprising of an id (int) and a name (string), how can i write a spec for a collection of people to ensure there are no duplicate IDs?
2017:06:10 16:36:15 potetm @mikecarter distinct??
2017:06:10 16:38:25 mikecarter @potetm would that catch [{:id 1 :name "Alex"} {:id 1 :name "Frank"}]?
2017:06:10 16:48:42 potetm Something like (apply distinct? (map :id people)) would I think.
2017:06:12 14:01:35 misha @lambder a and b in :a/value are not related to ::a in any implicit way, in case you might wonder
2017:06:12 14:02:53 lambder ok
2017:06:12 14:04:10 misha there, updated it to avoid any confusion
2017:06:12 14:53:01 mpenet @alexmiller should we expect a big reveal for your talk in euroclojure, given the recent mentions of "dependency" related work happening? 🙂
2017:06:12 14:55:58 alexmiller no one expects the spanish inquisition
2017:06:12 15:01:14 alexmiller there are a few projects I have worked on over the last year in this area and the abstract is sufficiently vague to cover whatever I can talk about by the time I do the talk
2017:06:12 15:06:33 misha @alexmiller
multispec assocs dispatch value back to the {} during s/exercise, right?
is there a way to re-use same enrich {} with a dispatch value, but during s/conform?
2017:06:12 15:13:21 misha oh, I can just call multimethod on an actual {} to get the dispatch value
... assuming defmethod's return keyword, not an inline spec-from
2017:06:12 19:18:40 souenzzo I have a query (all-enums-of :my/attr) that return's all my enums. There some how to dynamic gen (s/def :my/attr #{})?
2017:06:12 19:30:37 alexmiller not sure I understand what the last sentence means - looks like you are trying to spec the empty set, which doesn’t make much sense?
2017:06:12 19:33:47 souenzzo I want to dynamic gen the spec of :my/attr. In case, it's a enum so I describe it as a set of keys.... Maybe (s/def :my/attr (all-enums-of db :my/attr)) Should express better what I mean...
2017:06:12 19:36:15 souenzzo self reply - Yep. I can do a macro, It's a good way? It will be declared after connect with the db. It's a problem?``
2017:06:12 19:44:11 alexmiller or eval
2017:06:12 19:53:10 souenzzo Is it a reasonable solution? "Should I use in production"?
2017:06:12 20:16:57 alexmiller this of course has the significant downside that it only works in a few cases like literal sets
2017:06:12 20:19:36 alexmiller or maybe wider than just those but not for things like preds, anonymous functions, etc
2017:06:13 11:46:08 carocad hey guys, quick question: what is the clojure/spec.alpha project in github? is it an old version of clojure.spec?
2017:06:13 11:46:44 gfredericks I think it's the new version
2017:06:13 11:47:34 mpenet yes, the fact that the readme mentions nothing about what it is doesn't help tbh (or how to use it; you have to dig into the google group announcement instead)
2017:06:13 11:49:36 carocad wait what!
so if I update my version of Clojure then it would break 😞 ?
2017:06:13 11:49:43 carocad is it also like that for Cljs?
2017:06:13 11:50:45 mpenet yes, you need to update your require calls and potentially clj.spec/invalid kw
2017:06:13 11:50:54 mpenet dunno about cljs, but I guess so
2017:06:13 11:57:37 hkjels This problem had me spinning for close to an hour
2017:06:13 11:58:26 mpenet wasted quite some time on it too, even tho I knew what to do
2017:06:13 14:37:11 joshkh is it possible to use spec to ensure that one spec'ed key in a map containing a number is larger than the value another spec'ed key?
2017:06:13 14:38:25 joshkh starting with something like this:
2017:06:13 14:42:49 joshkh thanks. that makes perfect sense.
2017:06:13 14:43:41 joshkh s/def takes a keyword and a predicate (or composite of them)
2017:06:13 14:46:22 andrewmcveigh Yeah, the 2nd arg is a spec, which can be specs from the spec namespace, sets, or really any function that returns truthy/falsey, or any combination.
2017:06:13 14:47:15 andrewmcveigh The docs say spec, spec-name, predicate or regex-op
2017:06:13 14:49:26 joshkh i got as far as Usage: (def k spec-form) and didn't bother to read down 😉
2017:06:13 14:55:06 joshkh this is probably a dumb question, but when exercising a spec with a custom function (such as the <= above), is clojure generating ints at random and then only filtering out results that pass the spec? or is it truly generating valid specs on the first try?
2017:06:13 14:55:19 joshkh in other words, how hard is it working to exercise the spec?
2017:06:13 15:32:27 andrewmcveigh s/and uses the first spec as a generator, and the following specs as filters
2017:06:13 15:32:36 andrewmcveigh as far as I remember
2017:06:13 15:33:08 andrewmcveigh If you want something more intelligent, you'd have to provide the generator yourself
2017:06:13 15:40:21 joshkh gotcha, thanks again
2017:06:13 15:40:40 andrewmcveigh np
2017:06:13 16:12:56 mattly is the plan to move clojure.spec.alpha back to just clojure.spec after it's released?
2017:06:13 16:13:15 mattly I might just park at the version of clojure 1.9 I'm using until it goes back
2017:06:13 16:36:21 alexmiller It is likely that 1.9 will release before spec is finalized (that is, 1.9 final will depend on clojure.spec.alpha). Being able to do so is the main reason we did the split.
2017:06:13 16:37:06 mattly ok, thanks
2017:06:13 16:41:47 joshkh can someone help me understand why this is producing a vector of vectors rather than a single collection of numbers?
2017:07:26 21:00:01 joshkh it's all good! your guidance steered me true. i just had to adjust for javascript which... against all odds... supports positive infinity 🙂
2017:06:13 16:45:14 gfredericks it's an instance of Double
2017:06:13 16:45:19 joshkh but i'm still more confused about the deeply nested collections
2017:06:13 16:46:48 gfredericks ah
2017:06:13 16:46:52 gfredericks read the docstring for s/exercise
2017:06:13 16:47:25 gfredericks it's not a coincidence that you have pairs of identical collections
2017:06:13 16:47:50 joshkh pairs of generated and conformed values for a spec ahhh
2017:06:13 16:47:52 dpsutton it's generating ten collections of things satisfying number?, right?
2017:06:13 16:48:04 alexmiller @joshkh exercise generates values, and conforms them, then returns a collection of those pairs
2017:06:13 16:48:36 alexmiller you can also (gen/sample :test/coll) to just get the samples
2017:06:13 16:48:46 joshkh that's the second time i've stared right past the docs today. time for a coffee. thanks guys.
2017:06:13 16:48:53 alexmiller or (map first (s/exercise :test/coll))
2017:06:13 16:49:26 alexmiller @mpenet I updated the spec.alpha readme a little
2017:06:13 16:53:00 mpenet It's not pushed yet, right?
2017:06:14 04:02:50 seancorfield @souenzzo By way of explanation, s/cat -- as one of the "regex specs" -- combines with other regex specs by concatenation rather than nesting.
2017:06:14 04:24:37 souenzzo seancorfield: I was not remembering/understanding the spec usage. 😄
2017:06:14 14:49:20 odinodin Haven't pushed it yet, more of a proof of concept right now.
2017:06:14 14:53:20 richiardiandrea I would be interested in checking, the last missing piece in tooling is something like that
2017:06:14 15:24:04 odinodin Sure, I will push it some time this week
2017:06:14 18:25:50 royalaid @odinodin Looks great! I would recommend looking at other kinds of error reporting that happens in other langs, I hear elm-lang has great error reporting, and I trying to get inspiration from something like that because I don’t think there is a real effort around making something like what you have.
2017:06:14 18:37:41 odinodin Thanks for the tip, will do
2017:06:15 08:48:43 thheller @odinodin thats awesome, any chance of a non-html version? 🙂
2017:06:15 08:51:22 odinodin that would be great, but I won’t make it. However, anyone can do it, it’s not hard 🙂
2017:06:15 08:53:35 thheller damn 😉 … will that be standalone or something integrated in another library?
2017:06:15 08:53:42 thheller probably tied to reagent?
2017:06:15 08:56:15 odinodin What I’m working on is tied to reagent, but it is really not that hard to make.
2017:06:15 08:57:18 thheller I tried and failed a couple times … I’ll check out your implementation ... maybe I just missed something
2017:06:15 08:59:28 thheller probably shouldn’t have started with extreme errors like this one
2017:06:15 09:07:39 athos I'm also working on a similar project (https://github.com/athos/Pinpointer), though it's now broken because of the recent radical rewrite.
2017:06:15 09:11:15 athos For me, it's not so easy work, considering some corner cases.
2017:06:15 09:14:35 thheller @athos thats awesome and text as well
2017:06:15 09:20:11 athos @thheller thank you, and yeah, it's intended to be available from cider.
2017:06:15 09:21:39 thheller I’ll definitely check it out
2017:06:15 09:26:46 thheller I added pretty warnings to shadow-cljs recently which works nicely for cljs analyzer errors
2017:06:16 14:24:27 tap Why (s/valid? ::a [{:m 1}]) returns true, but an error is raised for (foo [{:m 1}])?
2017:06:16 14:25:45 nwjsmith I think that you want the spec for ::a to be (s/every (s/keys :req-un [::m]))
2017:06:16 14:28:44 nwjsmith s/* is a regexp operator, which are used for specifying sequences of data.
2017:06:16 14:29:34 nwjsmith In your example above, an error would be raised for (foo [{:m 1}]), but not (foo {:m 1}) or (foo {:m 1} {:m 2})
2017:06:16 14:29:39 nwjsmith (I think)
2017:06:16 14:32:23 nwjsmith Ah, looks like you'll want to used coll-of instead of every
2017:06:16 14:40:31 tap Ahh, ok. Thanks @nwjsmith
2017:06:17 06:44:15 colinkahn If I have some data like this [:a :b :c :b :a] where the structure is mirrored, but those could be any values, like [:x :y :z :y :x], what would I use from spec to write that?
2017:06:17 15:09:05 weavejester @colinkahn: You could have a predicate like #(= % (reverse %))
2017:06:17 15:15:06 colinkahn @weavejester thanks, I realized you could do something similar but my solution wasn't as concise 😄
2017:06:17 15:23:18 colinkahn is there a more declarative way to define it? I was searching around and found recursive regular expressions with this solution: (\w)(?:(?R)|\w?)\1 But that requires you to be able to reference the group, which I don't see a way to do that with clojure.spec. Maybe using tags?
2017:06:17 15:27:22 colinkahn that's from this article: http://www.rexegg.com/regex-recursion.html
I know it's for palindromes which wasn't what I originally posted, but curious if you could solve problems in that way using clojure.spec
2017:06:17 17:55:35 flyboarder Hello everyone, I have a clojure macro that is in a .clj file, I use this macro within .cljs files however it seems I cannot spec this macro???
2017:06:17 18:41:32 matan Can someone kindly explain the point of what is called in spec docs "instrumentation"?
2017:06:17 18:41:46 matan I am not sure I get what it is that it actually does, or why it is called that way
2017:06:17 18:42:32 matan Aren't args checked anyway, a la :pre ? if not, what is the rationale?!
2017:06:17 18:42:43 matan Thanks in advance for clarifying..
2017:06:17 18:57:38 weavejester @matan Instrumentation is a little like :pre, except that you can turn it on and off selectively.
2017:06:17 18:57:57 weavejester So during development you might have error messages that you do without in production.
2017:06:17 20:39:09 mobileink anybody else working on spec-based transformation? use case is representing html head/meta stuff as a clojure map. the task is to transform such a map into html <link>, <meta>, etc. my xform code mimics spec : register a transform for each specked kw, then crawl the validated structure to xform it. this seems like a natural offshoot of spec. i can't be the only person who wants it. what else is out there?
2017:06:17 21:19:56 zane Any idea why cljs.spec.test.alpha/check would return a 0-element vector?
2017:06:17 21:42:02 zane Seems like cljs.spec.test.alpha/check is #{}. Trying to figure out why that is.
2017:06:17 22:29:47 zane What's the recommended way to integrate testing the :ret part of function specs in an automated fashion via clojure.test?
2017:06:18 02:49:53 matan @weavejester thanks, so which parts of clojure.spec are ignored when code runs uninstrumented, and which parts always execute?
2017:06:18 02:50:53 weavejester @matan All parts are ignored. If you write a spec for a function, it doesn’t affect how the function runs unless you instrument it.
2017:06:18 03:35:45 matan mmmm now it all makes sense. thanks a lot @weavejester 🙂
2017:06:18 03:37:15 matan nice stuff, spec, IMO should have been there from the beginning of clojure, as a way of taking/extending the good parts of OO rather than dismissing OO benefits so dichotomously
2017:06:18 04:01:01 alexmiller I think Clojure has always embraced some parts of OO (like interfaces and polymorphism) while letting some parts go (encapsulation, concrete inheritance)
2017:06:18 04:28:25 seancorfield @matan I'm curious: how do you think spec relates to the "good parts of OO"?
2017:06:18 04:37:56 seancorfield (and I guess that would also prompt me to ask "What do you think the good parts of OO are?" -- and, full disclosure, I was on the ANSI C++ Standards Committee from '92 to '99, and got started with Java in '97, so with that background I'm often very interested when I hear folks talk on this topic)
2017:06:19 06:39:11 matan I think the good part, as it relates to what spec is, is that structure is kept validated (in OO, only in terms of types and composite types, which is what most objects in a program are) at most points. This helps 50% of the programmer population keeping programs in check, although, recently, unit-tests have culturally evolved into another aid in that regard.
2017:06:19 06:39:55 matan @U04V70XH6 obviously my thoughts aren't that interesting, I've not been on any committee.
2017:06:19 22:09:56 seancorfield The problem is that with Java, and several other OO languages, you have no way to separate type and structure — they are conflated. You can’t talk about data structures without giving them class/type names and your only form of “validation” really is the type system. It’s both “not enough” and “too much”. So I don’t find that to be a good part of OO — I find it to be a failing. The same with encapsulation: it makes it harder to treat data generically and so you end up with a stack of design patterns to workaround the restrictions of the OO type system — and I’ve just gradually found the downsides outweigh the benefits. Mind you, you’re right about the “50% of the programmer population”: Java is very effective at allowing large numbers of average developers to work together to build large, complex systems. Java makes developers fungible… But I don’t think developers should be fungible 🙂
2017:06:20 10:54:34 matan I concur on all points made. Just not sure how encapsulation makes it hard to generically treat data
2017:06:20 10:57:19 matan It might seem valid to encapsulate, especially when delivering a library (to make the codebase have a restricted API facade or making it explicit what are the internals of a module vs how to use it)
2017:06:20 10:57:41 matan But as said, I probably need help seeing the negative implications. I am sincerely curious about them
2017:06:20 16:13:45 seancorfield If you encapsulate your data in a specific type, then you need a function on that specific type to manipulate it. If you want to use a generic function, you have to use the underlying implementation data, “breaking” encapsulation, and then put all the data back afterward. In other words, in Java, if you have an object that contains an array of other data, you can’t use generic array functions without exposing the array directly, i.e., unencapsulating it. Clojure’s approach is that the “API” is just data — immutable data — so if the implementation is a vector (array), then you can use any generic sequence function on it.
2017:06:18 20:20:26 richiardiandrea Newbie question, why does s/or have keyword branches for predicates and s/and does not? I have never noticed that until now 😄 I was kind of assuming the same API but go burned there.
2017:06:18 20:39:38 gfredericks my guess is because it's useful for s/or to know which branch matched
2017:06:18 20:40:07 gfredericks and not useful for s/and, since you already know that all of them matched (though I suppose you could argue it would be useful to know which one didn't match in the case of an error
2017:06:18 20:40:08 gfredericks )
2017:06:18 20:47:54 richiardiandrea Yep that is true, it would be great to know which one didn't and also to have uniformity in the API I guess
2017:06:18 20:56:06 gfredericks the difference is that one is used for conforming and the other isn't
2017:06:18 20:56:28 gfredericks whether that's a good reason for the API to be different is a different question
2017:06:19 01:45:54 flyboarder @gfredericks thanks, I guess thats the difference between runtime specs for cljs
2017:06:19 01:46:21 gfredericks @flyboarder the same applies to clj-jvm
2017:06:19 01:47:32 flyboarder makes sense
2017:06:19 06:40:36 matan @alexmiller @seancorfield what's wrong with encapsulation, from the colujarians' world view? 🙂
2017:06:19 06:40:48 matan I don't recall a very convincing blog post, or have just forgotten if I saw one
2017:06:19 06:41:21 matan Oops should have moved it to #off-topic
2017:06:19 06:42:17 matan Anyway I am very happy about spec, it is really a "missing piece" for robust programs that teams can collaborate on
2017:06:19 12:15:16 alexmiller @matan in OO (particularly thinking of Java / C++ here), encapsulation is used to hide or protect the data inside an object and access it via methods. Because this data is mutable, the accessor/mutator methods are also a place where you can apply a locking strategy to protect that mutable data in multi-threaded usage.
2017:06:19 12:16:23 alexmiller Clojure turns this inside out and says, data is actually the most important thing here - make it visible, introspectable, and directly manipulable via a generic data interface, rather than via a custom set of API methods that is different for every class.
2017:06:19 19:34:20 mobileink alexmiller: #off-topic an alternative perspective, just for fun: "immutable datum" is just another way of saying "pure function". we could discard the notion of "data" altogether.
2017:06:19 12:17:24 alexmiller Immutable data structures allow you to do this while also being automatically thread-safe and avoiding the need for locking around access and manipulation of mutable data completely.
2017:06:19 12:24:12 gfredericks even more than thread-safety, you get independence of usage -- you can pass the same object to different pieces of code and not worry about what things they will each "change" that might affect the other
which may or may not be what alex's last sentence meant
2017:06:20 11:10:01 matan gfredericks: sorry for being late to return to the party 😅 I am not sure why I would not worry about different pieces collaborating on the raw data. Or how this is different than the claim of concurrency/thread-safety.
2017:06:20 11:26:09 gfredericks they're not collaborating, they're doing completely independent things
2017:06:19 12:29:08 leonoel core.async channels and go blocks are another good example of encapsulation
2017:06:19 12:33:17 alexmiller Go blocks are about process, not data. Channels are about conveyance of values, not encapsulation
2017:06:19 12:34:36 leonoel sure, but they both have opaque internal state
2017:06:19 12:35:18 leonoel which doesn't make them bad primitives, btw
2017:06:19 12:36:50 leonoel I get your point, but "encapsulation is bad" goes a bit too far imo
2017:06:19 12:37:55 alexmiller Agreed, but I didn't say that
2017:06:19 12:41:41 alexmiller Most encapsulation in OO is incidental, not useful
2017:06:19 12:44:53 gfredericks this is not quite the same topic but it just occurred to me that one of my biggest frustrations with legacy OO codebases that I haven't heard articulated before is that the classes I encounter have unclear lifecycles
2017:06:19 12:45:43 gfredericks the thing I want to understand about a class to help me understand the whole codebase is how/why the objects get instantiated, what you do with them while they're alive, how they get discarded, how that relates to the lifecycle of the program as a whole; and I've never seen an OO codebase where that's remotely clear
2017:06:19 12:46:37 gfredericks perhaps there's a similar question about how functions are used in a functional codebase, but it seems like less crucial of a question for some reason
2017:06:19 12:55:23 alexmiller “lifecycle” implies some degree of statefulness to me. mutable state requires coordination. all of that is often a mess in OO.
2017:06:19 12:56:14 alexmiller although I’d say that aspect is a distant second problem for me after the notion of not having a generic data interface and/or clear equality semantics in OO
2017:06:19 16:02:08 carocad alexmiller: that is an interesting thought. I am wondering, is there a consensus about “this is not generic enough”. Currently I am working on a program were I need to squeeze every inch of performance that I can get so I ended up defining some custom interfaces to access the data. Nevertheless this puts the clear equality semantics and generic data interface a bit to the limit since the interface is no longer Clojure-like but rather case-specific.
2017:06:19 19:37:51 alexmiller I have no problem with breaking every rule in the book in very narrow areas where you are seeking the ultimate performance :) hopefully that is about 0.1% of the code
2017:06:19 20:20:15 carocad Well that depends on how you define 0.1% of the code. The loc that use those custom interfaces are very short but the ones that consume it are probably very long and very different. Which is what got me thinking.
I think it is a similar situation to what Clojure does i.e defining algorithms and interfaces in java yet using them in Clojure.
2017:06:19 12:58:06 gfredericks the "no generic data interface" complaint also applies to type-oriented functional programming (haskell etc.), right?
2017:06:19 12:58:26 alexmiller I haven’t done enough of it to say
2017:06:19 12:59:25 gfredericks that feels like the biggest difference to me between clojure and most of the rest of FP, even more than static vs dynamic type system
2017:06:19 12:59:51 gfredericks though I'm sure the static vs dynamic dimension is highly correlated with generic vs typey
2017:06:19 18:07:09 spieden yeah not having equality, robust hashing and string representations built into your data structures is terrible
2017:06:19 18:09:19 gfredericks I had a giant dump of ruby data the other day and would have appreciated being able to read it into a repl and analyze it, but I just had to decide to not want to do that
2017:06:19 19:02:49 mobileink gfredericks: that must'v hurt like hell.
2017:06:19 19:04:45 gfredericks so much #<...>
2017:06:19 19:10:57 mobileink gfredericks: i hate sharps in my dumps.
2017:06:19 18:13:38 spieden one of my complaints with clojure is actually that the default string representation isn’t EDN — this decision confuses me. e.g.
(println {:foo "bar"})
{:foo bar}
=> nil
2017:06:19 18:14:08 spieden i use prn-str for everything, so it’s not a big deal
2017:06:19 18:16:23 spieden i guess not all clojure data can be represented as EDN because of nested Java objects, but why not quote “foo” above?
2017:06:19 18:28:54 gfredericks because then (println "foo") would be surprising
2017:06:19 18:29:05 gfredericks you'd need some other mechanism for printing a raw string
2017:06:19 18:31:57 spieden ah yeah, that makes sense
2017:06:19 18:32:12 spieden i guess you could special case strings inside of collections
2017:06:19 18:33:06 gfredericks that would bother me even more
2017:06:19 18:33:58 gfredericks but that might be a personality thing; some people like things that are less complicated to describe, others like things that are more likely to do what you wanted
2017:06:19 18:34:34 spieden well said. i lean towards the former too
2017:06:19 18:35:06 gfredericks ironically, now that I think about it, I feel like gripes about clojure are pretty evenly split between people wanting the former and getting the latter, and vice versa
2017:06:19 18:41:23 spieden a good knife’s edge to balance on =)
2017:06:19 18:43:37 gfredericks I suppose there are also gripes about situations where neither ideal is achieved
2017:06:19 20:27:57 shaun-mahood @alexmiller: Is there a place on the http://clojure.org site that would make sense for compiling links to blog posts, videos, and other external resources related to spec? There's a bunch of good ones on the Cognitect blog and elsewhere that I wouldn't know how to find if I were looking at it for the first time.
2017:06:19 21:39:48 alexmiller Depends what they are
2017:06:19 21:41:04 alexmiller The community/resources page is one place but may not be best for everything
2017:06:19 21:41:23 potetm @spieden Just to make sure it's said: pr does exactly what you want.
2017:06:19 21:42:01 potetm The docs actually say:
> By default, pr and prn print in a way that objects can be read by the reader
2017:06:19 21:45:25 alexmiller @shaun-mahood give me examples and I can help find answers
2017:06:19 22:10:06 shaun-mahood I just noticed your previous reply - the main motivation for my question was realizing that if I only look at http://clojure.org as it stands right now, none of the really interesting or exciting use cases jump out at me. Might not be a goal though, so I didn't want to put any work into a PR without figuring out what kind of thing might be useful for others.
2017:06:19 22:33:06 naomarik i could appreciate such a compilation
2017:06:20 05:26:06 alexmiller @shaun-mahood broadly, I think it’s hard to give a list of blog entries the right context to be useful and survive well over time. It just doesn’t make sense to try to track every spec blog (most of which are now out of date). What’s useful on the site is curated reference or tutorial content that is kept up to date. So I guess largely, I would say these should not be added to http://clojure.org. One thing that I have been (very slowly) working on is a reference page for spec which I think is a good complement in between the rationale, the guide, and the api docs.
2017:06:20 05:30:03 alexmiller and the spec workshop notes are not intended to be public - they are training materials copyright Cognitect and represent probably 100 hours of work over the last year.
2017:06:20 05:38:51 shaun-mahood That sounds reasonable to me. I love the idea of a reference page - I'm still trying to come to grips with the boundaries of the new clojure and clojurescript site.
2017:06:20 05:41:15 shaun-mahood I hope the workshop notes at least made the book easier to write.
2017:06:20 05:43:12 shaun-mahood It was definitely worthwhile to take at last years Conj, I would consider taking it again to pick up the new stuff and the bits I missed.
2017:06:19 21:50:46 spieden @potetm yes thanks
2017:06:20 11:17:53 matan Sorry but after all the discussion about encapsulation, I still find that hiding (encapsulating) internal implementation of "complex" mechanisms (not silly field values like in Java) is a reasonable thing for helping others/self use of components without shooting one's own foot or scanning dozens of fns to figure what is meant to be used from the outside
2017:06:20 14:17:15 alexmiller matan: well, you’re arguing against something I didn’t say. I didn’t say that all encapsulation was bad, I said needless encapsulation of data is bad and OO as typically practiced encapsulates all data.
2017:06:20 14:30:39 donaldball When I have a complex side-effecting machine with a simple use interface, I reach for a protocol. You can also convey intentions about var usage with namespace design.
2017:06:20 14:47:03 colinkahn @U04V4HWQ4 what’s an example of using namespace design? I might be overthinking the meaning, but I haven’t heard that term before.
2017:06:20 14:50:01 donaldball Maybe better examples but one that jumps to mind is core.async, where there is a public api (clojure.core.async) and a bunch of impl namespaces (e.g. clojure.core.async.impl.buffers)
2017:06:20 11:18:42 matan (and that's why we have namespace private declarations I think)
2017:06:20 14:12:52 alexmiller matan: note that even private vars are still accessible - anyone can obtain and use them still. The private meta is really more documentation than encapsulation.
2017:06:20 15:21:38 matan Right!
2017:06:20 13:47:46 bbqbaron that’s a good insight; i think the clojure difference is probably avoiding 1. colocating state with logic in memory 2. requiring encapsulated fn A to bring B, C, D, E, F and all their dependencies with it
2017:06:20 14:05:33 joshjones @matan this belongs in #off-topic since this is not about clojure.spec (or even clojure itself) FYI
2017:06:20 19:34:53 chillenious How would I rewrite this to avoid duplication?
(s/def ::PrimaryContact (s/keys :req-un [::FirstName ::LastName ::Email]
:opt-un [::Title ::Company ::Type ::Phone ::Fax]))
(s/def ::SecondaryContact (s/keys :req-un [::FirstName ::LastName ::Email]
:opt-un [::Title ::Company ::Type ::Phone ::Fax]))
2017:06:21 18:59:53 royalaid But it seems a bit hacky and the output on failure isn't very helpful
2017:06:21 19:03:07 mattly @royalaid I can't speak to clojure.spec, but with test.check I use test.chuck, which has a macro checking that behaves similar to testing
2017:06:21 19:03:24 mattly I'd link you to it except slack's electron app decided I can't use my clipboard anymore
2017:06:21 19:10:40 sneakypeet Hi all. started using spec in cljs today. I am running into a weird issue
(ns tenandsix.app.flow
(:require-macros [cljs.spec :as s])
(:require [cljs.spec :as spec]))
(s/def :flow/action-type keyword?) => works
(s/def :flow/action (s/tuple keyword? keyword?)) => No such namespace: s, could not locate s.cljs, s.cljc, or Closure namespace ""
2017:06:21 19:11:17 dpsutton :flow vs :flo possibly?
2017:06:21 19:12:12 sneakypeet no that was actually me testing if the keyword made a difference 🙂
2017:06:21 19:12:22 sneakypeet but no that does not fix it
2017:06:21 19:13:06 sneakypeet any thoughts? I mean this is as basic as it gets
2017:06:21 19:13:55 dpsutton thoughts, perhaps s/tuple is a macro and the import macro statement is malformed
2017:06:21 19:14:02 dpsutton but only when actually trying to macroexpand
2017:06:21 21:33:56 gfredericks probably with an extra # in a spec context
2017:06:21 23:24:16 alexmiller @adamfrey I would say (s/gen #{1})
2017:06:22 01:27:24 madstap In Rich's speculation keynote he mentioned that there are ways to check if backwards compatibility is broken in both set theory and regex theory.
The set one would presumably be something like:
Are there any links to info about the regex part? Or am I misunderstanding something?
2017:06:22 02:05:41 alexmiller madstap: the regex part is based on Matt Might’s papers with regex derivatives and I believe Rich has spent enough time with it to believe that the math there makes this computable. I’m having a strong sense of deja vu - have you asked this before?
2017:06:22 02:13:18 madstap I'm pretty sure I haven't. Thanks for the pointers 🙂
2017:06:22 03:14:54 alexmiller Must have been someone else, or my addled brain
2017:06:22 01:35:17 sooheon Hey is there function to coerce all keys in a map to namespace-qualified keys, other than the #::{} syntax? I’m receiving an unqualified map from an API call, and would like to spec the result.
2017:06:22 01:37:10 jjttjj @sooheon if you just need to spec un-namespaced keys you can just do (s/keys :req-un [::my-key1 ::my-key2]) to require those keys un-namespaced, ie {:my-key1 "x" :my-key2 "y"}
2017:06:22 01:39:15 sooheon Thanks, I did see that in the docstring, and it did work
2017:06:22 01:39:40 sooheon But now when I pass around this map and assoc my own keys to it, it will have a mix of qualified and un- keywords…
2017:06:22 01:41:11 jjttjj s/keys can take vectors for both :req-un and :req if you have both namespace qualified and unqualified keys
2017:06:22 01:41:34 sooheon Yeah, it’s just a bit of overhead for me
2017:06:22 01:41:47 sooheon would have preferred to just go all namespaced, you know?
2017:06:22 01:41:54 sooheon Thanks anyways :)
2017:06:22 02:02:26 seancorfield @sooheon You could just use reduce-kv and produce a new map from the old one with namespace-qualified versions of the keys (assuming you want the same qualifier on all of them).
2017:06:22 02:03:44 seancorfield Something like (reduce-kv (fn [m k v] (assoc m (keyword "qualifier" (name k)) v)) {} api-result)
2017:06:22 02:04:04 sooheon ah cool! hadn’t thought of that. Was just hoping something like (into #::{}) would magically work
2017:06:22 02:04:51 sooheon is this something that is recommended, btw? to use qualified keywords for everything?
2017:06:22 02:05:31 seancorfield I think it’s a reasonable approach for handling domain data within your system, yes.
2017:06:22 02:06:14 seancorfield Bear in mind tho’ that you probably want to keep those API result keys separate from your actual domain keys in order to avoid conflicts.
2017:06:22 02:06:59 seancorfield So you might have :external.api/name and :my.domain/name with different specs.
2017:06:22 02:08:01 seancorfield This is one of the reasons that the latest java.jdbc lets you specify :qualifier in the options on almost every call so you can get namespace-qualified keys in your SQL result hash maps.
2017:06:22 02:08:45 sooheon I see. I think it’ll just take some experience (just like knowing how to split up and organize ns’es in the first place)
2017:06:22 02:09:01 sooheon thanks!
2017:06:22 02:18:40 seancorfield Bear in mind that the namespace-qualifiers on keywords do not need to correspond to actual code namespaces so you have a lot of flexibility.
2017:06:22 02:19:50 seancorfield For example, we use a qualifier of wsbilling for all entities that relate to our (World Singles) billing system but we don’t have a namespace called wsbilling.
2017:06:22 02:21:27 seancorfield When we accept JSON/Clojure data from outside (APIs, databases), we can normalize it to whatever qualifier we want to distinguish it, within our system, from our billing data.
2017:06:22 02:22:58 seancorfield (if this was a public Clojure API we’d use a reverse-domain-name prefix, like com.worldsingles.billing, I expect)
2017:06:22 02:23:22 seancorfield You can choose how to keep unique groups of entity names separate from each other.
2017:06:22 02:23:31 joshjones @sooheon @seancorfield I made a function a while back to namespace-qualify a map. here's what i have, if it's useful to anyone
(defn map->nsmap
"Creates a namespaced-map from a standard one and a namespace (obj or string)"
[m n]
(reduce-kv (fn [acc k v]
(let [new-kw (if (and (keyword? k)
(not (qualified-keyword? k)))
(keyword (str n) (name k))
k) ]
(assoc acc new-kw v)))
{} m))
2017:06:22 02:23:58 sooheon Ah great, that’s very useful
2017:06:22 02:25:11 sooheon @seancorfield I’ve still got to experience first hand how ns qualifying keys works with stuff like destructuring, and making sense of how I want to organize things, but thanks for the pointers
2017:06:22 02:25:19 seancorfield That’s good for preserving existing name-qualification. Sometimes you need to override that. And of course my simplistic example only works for keys that you can call name on (and assumes you want all-keyword keys back out)!
2017:06:22 02:25:46 seancorfield @sooheon Yeah, it takes some getting used to, after so many years of unqualified keywords!!
2017:06:22 02:27:30 seancorfield In terms of destructuring:
user=> (let [{:foo/keys [a b c]} {:a 1 :foo/b 2 :c 3 :foo/c 4}] (println a b c))
nil 2 4
2017:06:22 02:27:40 joshjones yes, i wrote it to behave the way the #: reader macro behaves, namely, it does not disturb non-qualified keys
2017:06:22 02:27:42 joshjones
#:foo{:a 1 :bar/b 1}
=> {:foo/a 1, :bar/b 1}
2017:06:22 02:27:58 sooheon you mean already qualified ;)
2017:06:22 02:28:10 joshjones yes, sorry
2017:06:22 02:28:23 joshjones and non-keyword keys
2017:06:22 02:32:03 joshjones actually it seems to work by putting the :: in the vector of keys too, though i'm not sure it's a good idea
2017:06:22 02:32:04 joshjones
(let [{:keys [a b c]} {:a 1 :foo/b 2 :c 3 ::c 4}] (println a b c))
1 nil 3
=> nil
(let [{:keys [a b ::c]} {:a 1 :foo/b 2 :c 3 ::c 4}] (println a b c))
1 nil 4
2017:06:22 02:32:20 sooheon so the last example would be the way to mix and match ::b:foo.bar/c and :d
2017:06:22 02:32:31 seancorfield I expected that to be an error but apparently you can use keywords in the key vector? @alexmiller Is that supposed to be valid?
2017:06:22 02:33:41 sooheon You can’t have multiple :keys::keys and :foo/keys deconstructions separately at once, right? You’d do {:keys [a ::b foo/c]}
2017:06:22 02:55:25 seancorfield That’s my understanding, yes.
2017:06:22 03:10:55 alexmiller @seancorfield yes, we support keywords there so that you can use autoresolved keywords
2017:06:22 03:11:31 seancorfield Is that documented? (I guess the docs could have been updated since I last looked at them… 🙂 )
2017:06:22 03:11:34 alexmiller And you can have multiple keys destructurings at the same time for different namespaces
2017:06:22 03:12:47 alexmiller It was in the changelog when it went in but that was a while ago
2017:06:22 03:12:55 seancorfield Yup, the destructuring guide contains examples.
2017:06:22 03:13:12 seancorfield Not of ::k directly but of ::p/name
2017:06:22 03:13:27 alexmiller Rich and I worked on some updated docs for destructuring but I can't remember if those actually got all the way done
2017:06:22 15:49:15 hlship What I don't get is why it is trying to create a generator. It just supposed to be validating inputs into our schema/compile function.
2017:06:22 15:49:53 hlship Basically, the function I'm passing in doesn't have a fspec, so I'm going to add one and see if that helps.
2017:06:22 15:53:29 hlship Nope. Bad guess. Providing my own fspec didn't work.
2017:06:22 16:25:29 alexmiller Right, so that was my guess
2017:06:22 16:25:58 alexmiller the way that fspec args are validated in instrumentation is by using the fspec’s generator to generate values and invoke the function
2017:06:22 16:26:52 alexmiller and check the ret and fn spec to verify the function with random inputs is valid according to the fspec
2017:06:22 16:28:02 alexmiller thus all fspec’s should have an args spec that generates
2017:06:22 16:28:31 alexmiller the surprising bit here is that they need that not just for check, but also for instrument
2017:06:22 16:29:01 alexmiller same is also true of fdef for check, but not for instrument
2017:06:22 16:29:51 alexmiller (because checking an fdef involves generating using its args generator)
2017:06:22 16:36:40 hlship (in a meeting, will digest and get back to you)
2017:06:22 16:41:25 mpenet Just wondering, why not just wrapping args/ret of fspec fns with s/valid? when instrumented
2017:06:22 16:42:00 mpenet It would fail "later" (invoke time) but might be more intuitive (and faster)
2017:06:22 16:46:45 mpenet Could be an option
2017:06:22 17:05:16 alexmiller not sure if Rich considered that or not, haven’t talked to him about it
2017:06:22 17:31:51 mpenet Not the first time this comes up: clj-1936
2017:06:22 17:44:05 hlship So I'm a bit lost .... is there no way I can simply declare what my callback should look like without coming up with a generator for the callback?
2017:06:22 19:41:47 alexmiller hlship: in short no, if you want to also validate the fdef. other options are to override the fdef args generator or to use a generator override on instrument
2017:06:22 19:20:14 arohner @gfredericks in core.specs.alpha
2017:06:22 19:20:26 arohner the clojure.core specs that got split out into a separate lib when spec got split out
2017:06:22 19:20:34 arohner @alexmiller I’m confused by the definition of ::core.specs/map-bindings. What does it mean for (s/every ... :into {}) I don’t see a guarantee that the every coll has an even number of elements, nor a requirement that the input coll is a map
2017:06:22 19:20:52 arohner (this is coming up because I’m trying to run spectrum inference on clojure.core)
2017:06:22 19:28:54 arohner it seems very weird to me that :into is used for generating, but the resulting conform doesn’t change
2017:06:22 19:29:37 arohner the real source of my trouble is just below, when I have to handle (s/def ::map-binding-form (s/merge ::map-bindings ::map-special-binding))
2017:06:22 19:29:48 arohner where s/merge takes maps
2017:06:22 19:43:19 alexmiller sorry, just reading this now. where is your question at now?
2017:06:22 19:44:06 alexmiller note that s/merge doesn’t flow conformed values so only the last map spec in the merge will matter for the conformed value
2017:06:22 19:44:11 arohner ::map-binding-forms takes two maps, because of s/merge, but map-bindings is not guaranteed to return maps, because it uses :into and not :kind
2017:06:22 19:49:25 alexmiller I don’t agree with the last clause there?
2017:06:22 19:50:07 alexmiller both :into and :kind affect s/every gen
2017:06:22 19:50:15 arohner (s/conform ::clojure.core.specs.alpha/map-bindings '[[foo bar]]) is that expected?
2017:06:22 19:51:18 alexmiller I’d say that’s not a supported map binding form
2017:06:22 19:51:31 alexmiller I guess we could use :kind to narrow that
2017:06:22 19:51:51 arohner right, spectrum is currently picky about that, and I’m trying to understand it / make it work
2017:06:22 19:52:05 arohner spectrum says map-binding isn’t necessarily a map, so the merge isn’t guaranteed to work in all cases
2017:06:22 19:58:51 alexmiller yeah, that seems good. if you want to file a jira, we can try to get that in next release
2017:06:22 20:01:01 arohner sure. in CLJ, or does spec have it’s own project now?
2017:06:22 20:01:39 alexmiller CLJ
2017:06:22 20:01:56 alexmiller if you want to make a patch, that will be faster than me making it (as I can screen it)
2017:06:22 20:02:12 alexmiller patch against core.specs.alpha of course
2017:06:22 20:08:00 arohner sure
2017:06:22 21:40:27 peeja What's the best way to define a function :args spec for a function which takes a single argument?
2017:06:22 21:46:00 joshjones @peeja - same way as you do for any :args spec
2017:06:22 21:46:15 peeja So, still use s/cat then?
2017:06:22 21:46:24 joshjones yes, as it's still a vector of one element
2017:06:22 22:06:54 alexmiller agreed and to take that a step further, it’s useful to use a regex op (s/cat) here because the conformed value will be a map with the arg name as the key, and that’s useful in the :fn spec or in explanations
2017:06:23 12:02:10 odinodin given a spec (obtained from ex-data of an exception thrown by inspect, i.e the :cljs.spec.alpha/spec), how can one obtain the key of that spec in the registry?
2017:06:23 12:02:48 odinodin I want to go from spec -> the name of the fdef’ed function that failed
2017:06:23 12:28:31 odinodin My current solution is to parse it out of the exception message, which contains “Call to #‘name-of-var did not conform to spec…“. It would be nice if instrument included the key in the ex-data, but I might be misunderstanding something here.
2017:06:23 13:13:28 wilkerlucio @odinodin can you give please show the data you are trying to go from -> to?
2017:06:23 13:13:47 wilkerlucio I think you might want to use (s/form) or (s/get-spec), but I'm not sure yet what you are trying to get
2017:06:23 13:16:10 odinodin @wilkerlucio it is a spec record object, as provided by the exception thrown by instrument when a fdef’ed function is called with non-conforming input.
2017:06:23 13:18:29 odinodin it is the spec object that (s/get-spec 'the-fdefed-var) would return as far as I can tell
2017:06:23 13:19:59 wilkerlucio that's a long chain, hehe, I'm not sure what kind of data you are looking at, can you paste what your (ex-data) result looks like?
2017:06:23 13:24:48 odinodin This is what (ex-data) returns:
2017:06:23 13:25:15 odinodin I want to know what the :cljs.spec.alpha/spec object is referring to
2017:06:23 13:26:02 odinodin or it would probably be more correct to say, what key points to this spec in the registry
2017:06:23 13:28:41 wilkerlucio @odinodin gotcha, you can try calling (s/form) and (s/get-spec) on it, but I'm not sure if will work
2017:06:23 13:28:59 odinodin I’ll try that
2017:06:23 13:29:08 odinodin thanks
2017:06:23 13:39:00 odinodin @wilkerlucio (s/form) only returns the spec as defined on the (s/fdef) var. I’ll just parse the var out from the error message instead. Anyway, thanks for your suggestions 🙂
2017:06:23 13:39:36 wilkerlucio no problem, I'm glad you can at least work around it, maybe other people will have a better solution 😉
2017:06:23 22:04:28 bbrinck I searched JIRA but could not find an issue, so it may be a misunderstanding on my part
2017:06:23 22:05:11 bbrinck @samueldev ^-- (sorry, forgot to mention you, see my comment above)
2017:06:23 22:11:29 bbrinck @samueldev I think @ericnormand is also maybe working on something for more human-readable error messages?
2017:06:23 22:12:04 samueldev thanks @bbrinck ! I'll do some digging with @ericnormand's work and bruces!
2017:06:23 23:46:58 richiardiandrea I think also the above odinodin is working on something for re-frame
2017:06:24 17:46:52 drewverlee any idea why evaling (clojure.spec.alpha/valid? (clojure.spec.alpha/coll-of int? :kind vector?) [1 2 3]) would cause a stack over flow error?
2017:06:24 17:47:48 joshjones works fine over here -- can you give a context for how you're using it?
2017:06:24 17:49:35 drewverlee @joshjones from within the cider repl
2017:06:24 17:52:19 drewverlee hmm
2017:06:24 17:52:44 drewverlee i restarted my repl and it works fine. I think i know the interaction thats causing it, if not why…
2017:06:24 17:54:24 drewverlee nope, i was wrong, i have no idea what was causing it. 😁
2017:06:24 17:55:15 joshjones hmm strange
2017:06:25 16:34:18 jjttjj I know spec is maybe too new for idioms to develop but is there a clear answer to having :x.specs.user/email, :x.specs.user/password and then :x.specs.user/user for the whole user entity or is :x.specs/user a better name for the "entity", what's everyone's preferences here?
2017:06:25 20:16:27 jjttjj I'm personally convinced x.specs/user is clearly better than ::user which expands to :x.specs.user/user, just trying to get a second opinion before imposing my will on others.
2017:06:25 20:18:58 luchini We have been doing something similar to :x.specs.user/user in your example. It seemed like a natural choice at first and it quickly became a shared language across the team.
2017:06:25 20:20:19 luchini Same pattern you suggested: always the last s/def of the file and one ns per entity.
2017:06:25 20:21:32 luchini I’m also not sure if this will evolve into an idiom but surely has made the communication in our team much easier.
2017:06:25 20:22:36 jjttjj thanks for the input!
2017:06:25 20:26:50 luchini There’s an interesting side-effect that’s been great for the specifics of our project. We receive entity names from a remote system in a simple string. We can then convert the string into :x.specs.<string>/<string> and boom: we have our spec 🙂
2017:06:25 20:34:31 jjttjj @lucascs awesome yeah that's basically what I'm going for with the overall project set up but is there a reason you go with :x.specs.<string>/<string> instead of :x.specs/<string> for the composite entity at the end of the file? Because what you're saying could be done with either of those options just as easily right? (Sorry to painfully over analyze small details here)
2017:06:25 20:55:31 luchini Indeed it could. In practice the reasoning behind our choice was purely aesthetics. Kind of “everything related to the user entity is namespaced with :x.specs.user.” Otherwise we would need to always have the entity itself as a bit of an exception (defining it with :x.specs/user instead of autoexpanding with ::user).
2017:06:25 20:55:37 luchini Pretty much a personal taste thing
2017:06:25 20:56:05 luchini From a communication perspective, it’s one less cognitive level.
2017:06:25 20:57:25 jjttjj cool thanks again 🙂
2017:06:26 09:30:03 vikeri I have trouble running stest/check in a repl. It just returns an empty vector immediately, just as if the symbol was undefined. I have required the symbol and I can evaluate it in the REPL but it seems the check can’t find the symbol. How can I make sure that stest/check finds the function that I gave to it?
2017:06:26 11:14:48 deg I'm very excited about the goal of clojure.spec leading to improved error messages, but there is still a ways to go.
I just typo'd :require in
CompilerException clojure.lang.ExceptionInfo: Call to clojure.core/ns did not conform to spec:
In: [1] val: ((:requre [clj-http.client :as http] [raven.secrets :as secrets])) fails at: [:args] predicate: (cat :docstring (? string?) :attr-map (? map?) :clauses :clojure.core.specs.alpha/ns-clauses), Extra input
:clojure.spec.alpha/spec #object[clojure.spec.alpha$regex_spec_impl$reify__1200 0x1331d7d5 "
2017:06:26 11:16:18 mpenet there's room for improvement for sure, there isn't a lib that does human readable translation yet?
2017:06:26 11:18:19 deg It takes a lot less than that. Just a spec that had a list of valid keywords for the ns form could trivially pump out ":requre is not one of [:import :require :use ...]"
2017:06:26 15:48:15 alexmiller deg: It’s actually a lot trickier than this from a spec perspective due to the “form” structure of the dsl. If we were to design the ns api now, it is highly unlikely we would do it this way (would probably be a map). This turns out (due to the high fanout and nested differences) to be a particularly challenging case for spec to generically give a good error message.
2017:06:26 11:19:25 mpenet I guess it goes against the choice of making specs maps open to extension tho (which is arguably good or bad)
2017:06:26 11:20:00 deg The problem is that there are very few people in the community who are simultaneously likely to (1) hit this kind of error; (2) take more than a few seconds to spot the problem and feel the pain; and (3) have the comfort level to dive into the source of clojure core.
2017:06:26 11:20:02 mpenet even tho it's not a map here
2017:06:26 11:20:35 mpenet sure I agree, personally I wish we could specify strict sets for some kind of specs
2017:06:26 11:20:49 mpenet not sure how it's implemented for ns tho
2017:06:26 11:21:31 deg And you are right too, of course, that spec's philosophy is generally against closed lists of keywords. But, enough of us have written that for our own purposes. And, I'm sure that very few people would be against tight checking for the sake of error checking of special forms or canonical macros.
2017:06:26 15:49:23 alexmiller deg: just because it’s hard for spec to produce a good generic error here does not mean that the ns macro can’t take matters into its own hands instead to provide a customized response. not a done deal.
2017:06:26 11:21:43 mpenet yeah but here it actually is strict, my bad 🙂
2017:06:26 11:22:06 mpenet it's just the error that s cryptic
2017:06:26 11:22:17 deg Yup.
2017:06:26 11:23:16 deg And, relatively easy to fix any one error. The challenge is creating a framework where, anytime a newbie is bitten by one of these messages and reports it, it triggers a process that makes that message be forever better for the next user.
2017:06:26 11:23:37 luxbock if you specify the legal keywords for the NS form as a set, then that's still very much open to extension, no?
2017:06:26 11:24:02 luxbock just add more keywords to the set if the form ever gains more functionality
2017:06:26 13:34:59 wilkerlucio @jjttjj one other thing to consider is: what is user? because in my experience that can vary wildly inside of your system, a login user might require login and password while a registering user might require much more, think about this, and you end up having more specific entities (like: login-user and new-user) or you might drop the entities are all (that might vary a lot depending on much re-use you can give to those entities)
2017:06:26 14:09:15 bbrinck I’m a bit confused by the value of “in” when using “map-of”. When there is something wrong a value in the map, the in path doesn’t actually seem to point to that value. For example:
https://gist.github.com/bhb/6f06dd07bcf5b275a7d4faf3167bfc85
2017:06:26 14:11:47 joshjones @vikeri Are you sure you have fdef'd the function?
2017:06:26 14:15:54 joshjones @vikeri If you fail to either (1) fdef a defined function, or (2) give check a symbol which does not resolve to a function, it will exhibit the behavior you describe:
2017:06:26 14:17:10 stathissideris is there any way to provide a custom message in the output of s/explain?
2017:06:26 15:50:49 alexmiller stathissideris: no
2017:06:26 15:52:30 stathissideris thanks. Are there any plans for it, or is it out of scope/a bad idea?
2017:06:26 15:56:53 alexmiller there are no plans for it right now. the idea is that specs should be able to give you generically consistent errors. The explain-data error you get has enough info that you should be able to build custom user errors from that if desired - I believe Sean Corfield is someone doing a lot of this right now.
2017:06:26 16:17:10 stathissideris I get it. I guess I have a slightly more complex case where I use s/& to further validate a conformed value and s/explain reports the whole code of the anonymous function as the failing predicate, which is useful, but it would be even better to have a human readable message to say what was expected
2017:06:26 16:19:13 mpenet you can create custom Spec impl but it's risky
2017:06:26 16:19:51 stathissideris @U050SC7SV I don’t want it that bad, it’s more of a “nice to have” for me 🙂
2017:06:26 16:36:42 alexmiller s/& is indeed a special case and we are likely going to have to add support for custom forms there regardless (to support things like s/keys* which use it for implementation), but unlikely we would have a custom explain there beyond that
2017:06:26 16:53:24 stathissideris @alexmiller ok, thanks for the explanation and thanks for your efforts in general!
2017:06:27 09:50:35 stathissideris One way to make this a bit more self documenting would be for the second argument of s/& to be named function with a descriptive name instead of an in-place anonymous function
2017:06:26 14:17:50 vikeri @joshjones Hmm, I required the ns where the fn was defined but maybe that didn’t define the fdef. That is probably the issue.
2017:06:26 14:18:00 vikeri Thanks for the pointer
2017:06:26 14:30:05 bbrinck A simpler example: https://gist.github.com/bhb/c8d01c455494921a3698a9cf951272ff of how :in works with map-of. I think I’m beginning to understand how the path works. [:hi 0] means something like “construct a key/value pair from the map and the key :hi, then navigate to the 0th element of that k/v pair”
2017:06:26 15:51:32 alexmiller correct - maps are conformed as a sequence of map entries (spec’ed as a tuple of k and v)
2017:06:26 15:53:50 alexmiller however, that path won’t get you to the right place so I think that 0 is actually going to get you to the wrong place.
2017:06:26 15:57:10 alexmiller and that seems like a bug
2017:06:26 16:08:29 joshjones I've been looking at it for a few minutes now @alexmiller -- and I was going to say it looked like a bug too (but was not confident enough that my understanding about it was correct). the second element of the :in in this case is which element of the tuple spec caused the error. But the first element (`"hi"`) is coming from the every-impl spec, and this part particularly does not seem correct
2017:06:26 16:09:12 alexmiller It's tricky
2017:06:26 16:09:18 joshjones yes, it seems so
2017:06:26 16:10:48 joshjones putting some println's in the spec code:
2017:06:26 16:11:04 alexmiller I would consider this on top of the patch for 2080, which improves things in the explain case, whereas this is the conform case which doesn't currently use the kfn
2017:06:26 16:12:09 alexmiller Rich and I have an ongoing discussion/argument about what happens here :)
2017:06:26 16:13:39 joshjones i can see why, as it's not necessarily a clear cut answer as to what should be there. from one of your spec slides, I have that :in represents: "vector of specs from root spec to failing spec"
2017:06:26 16:15:54 joshjones @bbrinck it seems your confusion was justified 😉
2017:06:26 16:20:22 bbrinck @alexmiller @joshjones I appreciate the info. I will look at the tickets and patches mentioned above and file a follow up bug. I agree it’s tricky - especially in the case where the key is wrong. Given a nested data structure, how do I provide a path to a map key? AIUI, it doesn’t work if I consider a “path” to equal “a vector of keys” that would work with get-in.
2017:06:26 16:21:10 bbrinck What I’ve started to do is try to construct a function (maybe this already exists?) that takes some data + an :in path and retrieves the value.
2017:06:26 16:22:40 bbrinck That’s really what is driving this: I’d like to be able to take some (invalid) data + a problem (which contains the :in) and be able to use the unique :in value to get the problematic value, which is likely deep inside the original data
2017:06:26 16:23:33 bbrinck I had originally (naively) thought that I could use get-in for that function, but I think it’s more subtle than that, so I’m writing my own to use these special :in paths.
2017:06:26 16:30:27 alexmiller the idea is that you should be able to do that
2017:06:26 16:32:56 bronsa isn't that impossible if you also want to be able to reach keys?
2017:06:26 16:34:16 alexmiller it’s not possible in all cases
2017:06:26 16:34:55 alexmiller but I don’t know that it’s useful to specify [<key> 0] either
2017:06:26 16:38:44 bbrinck Yeah, tricky. For my cases, I’m considering changing my code to accommodate a “spec path” that allows me to reach keys i.e. writing new functions that work like get-in or update-in with this new type of path
2017:06:26 17:14:50 stathissideris just a thought, not a question: I think I’ll write a few “readable” and size constrained generators for strings, keywords and symbols so that my eyes don’t bleed whenever I try to read the output of s/exercise (and I know it’s a good thing that it produces “challenging” output)
2017:06:27 11:44:46 abhirag Hi, kinda new to clojure as well as clojure.spec
2017:06:27 11:45:30 abhirag Was trying out specifying the above function and stest/check hangs forever
2017:06:27 11:45:48 abhirag am I doing anything wrong here?
2017:06:27 13:44:03 alexmiller @abhirag how long did you wait? (presumably not “forever” :) it works for me, but takes a while.
2017:06:27 13:44:21 alexmiller “a while” being a couple minutes
2017:06:27 13:48:04 gfredericks 😳 why should that take a couple minutes?
2017:06:27 13:49:28 alexmiller a fair question :)
2017:06:27 13:50:07 abhirag I did try and restrict the count to 10 to reduce time, my dev machine might be slower than yours Alex, I will try again and try and not get afraid of the cooling fans revving up :)
2017:06:27 13:50:32 abhirag I also tried to restrict the :test-nums to 10
2017:06:27 13:51:03 abhirag But still the running time wasn't getting shorter
2017:06:27 13:51:50 gfredericks that definitely sounds like a problem to me
2017:06:27 13:51:57 alexmiller yeah, that seems weird
2017:06:27 13:52:20 alexmiller s/coll-of with a :count parameter will create a gen/vector with a size
2017:06:27 13:53:06 abhirag Yeah I did learn that from your guide :)
2017:06:27 13:54:19 abhirag And also if I just specify the function using fdef and don't give an implementation
2017:06:27 13:54:27 abhirag And then try check
2017:06:27 13:54:39 abhirag It just gives a null pointer exception
2017:06:27 13:55:54 abhirag No error message regarding the fact that no implementation was provided :P
2017:06:27 13:56:03 abhirag Thought I should mention that too
2017:06:27 13:56:08 abhirag :)
2017:06:27 13:56:46 alexmiller @gfredericks (g/sample (g/vector g/large-integer 10) 1000) seems like it should be close to what I’d expect and that’s fast
2017:06:27 13:59:05 gfredericks alexmiller: yeah that's what I was imagining
2017:06:27 14:01:23 alexmiller although it’s also going to put those in lists, then put that in a vector for the args, then invoke the function, then check the return is a list, reverse that list, and compare it to the input list
2017:06:27 14:32:25 alexmiller I’m still looking at your gen stuff too. It shouldn’t be that slow. I believe it’s the :kind list? that is causing the slow-down.
2017:06:27 14:34:33 abhirag yeah I had tried property testing in other languages before, that snippet was an overkill, my main motive with that was that if we already have a function tested and need to rewrite it for optimization etc. we could just use the old function to test it
2017:06:27 14:35:15 abhirag now I realize that with spec all I really need is that the rewritten function have the same spec
2017:06:27 14:37:06 wilkerlucio @abhirag I noticed on your REPL example that you didn't defined the my-reverse function
2017:06:27 14:38:32 abhirag yeah I realize that 🙂 that was just to get that null pointer exception, I was gonna log a ticket to make that error message better
2017:06:27 14:40:31 wilkerlucio cool, just wondered if was on purpose :)
2017:06:28 16:44:09 lwhorton for some reason i get complaints about unable to resolve spec :bar.alice/bob using the above method
2017:06:28 16:44:56 lwhorton might i be required to be explicit in :require [bar.alice :as bar.alice] so load-order is taken care of properly?
2017:06:28 16:50:58 seancorfield The ns containing that spec (`:bar.alice/bob`) must be loaded before you can use it in another spec. Otherwise the s/def will not have been executed.
2017:06:28 17:16:27 stathissideris is there a way to override the default generator of s/keys but only for a single key? (without renaming the key)
2017:06:28 17:19:47 bbrinck We have some JSON data coming in over the wire that uses strings as keys e.g. {"city" "Denver" "state" "CO"}.
We’ve been converting all strings keys to keywords in order to spec them with s/keys, but of course, doing the mapping takes a call to clojure.walk/keywordize-keys and then, if there is a problem that we want to report to the dev, we might need to convert back to strings to make a sensible error about the data.
In this case, do people do the conversion like this? It would seem like being about to have string keys would be useful e.g.
(s/keys :req-str [:location/city :location/state]) where :req-str is like :req and :req-str-un would be like :req-un, but for string keys
2017:06:28 19:27:32 alexmiller bbrinck: would be reasonable to file a jira enhancement for htis
2017:06:28 19:28:02 bbrinck @U064X3EF3 Can do!
2017:06:28 17:40:55 grzm I had some slow checks that I was trying to performance tune. One approach I wanted to try was swapping out the fdef spec during the stest/check run, similar to how you can swap out generators using the :gen option. I tried instrumenting the function and using the :spec option prior to the stest/check run, but that didn't have an effect. Does this seem like a reasonable thing to do? Is there a way to do it?
2017:06:28 19:28:40 alexmiller grzm: check takes a generator override map - did you try that?
2017:06:28 19:32:31 grzm It's not clear to me how providing a generator would replace the function spec: I'm not trying to change the values that are supplied to the function: I'm trying to change the spec that's applied to the function as a whole for the scope of the check. Or am I misunderstanding?
2017:06:28 19:32:56 grzm What would the generator be generating in this case?
2017:06:28 19:33:36 alexmiller oh, then instrument with a replacement spec should be what you want
2017:06:28 19:34:21 alexmiller however, calling instrument correctly is sometimes hard
2017:06:28 19:40:41 alexmiller in particular, anything that is being changed (including the thing whose spec is changing) needs to be in the list of instrumented vars. You should see that var in the return value from instrument as well - if you don’t, it wasn’t changed.
2017:06:28 19:41:36 grzm I'm working up a short example that describes what I'm trying.
2017:06:28 19:43:04 alexmiller here’s a spec replacement example:
(defn a [x] (if (zero? x) "a" (inc x))) ;; special behavior on 0
(defn b [y] (if (zero? y) 0 (+ (a y) 10))) ;; but b guards this
;; so use simpler spec when testing b
(stest/instrument `a
{:spec {`a (s/fspec :args (s/cat :a int?) :ret int?)}})
(stest/check `b)
2017:06:28 19:48:01 grzm
(defn a [x])
(s/fdef a
:args (s/cat :x int?)
:fn (fn [_] true))
(s/fdef b
:args (s/cat :x int?)
:fn (fn [_] false))
;; should pass
(stest/check `a)
(stest/instrument `a {:spec {`a `b}})
;; should fail
(stest/check `a)
2017:06:28 19:49:08 grzm What I've been trying is more direct. In your example, you're checking b which calls a. I'd like to check a directly.
2017:06:28 19:52:19 alexmiller I’m not sure the val of the :spec map will actually resolve that via the registry - did you try passing the actual spec there?
2017:06:28 19:57:14 grzm That doesn't work either, btw (for completeness)
2017:06:28 19:58:48 grzm I'm considering taking your spec course at the Conj. How far into the weeds are you going to get? Do you think you'll have a syllabus available?
2017:06:28 20:00:58 alexmiller the example I gave you above is from the spec course
2017:06:28 20:01:05 alexmiller so about that far :)
2017:06:28 20:03:51 grzm Gotcha. 🙂
2017:06:28 20:05:50 grzm Would you like me to open a ticket for this?
2017:06:28 20:07:30 alexmiller sorry, I was looking at the code. I understand why it doesn’t work, still thinking about what that means though.
2017:06:28 20:07:57 alexmiller the spec override is used when building the wrapped var in instrument
2017:06:28 20:08:35 alexmiller but check still uses the version in the spec for checking ret and fn, not the overridden spec
2017:06:28 20:09:03 alexmiller and indeed check can’t even see that - it’s just part of the check-fn on the instrumented var
2017:06:28 20:09:37 alexmiller seems like it would be totally reasonable to want to do something like this in the context of check though
2017:06:28 20:10:06 alexmiller instrumented vars will only catch invalid invocations, not invalid results.
2017:06:28 20:11:21 alexmiller so I think a ticket to add the ability for check to take spec overrides etc would be reasonable. we have talked about making things like generator overrides, spec overrides etc a consistent api used across exercise, exercise-fn, instrument, check, etc which would be in this area as well
2017:06:28 20:12:51 grzm Okay. I'll open one. (No need to apologize, by the way. I suspected as much, and even if you were doing something else, I'm grateful for the time and attention you're providing.)
2017:06:28 20:17:23 grzm I was similarly surprised about generator overrides with instrument and check. I expected generator overrides to apply globally. From what I can tell, they only apply to the checked function, not the functions called by the checked function.
2017:06:28 20:17:47 alexmiller it’s a little more subtle than that
2017:06:28 20:18:01 alexmiller they do apply globally, but there are a couple cases where they don’t get picked up
2017:06:28 20:18:05 alexmiller there are some tickets on this
2017:06:28 20:19:10 grzm Happen to have pointers to those tickets handy to see if they cover my use case?
2017:06:28 20:29:36 grzm Are you guys happy with Jira? Is it worthwhile for someone to make a Clojure syntax highlighter plugin?
2017:06:28 23:51:38 grzm Here's a gist showing the instrument generator override behavior I mentioned above. https://gist.github.com/grzm/0ada176caee5bcdab39d820577ed3823 I'll bring this up in #clojure-spec as well and continue discussion there unless you bring in back here. Thanks again for your help today.
2017:06:28 17:54:55 lwhorton @bbrinck what’s your concern, performance?
2017:06:28 17:57:47 bbrinck Performance and the fact the validated data doesn't match the original, so error messages either need to be converted back or are somewhat confusing.
2017:06:28 17:58:26 lwhorton as always with performance i would test and verify that it’s going to be an issue before worrying about doing 2x conversions
2017:06:28 18:00:31 grzm @bbrinck with respect to the validated data doesn't match the original, you're already doing a conversion (I suspect) from JSON objects to Clojure maps. The similarity between the two (including syntax) obscures this a bit.
2017:06:28 18:01:57 bbrinck @lwhorton. Fair point about perf. My greater concern is complexity of implementation when converting to validate and then back to report for what seems like a common case of string keys (when working with JSON).
2017:06:28 18:03:24 bbrinck Frankly it's not a huge cost. Just wondering if this workaround is how others are tackling this case.
2017:06:28 18:03:46 lwhorton maybe im looking at it wrong, but i dont see complexity around (keywordize-keys json) and (string-keys edn)
2017:06:28 18:04:51 lwhorton if you convert to edn, run a spec/validate and it fails, then convert it back to json and make a POST… that all seems fairly straightforward?
2017:06:28 18:06:32 bbrinck I'd ideally like to give error messages that are relevant to the original data. Imagine an API that accepts JSON. If I want to give an errr message about non conformance, I want this to be about strings not keywords.
2017:06:28 18:07:16 bbrinck I suppose I can explain-data and then walk each problem and convert back to strings. The n format the data?
2017:06:28 18:08:14 lwhorton or hold a reference to the original json, right?
2017:06:28 18:08:52 bbrinck But that's just the whole thing right? That's not the specific section that has the problem
2017:06:28 18:08:57 lwhorton i suppose if you wanted to get into the exact location where a conform failed, yes you might have to do explain-data
2017:06:28 18:09:24 bbrinck You'd need to walk each problem and reconvert, then reformat. But you are correct, it's not a huge amount of work
2017:06:28 18:09:57 lwhorton you could also write your spec to be some form of s/cat :value string? ... etc.
2017:06:28 18:10:06 bbrinck And I reformat I just mean reimplement 'explain-str'
2017:06:28 18:10:08 lwhorton but that might be way more complex than what is already offered by (s/keys)
2017:06:28 18:10:43 lwhorton (essentially spec out the JSON implementation)
2017:06:28 18:11:22 lwhorton i would also look around for some libs that might already exist for this sort of thing
2017:06:28 18:11:38 bbrinck Right. Thanks for the ideas! Good to know I'm not missing an obviously simpler way :)
2017:06:28 18:12:34 lwhorton with plumatic.schema there was coersion, but i’m not sure if spec went down that path as well
2017:06:28 18:16:59 bbrinck AIUI, spec is avoiding coercion but I could be mistaken. I might just end up implanting a version of 's/keys' that accepts strings (and produces a generator that uses string keys). Could be useful for JSON validation
2017:06:28 18:17:31 bbrinck Note I agree the specs themselves should be keywords. I'm just thinking that they could be included in maps as strings.
2017:06:28 18:17:39 lwhorton I’ve got a question for you 🙂 … have you ever used branching in a spec to define a spec? Say you have a key :foo and :bar, but :bar can pull from set A or B of valid values. Is there a way to make :bar read :foo and decide which is a valid spec?
2017:06:28 18:19:15 grzm @lwhorton Does multi-spec do what you want? Or are you getting at something different?
2017:06:28 18:22:25 lwhorton hah, that looks exactly like what I’m talking about
2017:06:28 21:32:28 dealy spec newbie here, please excuse my ignorance. I have a data structure which I've defined a spec for. I've just needed to add a channel as one of the items. How do I update my spec for this type of a field?
2017:06:28 21:44:59 seancorfield Pretty sure there’s no way to spec a channel (same as there’s no way to spec atom/delay/etc) so you can just use any? (or not provide an actual spec for that key).
2017:06:28 21:46:42 seancorfield Spec’ing something to be a channel wouldn’t be very useful: you couldn’t generate values for it (since it has type channel).
2017:06:28 21:47:30 dealy oh, ok, there's a lot of spec I'm still pretty confused about. So, spec'ing data structures is primarily for building up things from basic types (ints doubles bools and functions) as opposed to interfaces and complex objects?
2017:06:28 21:49:19 seancorfield Hmm, not really…
2017:06:28 21:50:23 seancorfield You can spec domain-level entities — so you can give things meaning — but you need to think about what you’re trying to achieve with spec. Why are you spec’ing a particular structure? What are you going to use the spec for?
2017:06:28 21:50:57 seancorfield There’s not much point in spec’ing everything and I think it’s a common misunderstanding to try to treat spec like a type system and “spec everything”.
2017:06:28 21:52:25 seancorfield You also need to consider whether you want specs to be “generatable” (you probably mostly do) so you need to avoid specs with types that can’t be generated (since they’re not very useful, in general).
2017:06:28 21:54:11 dealy yea, that makes sense. I haven't gotten into that part yet. Still only using it as a way of just making sure my datastructure remains in the format I expect/need.
2017:06:28 21:57:23 seancorfield Are you using conform or valid? or some other way of checking against the spec?
2017:06:28 21:58:03 dealy valid?
2017:06:28 21:58:38 dealy this is all part of my first GUI app using re-frame so its validating the main app-db as the UI changes
2017:06:28 21:58:56 seancorfield Since spec checks are all runtime, if you really need to “assert” your data structure has a specific key, you can always spec it as any? and if you use something as a channel and it isn’t, it’s going to blow up at runtime anyway.
2017:06:28 21:59:09 dealy its a lot to learn all at once
2017:06:28 21:59:13 seancorfield I would be a bit suspicious of data including a channel tho’…
2017:06:28 22:00:00 dealy normally yes, but it is an artifact of a timer that needs to be cleaned up later
2017:06:28 22:00:01 seancorfield after all, that’s not data you can serialize and pass around so it’s not really data…
2017:06:28 22:00:19 seancorfield (I may be taking a bit of a purist view here — not sure how others feel?)
2017:06:28 22:11:31 lwhorton can someone point out my (probable) misuse of multi-spec? what i’m trying to go for, in plain english, is to specify a map where some key’s spec depends on the value of another key. If key a is one of “cat” or “dog”, key b should be spec’d differently in either case.
2017:06:28 22:14:35 lwhorton I would expect to be able to run something like (s/conform :: foo {:a "cat" :b ...}) or (s/conform :: foo {:a "dog" :b ...}) and have the response [:cat-only-thing …] or [:dog-only-thing …] or [:cat-or-dog-thing ...] depending on the value of b.
2017:06:28 22:14:39 lwhorton Is this totally off?
2017:06:28 22:24:02 seancorfield I think your multi-spec needs to be a hash map that has a key :a
2017:06:28 22:24:49 seancorfield So ::b would be spec a hash map — do ::cat-thing etc spec hash maps?
2017:06:28 22:27:08 lwhorton the call to s/multi-spec event-type in that example is calling on a defmulti event-type, unless I’m misunderstanding
2017:06:28 22:27:53 seancorfield The :event/type key is common to all the (hash maps) that are valid for that set of specs.
2017:06:28 22:29:02 seancorfield In your example, you should have a hash map that has a key :a that is the “tag” and it can have the value "cat" or "dog" and the defmethods should returns specs for the cat, or dog, hash map that includes the tag :a.
2017:06:28 22:29:47 lwhorton oh you mean the object on which i’m invoking the spec? i.e. s/conform ::foo {:a "cat" (or "dog") :b something-else}?
2017:06:28 22:30:37 seancorfield Right… so your defmethods should be (s/keys :req-un [::a :cat/b]) and (s/keys :req-un [::a :dog/b])
2017:06:28 22:31:23 seancorfield then the multi-spec selects which spec to use based on the tag :a
2017:06:28 22:31:29 lwhorton ah so i guess I cannot get away with what I was originally trying to do
2017:06:28 22:31:58 seancorfield Depends what ::cat-thing, ::dog-thing, and ::shared-things are…
2017:06:28 22:32:18 lwhorton which is something like “if the multimethod matches on ‘cat’, return a spec which is an (s/or …)
2017:06:28 22:32:46 lwhorton so you are saying ::cat-thing, ::dog-thing, ::shared-things would have to be maps containing the key :a, correct?
2017:06:28 22:34:43 seancorfield What are you trying to do?
2017:06:28 22:35:12 seancorfield If you explain how you want :b to vary, I can probably show you…
2017:06:28 22:35:17 lwhorton i have a piece of data that looks like this {:status "" :operation "" :state ""}
2017:06:28 22:35:41 lwhorton actually let me thin that down even more, i have a map {:status "" :operation ""}
2017:06:28 22:36:16 lwhorton operation can belong to any member of the set #{:idle :downloading :patching}
2017:06:28 22:37:41 lwhorton but depending on the value of :operation, the spec for status should be “limited”. that is, given the idle operation, the available statuses should only be one of #{:a :b :c}, and given the downloading operation, statuses should only be one of #{:c :d :e} (note the overlap)
2017:06:28 22:37:58 lwhorton i was trying to use spec to enforce this pattern-matching behavior
2017:06:28 22:38:57 lwhorton the (s/or ..) comes into play because there is some overlap between :downloading / :patching / :idle, but only 1-2 states
2017:06:28 22:39:08 seancorfield Sure… so you need to spec the different types of status values, e.g., (s/def :downloading/status #{:c :d :e}) and (s/def :idle/status #{:a :b :c})
2017:06:28 22:40:01 seancorfield and then your defmethod would return (s/keys :req-un [::operation :downloading/status]) for the downloading branch and (s/keys :req-un [::operation :idle/status]) for the idle branch
2017:06:28 22:41:05 lwhorton oof, so all this pain and confusion because i was trying to do an s/or (to union) where I should have just defined the full sets outright, regardless of overlaps?
2017:06:28 22:42:49 lwhorton ill give that a whirl and get back to you.. regardless of the outcome thanks for your help
2017:06:28 22:45:41 lwhorton indeed it does, and a gen/sample gives me 10 good values !
2017:06:28 22:46:22 lwhorton well it’s good to know now that defmethod has to return “the original match” + the spec, not some hob-goblin pseudo spec
2017:06:28 22:46:26 lwhorton many thanks for your guidance
2017:06:28 22:46:53 seancorfield Thanks for the question — I hadn’t played with multi-spec before and this gave me a good opportunity to try it out.
2017:06:28 22:47:58 seancorfield The different branches could also have different keys etc.
2017:06:28 22:48:28 seancorfield And, indeed, doesn’t have to be a hash map but then the tag function would need to be something other than a simple keyword.
2017:06:28 22:49:10 lwhorton so does the tag arg on (multi-spec spec tag) have to be identical to the fn/kwd in defmulti name fn/kwd?
2017:06:28 22:50:09 lwhorton not really sure why the tag is there.. unless its a way to generate fields under a different key from the one of the multi-match?
2017:06:28 22:53:16 bbloom random thought: i kinda wish there was a way to alias keys, just as there is a way to alias attributes in datomic - would help with refactoring and “subtyping”
2017:06:28 22:58:11 seancorfield @lwhorton I believe the tag is needed in multi-spec so that the generator can figure out what key to work with.
2017:06:28 22:58:29 seancorfield (since defmulti could take a function etc)
2017:06:28 22:59:09 lwhorton i see .. if you still have that ^above code in a register, what happens when you try to generate some samples?
2017:06:28 22:59:18 seancorfield If defmulti was some complex selector function, the argument passed to multi-spec would have to kind of reverse that I think…
2017:06:28 22:59:45 lwhorton do you end up with {:data-state "" :data-status {:data-state "" :data-status ""}?
2017:06:28 23:00:34 lwhorton interesting, i probably typo-d somewhere
2017:06:28 23:00:47 lwhorton this spec gen stuff is so great
2017:06:28 23:03:35 seancorfield Yup, I pretty much always try to s/exercise my specs as a sanity check, and also to see the sort of crazy stuff they can produce. We use regex fairly heavily so we rely on @gfredericks test.chuck which has a generator for regex string specs — awesome stuff!
2017:06:28 23:16:28 spieden great clojure.test integration too =)
2017:06:29 10:52:36 harrigan Is the following behavior for s/& expected?
2017:06:29 13:21:54 alexmiller @harrigan ^^ I am bad at notification
2017:06:29 13:22:55 gmercer thanks Alex - any idea about the double 'exec test' on the
(s/conform (s/& (s/* (fn [i] (prn "pred of " i ) (int? i))) (fn [x] (prn "exec test of " x) (= (count x) 1))) [1])
2017:06:29 13:24:45 alexmiller Can happen - there is no guarantee on how many times a predicate may be called
2017:06:29 13:25:13 alexmiller I'd have to spend more time to explain why here
2017:06:29 13:25:26 alexmiller To understand why that is
2017:06:29 13:25:28 gmercer nw - but interesting!
2017:06:29 15:12:09 stathissideris has anyone noticed multi-spec being slow to generate values? it also sometimes fails randomly
2017:06:29 15:14:01 stathissideris I’m doing a gen/sample with a size of 1, and I get ExceptionInfo Couldn't satisfy such-that predicate after 100 tries.
2017:06:29 15:14:14 stathissideris but not always
2017:06:29 15:15:00 stathissideris the spec is a multi-spec that contains maps with strings, nothing fancy
2017:06:29 15:15:12 stathissideris one of the keys is a multi-spec itself
2017:06:29 15:15:23 gfredericks the such-that exceptions happen when you have an s/and somewhere and the generator for the first clause isn't likely to satisfy the remaining clauses
2017:06:29 15:19:21 gfredericks in the worst case the problem is solved by writing a custom generator for the (s/and ...)
2017:06:29 15:19:42 gfredericks (the slow generation is probably a different symptom of the same problem)
2017:06:29 15:20:10 stathissideris the only suspect thing in there was this:
(def email-re #"[a-zA-Z0-9._%+-]
2017:06:29 15:20:26 stathissideris but I removed it and still got the same behaviour
2017:06:29 15:20:34 gfredericks well that looks like it already has the custom generator
2017:06:29 15:20:47 stathissideris correct
2017:06:29 15:21:07 gfredericks is it possible you're using some higher-level spec that expands to an s/and?
2017:06:29 15:22:47 stathissideris hm, I think s/merge might
2017:06:29 15:24:33 gfredericks 🤔 that sounds strange
2017:06:29 15:25:45 stathissideris I think I know what it is: I’ve been using this macro to disallow “unknown” keys (I know it’s discouraged):
2017:06:29 15:26:06 stathissideris I don’t think it plays well with generation
2017:06:29 15:26:27 stathissideris removing it fixed my problem
2017:06:29 15:26:33 stathissideris (it seems)
2017:06:29 15:26:51 gfredericks I would've expected s/and to be used there instead of s/merge😕
2017:06:29 15:27:18 gfredericks if that's equivalent, I bet the s/and would actually generate just fine
2017:06:29 15:27:25 gfredericks since I don't think s/keys generates any extra keys
2017:06:29 15:37:05 l0st3d I'm having a problem with macro expansion and specs ... sometimes the specs are not checked
2017:06:29 15:37:17 l0st3d has anyone else seen this?
2017:06:29 15:37:47 gfredericks specs on macros?
2017:06:29 15:37:52 l0st3d i'm having trouble working out the circumstances where it works or doesn't
2017:06:29 15:37:55 l0st3d yes
2017:06:29 15:38:12 gfredericks these are specs about the compile-time forms, or the runtime data?
2017:06:29 15:38:23 l0st3d i've narrowed down this test case:
2017:06:29 15:39:11 l0st3d which fails if i use lein test
2017:06:29 15:39:28 l0st3d but passes if I load the code with C-c C-k in cider
2017:06:29 15:39:49 l0st3d @gfredericks compile-time forms
2017:06:29 15:40:05 gfredericks "load the code in cider" means actually running the test, not just compiling it?
2017:06:29 15:41:09 l0st3d no ... load the code in cider means running cider-load-buffer
2017:06:29 15:41:41 l0st3d and then either running the tests with C-c C-t C-n ... or running the tests at the repl
2017:06:29 15:41:50 stathissideris @gfredericks thanks, I’ll give it a try
2017:06:29 15:41:52 l0st3d with run-tests
2017:06:29 15:42:11 gfredericks @l0st3d okay, so if you add (is false "dangit") as a second assertion in your test, it will fail there but not with the first assertion?
2017:06:29 15:42:47 l0st3d yup
2017:06:29 15:43:07 gfredericks okay, I have no idea then
2017:06:29 15:43:28 l0st3d it reports that the test was run, but that succeeded ...
2017:06:29 15:44:50 l0st3d ok ... @gfredericks ... thanks for your help
2017:06:29 15:45:20 l0st3d i'm stuck then
2017:06:29 15:50:33 lwhorton spec is my new addiction. how did i ever do front-end development without you, my love?
2017:06:29 15:51:36 lwhorton if i need to build a feature but the backend team is busy and won’t get to an api data endpoint in time, no worries: spec can auto-generate a static api response for me and I can go on my way building the html.
2017:06:29 16:02:49 gfredericks @l0st3d if I were you I would try to reproduce the problem without cider; I assume you've tried restarting the cider process?
2017:06:29 16:03:51 l0st3d yes ... the 3 files I posted above always fail to check macro expansion when run from the terminal using lein
2017:06:29 16:04:23 l0st3d cider seems to fix it ... which is how i didn't notice at first 😉
2017:06:29 16:10:08 gfredericks oooh
2017:06:29 16:10:12 gfredericks I think I read the whole thing backwards
2017:06:29 16:10:23 gfredericks due to ambiguity of "fail" and "pass" in this case
2017:06:29 16:10:31 gfredericks so it's leiningen that has the problem, not cider
2017:06:29 16:11:35 gfredericks in that case I'd try reproducing the problem using just java -cp ... clojure.main <file>
2017:06:29 16:22:54 l0st3d yeah ... started doing that ... what's the minimal classpath that you need to start a repl with 1.9.0-alpha17?
2017:06:29 16:28:10 gfredericks you could try running lein classpath and using that
2017:06:29 16:28:32 gfredericks which reminds me that if you have deps in your ~/.lein/profiles.clj then it can be helpful to turn that stuff off via LEIN_NO_USER_PROFILES=1
2017:06:29 16:28:56 l0st3d fair point
2017:06:29 16:29:02 l0st3d thanks
2017:06:29 16:29:14 l0st3d there's nothing in my lein profile
2017:06:29 16:30:03 l0st3d ah ....
2017:06:29 16:30:08 l0st3d it's the ~'s
2017:06:29 16:30:21 l0st3d zsh is expanding the first ~ but not the rest
2017:06:29 16:30:50 l0st3d so it's not finding the jar on the class path ... ~/.m2 is not a dir
2017:06:29 16:30:52 l0st3d doh
2017:06:29 16:31:09 gfredericks 😭
2017:06:29 16:31:38 gfredericks computers are terrible
2017:06:29 16:35:29 l0st3d 😉
2017:06:29 16:35:40 l0st3d yeah ... and the minimal case there works fine
2017:06:29 16:53:23 rickmoynihan are there any plans to attach doc strings to s/defs?
2017:06:29 17:29:19 mpenet In theory you could hack your own doc registry, but it d be nicer to have this built-in
2017:06:29 17:30:44 mpenet Proper metadata support would be nice, not just docs imho
2017:06:29 17:31:35 rickmoynihan one other small issue I have is that if I have two namespaces foo.bar.baz and foo.bar.baz.impl and I want to add a spec alias for foo.bar.baz so I can do ::baz/blah inside foo.bar.baz.impl there doesn’t appear to be a way to do it when foo.bar.baz requires …impl. Basically I’d like a version of alias that doesn’t require the namespace to be loaded.
2017:06:29 17:33:02 rickmoynihan not sure if there’s a solution to that one yet… other than fully qualify the keywords
2017:06:29 17:34:51 rickmoynihan I guess that would also allow you to alias keywords to namespaces which don’t exist — which would also be useful as keywords don’t have to map to namespaces
2017:06:29 17:39:49 alexmiller @rickmoynihan @mpenet yes, Rich, Stu, and I all want this, just haven’t gotten to it yet (and I think it’s trickier than it appears). would be happy to see someone dig in and suggest alternatives to move it forward
2017:06:29 17:40:24 rickmoynihan @alexmiller: doc strings or aliasing, or both?
2017:06:29 17:40:32 alexmiller that was about doc strings
2017:06:29 17:41:13 rickmoynihan and other metadata?
2017:06:29 17:41:16 alexmiller on aliasing, Rich has some thoughts on an approach to making keyword aliasing better, but I have not gotten info on it yet
2017:06:29 17:41:21 mpenet What's so tricky?
2017:06:29 17:41:37 alexmiller @rickmoynihan maybe. needs someone to dig in to see.
2017:06:29 17:41:45 alexmiller @mpenet where do you put them?
2017:06:29 17:42:14 alexmiller a spec can be a keyword reference to another spec - in that case, there is nowhere to hang meta
2017:06:29 17:43:10 mpenet Could be a simple metadata registry, atom with a map from kw -> metadata no?
2017:06:29 17:43:54 alexmiller I don’t think it’s useful to do so here in this chat, but someone needs to evaluate what the options are. if you want to write something up in the ticket or elsewhere, please do.
2017:06:29 17:44:35 mpenet Sure, I can do that tomorrow morning (cooking dinner atm :))
2017:06:29 17:44:46 alexmiller off the top of my head some options include: explicit support in the spec protocol, hanging meta off the values in the spec registry, wrapping the values in the spec registry, creating a separate registry for docs or meta
2017:06:29 17:45:04 alexmiller these all have various pros and cons
2017:06:29 17:45:56 alexmiller they need to be checked against all the things that can be specs - sets, pred functions, anon functions, keyword refs to other specs, things that implement the Spec protocol, etc
2017:06:29 17:47:28 alexmiller if other kinds of meta, what are some examples? would some meta be set automatically by spec or is it all custom per use?
2017:06:29 17:47:56 alexmiller all of this stuff needs to be answered
2017:06:29 18:00:34 rickmoynihan yeah it seems like quite a big job - but spec has a pretty big surface area so not surprising
2017:06:29 19:17:08 lwhorton i’ve looked around at s/conformer and browsed a bit for conforming data with spec… but it seems like a not-great idea to use spec as a sort of “api parser”. is this correct?
2017:06:29 19:17:56 lwhorton i.e. if a service provides me some json, i can’t really write a spec on the client saying “this is my shape”, then somehow use spec to convert json -> my shape, can I?
2017:06:29 19:18:04 seancorfield The general advice is don’t coerce/transform data via s/conform unless you intend all potential clients of your spec to need that conversion!
2017:06:29 19:19:19 seancorfield At World Singles, we do some coercion with our API specs, but we already have JSON converted to Clojure data structures as input (using JSON middleware for Ring), and so we only coerce strings to keywords and a few other “minor” things in our specs.
2017:06:29 19:20:03 seancorfield So I’d recommend keeping the JSON -> Clojure data step separate from the Clojure data -> conformed input step.
2017:06:29 19:20:33 lwhorton okay, so considering that advice I guess i will write custom (parse-*-entity) which converts edn (xformed in a handler) from the wrong shape to edn of the client shape.
2017:06:29 19:21:29 lwhorton would be really neat if there’s a tool similar to spec that handles this, though i’m not really sure what it would look like. the real answer is have a properly shaped api response, but beggars cant be choosers
Why does the first print give [:query [[:prop :one] [:prop :two] [:prop :three]]] but the second give (:one :two :three) (sequence not a vector)?
2017:06:30 08:32:49 danielstockton If query is spec'ed as being a vector? shouldn't unform also return a vector?
2017:06:30 10:01:41 jwkoelewijn hi there, I’m having some troubles regarding circular refs in my specs. I have a spec defining an office which has employees in the office ns, and I have an employee (in an employee ns), which should have an office. As you cn guess, this results in circular dependencies (or when I don’t require one of the namespaces, it results in Unable to resolve spec. Is there any recommended method to deal with issues like this?
2017:06:30 13:44:00 alexmiller @jwkoelewijn code would help. What you're describing should be possible
2017:06:30 14:01:37 rickmoynihan Does anyone know of any ring middleware that pretty prints ex-info‘s that raise spec explain-data? Thinking of something like prone for dev but for spec problems.
2017:06:30 14:02:09 rickmoynihan prone currently gives me a huge wall of text
2017:06:30 14:02:11 bbrinck Has anyone worked on a partial spec for specs? Or a test.check generator? I’m mostly interested in generating specs rather than conformance or validation
2017:06:30 14:52:50 alexmiller bbrinck: https://dev.clojure.org/jira/browse/CLJ-2112 is about specs for specs. Those also are a step towards spec generators which I’ve also done some more work that’s not public yet.
2017:06:30 15:01:36 bbrinck Excellent news. I’m voting for that one now 😉
2017:06:30 15:02:15 bbrinck I may work on some generators soon. I’m guessing it will be a subset of what you’ve done, but it’ll be public, so I’m happy to share in case it’s useful.
2017:06:30 15:25:38 alexmiller I’ve in particular worked on the generators in the direction of testing spec itself. Namely, if you have spec specs with working generators, you can generate specs, then use those to generate data, then validate that the generated values are valid according to the generated specs
2017:06:30 15:26:47 alexmiller the last time I worked on this I mostly convinced myself there was a bug in multi-spec gen for the non-keyword case that was giving me grief, but I never ran it down
2017:06:30 18:50:38 eraserhd I wonder how you spec that s/conform returns :clojure.spec/invalid as a valid result.
2017:06:30 14:06:30 bbrinck @rickmoynihan This isn’t directly applicable, but data-frisk includes some nice-looking spec messages. I wonder if that could be ported.
2017:06:30 14:07:01 bbrinck Do you need to return explain-data or could your raise an ex-info that contains explain (or some other string representation of the errors)?
2017:06:30 14:10:20 bbrinck I’m just wondering if maybe you could solve the issue by formatting the errors at the source. That might not be desirable though.
2017:06:30 14:14:23 danielstockton Can I destructure metadata? For example, if I have a vector [1 2 3] which has meta {:some :meta} -> [:vector [1 2 3] :meta {:some :meta}].
I can't work out how I'd spec this.
2017:06:30 14:15:24 mpenet @danielstockton with a predicate I guess
2017:06:30 14:16:37 danielstockton @mpenet Tried that but I couldn't get destructuring as above when conforming. My understanding of spec is still fairly superficial.
2017:06:30 14:19:05 mpenet you can do this with a custom conformer I think
2017:06:30 14:20:53 danielstockton Ok, I'll look into that, thanks.
2017:06:30 14:21:27 danielstockton I don't need the conformed output exactly as above but I need to be able to easily identify the two parts.
2017:06:30 14:22:48 mpenet (s/conform (s/and vector? (s/conformer #(hash-map :vector % :meta (meta %))))) (with-meta {} {:foo 1}) -> {:vector [] :meta {:foo 1}}. this example conformer is a bit broken but that's the idea for the coercion part
2017:06:30 14:28:20 danielstockton Cool, why do you say it's broken?
2017:06:30 14:29:24 mpenet it's ok actually, before the edit it was 🙂
2017:06:30 14:29:27 danielstockton Am I right that conformer takes a second argument to unform? This might address the issue I had above with unforming to a seq.
2017:06:30 14:29:35 mpenet true
2017:06:30 14:30:09 mpenet never used unform but yes, it might work
2017:06:30 14:30:10 danielstockton Still curious to know exactly why it unforms to a seq. I noticed for simpler specs (just vector?) it unforms correctly so it must be a side effects of something.
2017:06:30 14:30:16 mpenet I think that's a bug
2017:06:30 14:30:56 danielstockton Should I be calling it a vector in the conformed output or is it a 'tuple'?
2017:06:30 14:37:50 danielstockton If it is a bug, I can't work out where it comes from.
2017:06:30 14:38:21 wilkerlucio @danielstockton for that case I think a tuple makes more sense than a cat, a cat is for more regexy stuff
2017:06:30 14:39:17 wilkerlucio I think it unforms to a seq because you are using a cat, which is for regexy stuff, and I believe it will always conform to a seq
2017:06:30 14:39:42 danielstockton @wilkerlucio Which cat are you referring to?
2017:06:30 14:40:01 danielstockton The only ones I see are deeper in the spec
2017:06:30 14:40:19 wilkerlucio or I'm looking at the wrong place?
2017:06:30 14:41:30 danielstockton I don't think the query I'm unforming has any idents: [:query [[:prop :one] [:prop :two] [:prop :three]]]. The [:prop :one] is the conformed value for a :prop::query-expr which is just a keyword?
2017:06:30 14:41:55 danielstockton The original query is just [:one :two :three]
2017:06:30 14:42:48 danielstockton I did think it might be related to a cat or similar, somewhere, but I couldn't explain it.
2017:06:30 14:44:00 wilkerlucio ah, I think I was reading it wrongly, sorry, so it seems something is not aligned between conform and unform?
2017:06:30 14:45:04 danielstockton I mean, perhaps I just misunderstand unform because it doesn't promise to give back exactly what you put it, only undo the destructuring..
2017:06:30 14:45:29 danielstockton I'm just curious exactly what's causing this behaviour.
2017:06:30 14:55:36 alexmiller it’s unclear to what is not working as you expect from the back chat - can you restate?
2017:06:30 14:56:23 danielstockton Why does the first print give [:query [[:prop :one] [:prop :two] [:prop :three]]] but the second give (:one :two :three) (a lazy sequence, not a vector)?
2017:06:30 15:01:13 alexmiller looks like this is sufficient repro:
2017:06:30 15:02:17 danielstockton Yep thanks, I made it look unnecessarily complicated.
2017:06:30 15:03:18 alexmiller so in conform, the general approach is to avoid rebuilding a collection if at all possible, so there you just end up with the original vector on conform
2017:06:30 15:03:46 alexmiller unform of a regex spec however will always (I think) yield a seq
2017:06:30 15:05:17 alexmiller unform of an s/and walks the preds in reverse
2017:06:30 15:09:02 danielstockton Make sense, so it's the s/+ specifically. And if I wanted unform to yield the same collection type, I'd use a customer conformer with a second argument?
I'm trying to use spec to implement a parser and I want to be able to go from query <-> ast. I don't know if this is a good use case for spec or not.
2017:06:30 15:10:20 alexmiller the broader problem here is: how do I use a regex spec but constrain to vectors and this is a known problem area that Rich and I have discussed at length as it came up a lot in spec’ing the core macros
2017:06:30 15:11:02 alexmiller there is currently no simple solution that works as you’d like for all of conforming, generation, explain, and unconform (although you can make subsets of those work)
2017:06:30 15:11:58 alexmiller I’m trying to work out if in this case you could actually somehow supply a custom unform function to the vector? pred as that’s not something I’ve tried
2017:06:30 15:23:32 alexmiller looks like there is no easy way to do so although there is certainly the potential to do so
2017:06:30 15:43:09 danielstockton Ok, I'll keep my ear to the ground.
2017:06:30 18:53:27 eraserhd Can i recursively override a generator? e.g. pass a function to s/spec's :gen that overrides just the recursive part, and close over such this also overrides the recursive part, making a depth-limited generator?
2017:06:30 18:54:56 eraserhd I've done this from scratch with clojure.test.check.generators, just trying to see if I can reuse most of what spec would have done.
2017:06:30 22:05:35 alexmiller Recursive generators are already depth limited based on the dynamic var in spec
2017:06:30 22:05:46 alexmiller In case that solves the problem
2017:07:01 03:37:03 bbloom saw some javascript/flowtype code today that has this type:
2017:07:01 03:38:33 bbloom sure, it isn’t checked statically - but at least it represents the actual right thing!
2017:07:01 17:57:19 arohner @bbloom soon 🙂
2017:07:01 17:57:50 arohner well, not soon, but I’m actively working on it
2017:07:01 18:25:27 bbloom 🙂
2017:07:02 02:13:34 wildermuthn I recently read an article about how Reddit (or parts of it?) are switching to TypeScript, and one of their main criteria for choosing a compile-to-js lang was it needed to be typed. Been thinking about that, and having a lack of experience with typed languages, what other Clojurists think about that argument, particularly with respect to Clojure Spec. Link: https://redditblog.com/2017/06/30/why-we-chose-typescript/ Any articles or thoughts out there about how Spec fits into this question of typed vs. dynamic?
2017:07:02 10:59:15 carocad wildermuthn: you should probably check the core.typed library from clojure. It adds static type annotations on top of clojure code. Also afaik one of the current initiatives in google summer of code for clojure is precisely to use specs to create those static types. I think another project about that is spectrum which is already pursuing that initiative.
2017:07:02 18:49:34 seancorfield I think developers either naturally lean towards types or not, and that influences their language choices. At World Singles, we’ve historically worked with dynamically typed languages and our forays into typed languages (C#, Scala) have not felt very “natural” to us. Since we adopted Clojure in 2011, we’ve also tried Schema and core.typed, and given up on them, then tried them both again and given up on them again. We’re heavy users of Spec, however, and we focus mostly on specifying data structures and using conform to validate (with a little coercion).
2017:07:02 18:53:12 seancorfield Myself, personally, I’ve worked with both statically typed and dynamically typed languages across most of my career and, whilst I like the idea of static types, I don’t generally like the reality of them. I was an early advocate of Haskell (because of my PhD work on FP language design in the 80's) but I have something of a love/hate relationship with Haskell’s type system 🙂 So, overall, I think I’m one of those that leans naturally to dynamically typed languages.
2017:07:02 19:30:55 carocad @U04V70XH6 would you mind sharing the reason why you abandoned the type annotations for Clojure? Although I have not used them I don't see a reason to abandon them completely. I think specially for libraries api it could be quite useful, even though personally I would prefer core.typed to infer the type of my arguments based on their spec I.e, go from the more generic approach to a more specific one and definitely avoid having to do both
2017:07:02 19:32:57 seancorfield It’s very hard to provide type annotations that satisfy core.typed for some idiomatic Clojure code. Also, core.typed doesn’t do a great deal of type inference so you have to annotate a lot of code to satisfy core.typed. The former means you often need to rewrite Clojure to a less idiomatic style to satisfy the type checker. The latter means you end up with a lot of “noise” in your code.
2017:07:02 19:34:09 seancorfield I don’t remember how many of my discussions with Ambrose about this were done on public mailing lists. Some. So maybe look in the archives (I think there’s a core.typed mailing list?).
2017:07:02 19:37:10 seancorfield By contrast, Spec is very much purely opt-in. You can specify just what you want, and test for validity/conformance exactly where you want. You can specify just data or just functions or a bit of both. You can choose exactly what to instrument and when. And you also get the benefit of being able to generate conforming test data and conduct generative testing as and when you need it.
2017:07:02 19:44:39 carocad I guess the marketing idea of core type is greater that its implementation. Have you actually tried spectrum? I know it is quite experimental but I think it child be a great tool, although from what I have read you also have to annotate a lot of code for it to stop complaining
2017:07:02 19:45:31 seancorfield We have not tried Spectrum yet. It seemed a bit too experimental for production code at present.
2017:07:02 19:48:35 seancorfield And to be fair to core.typed, it’s improved dramatically over the years and it is an amazing piece of engineering work. Clojure is just a really hard language to perform full and useful type inference on so it’s a brutally hard problem to solve. I understand that the latest work being done focuses handling the boundary between typed code and untyped code and that Ambrose is working with the Typed Racket folks on that?
2017:07:02 19:48:47 carocad Yeah I'm looking forward to the results of this gsoc and see the result of spec/type projects (hopefully they are implemented) :)
2017:07:02 19:49:13 seancorfield Part of my PhD work was looking at type inference so I know how difficult this problem can be with some languages 😐
2017:07:02 19:52:52 seancorfield This was a fascinating talk at Clojure/West BTW http://2017.clojurewest.org/emina_torlak/ — back in the late 80's I was working with MALPAS which was a program verification system and we were transforming C code for analysis and verification. Emina’s talk showed both how far this area of research has come in three decades, as well as how difficult it is and how slow progress really is.
2017:07:02 19:53:13 carocad Mind if I ask: in your opinion is the type inference that core.typed is doing "better" than for example flow for JavaScript. I guess I don't have to ask about java :/
2017:07:02 19:53:40 seancorfield I don’t follow anything in the JS world — can’t stand the language 🙂
2017:07:02 19:56:25 carocad Well funny because flow is written in Ocaml so I thought the techniques might be familiar or general for type inference. So I was wondering about their difference. I guess I ignore too much about the topic 😫
2017:07:02 20:17:00 seancorfield The implementation language has no bearing on the language you're analyzing, interpreting, or compiling. I implemented both APL and my various experimental FP languages using Pascal, and I wrote the type inference engine in Prolog simple_smile
2017:07:02 20:18:34 seancorfield Given a choice back then I'd have preferred Algol 68 to Pascal but my reviewers were not familiar enough with Algol 68 so I wasn't allowed.
2017:07:02 20:31:55 carocad Jesus I don't even know those languages ... :/
2017:07:02 20:48:26 seancorfield The "Pragmatic Programmer" suggests learning a new language every year -- good advice! 😈
2017:07:03 01:03:14 wildermuthn Good info here! @U04V70XH6, did you find that using typed langs helped to do the things the article said it wanted: catch bugs earlier, make it easier to refactor, have a larger team working on the codebase?
2017:07:03 01:03:39 wildermuthn And related, whether Spec can actually address those issues?
2017:07:03 01:04:54 wildermuthn My cljs team uses Spec mostly to conform/validate api request/responses. But thinking about whether to expand it to be a keyword namespaced kind of typing.
2017:07:03 01:37:11 seancorfield Sure, a type system will catch some bugs earlier (by definition, since it takes effect before your code even runs) but a type system can’t catch all classes of bugs. In the context of Clojure — working with small, pure functions on immutable data (for the most part), and building up code with a REPL — there are whole class of bugs you won’t introduce in the first place. I don’t think there’s been enough analysis in the FP arena comparing static/dynamic type systems for any conclusion to be drawn. If you compare, say, Java and Kotlin, you have two languages with static types and one catches potential NPEs (Kotlin) and one doesn’t. But if you look at Clojure, nil-punning is idiomatic and makes annotating code much, much harder because “nilable” infests all the types in many places in your code. Type signatures for generic code tend to become very complex — try writing a Spec for map (and remember its return type will be different with just a function argument compared to with one or collection arguments).
2017:07:03 01:39:02 seancorfield Spec isn’t a type system and you’re not really comparing apples to apples if you try to compare it to one. I don’t know what Typed Clojure (in a true language sense, not core.typed) would look like but I don’t think it would be much like the Clojure we know and love today.
2017:07:03 07:13:04 carocad @U04V70XH6 I partially disagree with what you said. Building up code in the REPL and having small pure functions sure helps but there are times where the datastructures are soo deep that creating those by hand becomes a nightmare. Specially in those cases I find a type system to be way more useful than a repl. It gives you a first overview of whether a big refactoring is going in the right direction. It is not something to rely on, of course, but I think it eases those tasks
2017:07:03 10:29:24 seancorfield @U0LJU20SJ I'm curious about the business domain where such deeply nested structures occur? It's not something I've ever run into. I'd expect Spec to be very helpful in such a situation then?
2017:07:03 13:56:05 carocad @U04V70XH6 take a look at the response from Mapbox Directions API: https://www.mapbox.com/api-documentation/?language=JavaScript#route-object. They need to returns lots of objects in a json api. In my case I am trying to provide a customized routing which means that I have to mimic their api response. The problem obviously arises once they change (currently they are at v5). In those cases you have to throw away pretty much every spec that you had for that and start refactoring lots of code. That is what I meant with deeply nested objects involved in a big refactoring process. I guess @U07CTDKT7 tried to raise that topic as well.
2017:07:03 14:00:11 carocad To mention another example. Since we are not all senior developers ( 😉 ) we might just go with the first idea of what you should return at a specific place. Later on as you learn more both about the topic and about the language you realize that you need a better design for it. More often than not, those are deeply nested objects which require quite some refactor. In that case, having a static type analyzer to back you up is great. Obviously that means that the system was poorly designed from the beginning but then again … we beginners 😛
2017:07:03 15:21:29 wildermuthn @U04V70XH6, what you are saying about immutablity, pure functions, and the repl being a bug-reducer has been my experience too (js vs ClojureScript). @U0LJU20SJ, I've had the same case with denormalized api responses. We're using DataScript to renormalize the data, but that's really tedious (using Spec has helped). In terms of refactoring, though, my impression is that not having to deal with types makes refactoring easier (but not simpler?). I'm impressed by Rich Hickey's point in his Spec talks that it is probably better to just have new functions that to modify old ones in a breaking manner (which Spec should make easier to identify).
2017:07:03 15:22:42 wildermuthn But this would mean Spec'ing every function you ever write! :) Which makes me wonder if it is worthwhile.
2017:07:03 15:25:37 carocad @U07CTDKT7 I actually tried to spec every function before as well and it was a nightmare. But at some I realized that specifying the API functions is actually enough. If the user facing function works as expected, then all other internal ones should also do it, otherwise the external would have failed. At least that is the way that I am approaching the problem now 🙂
2017:07:03 16:46:49 seancorfield Agreed, re: spec’ing lots of functions. We spec key data structures (“data as API”) and we spec some functions. Nearly all of our use of Spec is explicit (`conform`, valid?) rather than implicit (`instrument`). Regarding deeply nested API results — I’d probably wrap the API and flatten results out quite a bit and then work with that everywhere else in my code, probably with namespaced keys. Easier to read and manipulate. Inside the wrapper, I’d probably use Spec to describe the exact API result to better support the transformation to a flatter data structure.
2017:07:03 17:00:39 joshjones Though it's still preferable to not add a restriction like this, as having both keys should do no harm, it's good to know that spec provides the flexibility for this to work
2017:07:04 10:17:48 dm3 how do you properly spec a structure where the contents depend on type of one of the fields? E.g.
(def m {:standard/source :the-source, :standard/content {:standard.content/details {... specific to :the-source ...}})
I’d like to make a spec for :standard.content/details based on :the-source
2017:07:04 10:19:03 dm3 I can’t see the way out except for having :standard.content/details renamed to :the-source/details - is there any?
2017:07:04 10:21:51 andrewmcveigh Sounds like a job for multi-spec
2017:07:04 11:04:28 rickmoynihan is it just me or does (s/keys :req [(or ::foo ::bar)]) default generator always generate both ::foo and ::bar rather than one or the other, and both?
2017:07:04 11:09:32 rickmoynihan I appreciate there are other ways to write this by moving the or over two keys specs
2017:07:04 11:13:30 bronsa there's alraedy a ticket for this IIRC
Uncaught Error: nth not supported on this type cljs.core/Keyword
It seems to be due to ::join-key in map-of, if I replace that with keyword? then it works.
2017:07:04 20:13:01 danielstockton Oh no, sorry. I need to read the documentation better. There is an option to conform keys for map-of.
2017:07:04 20:16:26 danielstockton Perhaps it shouldn't throw an error though?
2017:07:04 20:16:57 danielstockton If the key isn't conformed, unform could just return it as is.
2017:07:04 20:46:37 jcf Hi all. Has anyone come up with an elegant way of specifying :args to a zero-arity function?
(s/fdef f
:args empty? ; kinda cryptic, no?
:ret map?)
empty? kinda does what I need I think because it'd end up applying an empty list of args to the function, but that seems a little cryptic.
2017:07:04 20:47:29 jcf I guess I could use (s/coll-of any? :count 0) but again that doesn't sit well with me.
2017:07:04 20:48:23 jcf Maybe I'll create ::none somewhere in my codebase to return an empty collection (s/coll-of any? :count 0 :into []).
2017:07:04 20:48:40 jcf Or ::zero-args to make it clear what I'm doing.
2017:07:04 20:50:34 jcf
(s/def ::zero-args
#{[]})
That works but I don't ❤️ it.
2017:07:04 21:29:30 gfredericks @jcf (s/cat)?
2017:07:05 10:13:35 curlyfry Hi, I just realized that spec/instrument is incompatible with event handler-type systems like re-frame (systems where functions are stored in a data structure prior to instrument being called). Since instrument wraps the original var, the function calls that are made through lookup in a data structure (e.g a map of event->handler-fn) aren't spec checked. Since this is a clear majority of the calls being made in for example re-frame, the value of spec:ing (with fdef) the handler functions is decreased.
Is there any way around this at all? I guess you could add the vars themselves in the map instead of the functions they point to, but that seems like a big change to make in an existing system, and feels pretty unidiomatic.
2017:07:05 10:35:17 carocad curlyfry: why do you need to fdef the function? doesnt it suffice to just spec your :db such that it is always in the right state?
you could also avoid instrumenting your functions and have them tested separatedly
2017:07:05 11:02:40 curlyfry I'd like to use the philosophy of spec: instrument ensures that functions are called correctly, tests ensures that they have the correct return values. Checking the whole app-db after every event is also not viable when the app gets large.
2017:07:05 11:30:22 carocad > tests ensures that they have the correct return values.
not really. You can fdef a function without instrumenting it. Then on your test spec will generate random input against which you can check the output of the function and the relation between them.
> I’d like to use the philosophy of spec: instrument ensures that functions are called correctly
If you only interested in that then I dont know how to 😞
2017:07:05 14:40:38 carocad hehe sorry for the confusion. In retrospective my message is quite confusing 😄. What I meant is that if you can test your function with generated random input and it returns what you expect then instrumenting at runtime is not necessary since you can be sure of the integrity of your function. It is definitely not so dynamic since changing your functions would break that certainty but so far I dont know any other option
2017:07:05 15:01:23 curlyfry I guess I'd just prefer to be able to do both... Anyway, thanks a lot for your input! :)
2017:07:05 15:12:07 carocad oh another idea. Can you not redefine the handlers so that re-frame uses the new instrumented functions? I use Cursive so I usually just do sync file to repl and it automatically loads all changed files which would trigger the reg-event-handler … I guess. Have not tried it yet 🙂 Hope it helps
2017:07:06 00:07:25 danielcompton @U0J30HBRS with re-frame in particular I suspect we'll have to add support for turning on spec checking so you can do this. Would be nice to integrate with spec/instrument, but doesn't seem like its possible
2017:07:08 10:05:38 stathissideris a similar problem is when you have a chain of ring handlers and you reload the code for one of them, but because the data structure is already referring to the original var, reloading the handler has not effect. In cases like that, I’ve seen people adding the original vars as you suggested and I don’t think it’s unidiomatic. Just a bit unusual!
2017:07:05 10:16:19 andrewmcveigh I guess you could potentially spec the data structure and use fspec at the leaves where the event-handler functions are.
2017:07:05 10:16:56 andrewmcveigh Seems complicated though
2017:07:05 10:21:43 danielstockton Not possible to def the functions outside the datastructure?
2017:07:05 11:30:44 curlyfry @danielstockton That's how I do it. The problem is that the functions, not the vars, are what are added to the datastructure. If you try it out, you'll notice that the function calls made from lookups in the map aren't checked.
In my particular case the functions are added to a data structure behind the scenes, by re-frames reg-event.
2017:07:05 12:39:21 rickmoynihan I’m trying to instrument an fdef’d function with a stub and a custom generator but get an exception ClassCastException clojure.test.check.generators.Generator cannot be cast to clojure.lang.IFn clojure.spec.alpha/gensub (alpha.clj:268)
2017:07:05 12:40:15 rickmoynihan the instrument call itself raises the exception
2017:07:05 12:42:35 rickmoynihan ahh I see the problem - I need to obviously gen the return value not the args… doh
2017:07:05 12:43:08 rickmoynihan changing it to :ret ::num works
2017:07:05 13:01:07 andrewmcveigh Is it that (g/elements [1]) should be #(g/elements [1])?
2017:07:05 13:01:27 rickmoynihan ahh yeah sorry I’d fixed that locally too
2017:07:05 13:01:40 andrewmcveigh np
2017:07:05 13:02:12 rickmoynihan actually that’s one thing I’m never really clear on… when to pass a generator and when to pass a 0-arg function returning a generator…
2017:07:05 13:02:22 rickmoynihan is there a resource somewhere explaining that?
2017:07:05 13:04:09 andrewmcveigh I think you always need to give a function to something that comes from spec (and is deffed). As far as I worked out, it's to do with the way spec lazy-loads test.check. Not seen it written down anyway though.
2017:07:05 13:05:19 andrewmcveigh I can't think of a place where spec requires a plain generator.
2017:07:05 13:05:38 andrewmcveigh Only if you're calling test.check stuff directly
2017:07:05 13:21:00 rickmoynihan ok
2017:07:05 13:21:10 rickmoynihan at least I know the heuristic now 🙂
2017:07:05 13:51:58 rickmoynihan Is it just me or does instrument with a stub also try and generate conforming values for the :args spec; even though it’ll never need a generator for them - as it only needs to generate return values?
2017:07:05 13:53:19 rickmoynihan basically I’m getting a such-that error on instrument but if I replace the :args spec with something like (s/* any?) it works.
2017:07:05 14:05:48 jannis Hi folks! Is there a good approach to writing function specs and running clojure.spec.test.alpha/check in a way that allows functions to either return a value (checked with :ret) or throw an exception?
2017:07:05 14:09:01 jannis Or, simplified, is there a way to spec a function so that the fact that it throws under certain conditions is part of the spec?
2017:07:05 14:24:59 wilkerlucio @jannis I remember reading something here about spec not contemplating exceptional cases, I believe the idea is that specs are about data only
2017:07:05 14:25:10 jannis Yes, that’s what I was thinking.
2017:07:05 14:26:29 jannis I think I’ve found a solution for myself: have the spec verify the data is ok, then have the function only operate on valid data. That way generative tests only generate valid data but invalid data would still be caught at runtime if I use instrumentation.
2017:07:05 14:26:50 jannis That completely eliminates the need for exceptions. 🙂
2017:07:05 14:35:43 joshjones You don’t want to use instrument at runtime, generally speaking @jannis
2017:07:05 14:36:13 jannis I think I do—but only while developing.
2017:07:05 14:36:29 jannis And while not testing performance perhaps 😉
2017:07:05 14:36:35 joshjones yes, that’s what i meant and should have said “production” 😉
2017:07:05 14:38:44 jannis @joshjones No worries, I assumed that’s what you meant already.
2017:07:05 20:38:59 hkjels Is there a clever way to get the merged form of a spec that uses spec/merge?
2017:07:05 22:20:33 rickmoynihan Just ran into this issue ( https://dev.clojure.org/jira/browse/CLJ-1989 ) and surprised that the lazy load coverage doesn’t extend to all of test.check.generators
2017:07:06 01:42:26 gfredericks let in particular is a macro, and the only macro of the test.check generators
2017:07:06 01:42:42 gfredericks and it looks like the patch doesn't lazy-load it, it completely reimplements it, with different behavior
2017:07:06 01:44:35 gfredericks I'm not sure if lazy-loading a macro is even possible
2017:07:06 10:05:52 jimmy hi guys, is there any good way to test all fdef in my project instead of using clojure.spec.test.alpha/check for each one explicitly.
2017:07:06 10:18:31 rickmoynihan @nxqd: guessing you mean fdef not fspec but if you call check with 0-args it should run all var specs that have been loaded
2017:07:06 10:19:36 jimmy rickmoynihan: really, nice. I will give it a try. Thanks
2017:07:06 10:20:21 jimmy nice, it works 🙂
2017:07:06 10:20:34 rickmoynihan cool
2017:07:06 10:27:25 jimmy btw, does stest/check run all the tests in parallel ?
2017:07:06 10:51:46 rickmoynihan no idea - guessing not
2017:07:09 09:19:20 jimmy @U06A9614K thanks for your info ( And sorry for my lazy ass ... )
2017:07:06 10:21:24 rickmoynihan Hmmm… wondering if generator errors like ExceptionInfo Couldn't satisfy such-that predicate after 100 tries. clojure.core/ex-info (core.clj:4725) would be better if they told you which generator/spec failed to generate a value. The stack trace doesn’t seem to contain many useful clues.
2017:07:06 10:24:01 rickmoynihan Would be nice if the ex-info data contained something like {:generator #function[foo.bar/bar-gen] :spec :foo.bar/valid-bar?}
2017:07:06 11:19:58 gfredericks rickmoynihan: test.check didn't provide a mechanism for this until 0.10.0-alpha1; now that that's released, clojure.spec can be modified to make use of it
2017:07:06 11:25:47 rickmoynihan Cool!
One thing I’m not quite clear on is the precise relationship between clojure.spec and test.check. I obviously understand spec uses it, and wraps it via the lazy-load mechanism. But is spec promising to expose everything inside test.check (plus its own stuff), or is it deciding on what to expose/not-expose? If so how are those decisions being made?
2017:07:06 11:30:36 gfredericks the exposing-vs-not distinction isn't very impactful since you can reference test.check generators directly in your spec usage; I don't think that was ever disintended
2017:07:06 11:31:45 gfredericks I don't know about why some generators don't have proxy versions in spec (other than let)
2017:07:06 11:33:44 rickmoynihan Yeah, I’ve been thinking I might be better just using test.check.generators rather than specs lazy loaded ones… What would the disadvantage be of doing that? Presumably I’d have to make sure the non-lazy-loaded generators were only ever included in dev envs?
2017:07:06 11:34:17 rickmoynihan It really frustrates me that the lazy loaded ones don’t have docstrings.
2017:07:06 11:34:21 gfredericks "have to" is strong, depends on how much you care about your production deps
2017:07:06 11:34:28 gfredericks that sounds fixable
2017:07:06 11:35:34 rickmoynihan > depends on how much you care about your production deps
Yeah, I guess I don’t when its an app. A library should probably be more conservative though.
2017:07:06 11:35:58 gfredericks but yeah, you could stick the generators in /test or /dev or something, and then have your own sort of lazy loading thing if you need to reference them from /src
2017:07:06 11:36:55 rickmoynihan its a shame about let not being included. It feels much nicer than fmap and others.
2017:07:06 11:37:12 rickmoynihan but understand the macroness of it complicates it
2017:07:06 11:37:44 gfredericks yeah;
I suppose you could have a weird thing where the macro tries to require test.check and if it fails it expands to exception-throwing code
2017:07:06 11:38:42 rickmoynihan That seems like a nice idea
2017:07:06 11:41:36 rickmoynihan As an aside are you involved at all in the development of clojure.spec beyond your work on test.check?
2017:07:06 11:51:05 gfredericks nope
2017:07:06 11:58:37 gfredericks not beyond the details of the integration
2017:07:06 12:17:34 gfredericks only nitpick is that the macro wouldn't expand to a require call, it would actually call require before constructing any expanded code
2017:07:06 12:18:35 gfredericks e.g.,
(defmacro let [& args] (try (require ...) `(...) (catch Exception e `(throw ...))))
2017:07:06 12:23:06 rickmoynihan yeah fair point, that’s what I meant… will update the ticket
2017:07:06 10:53:10 rickmoynihan Hmm also just noticed a small problem with check which is that you can’t both run it on all fdefs and provide custom options
2017:07:06 10:53:34 rickmoynihan ahh its ok checkable-syms is exposed so can do it by hand
2017:07:06 12:15:45 rickmoynihan @gfredericks: Do you know if test.check 0.10.0-alpha2 is compatible with the latest clj 1.9 / spec?
2017:07:06 12:18:54 gfredericks @rickmoynihan should be, yeah
2017:07:06 12:22:35 rickmoynihan awesome
2017:07:06 17:48:02 peeja Are we not supposed to be able to add generators to predicates? Is that a closed set?
2017:07:06 17:48:30 peeja (That is, if I define a predicate, am I not able to add a generator for it?)
2017:07:06 18:02:02 wilkerlucio @peeja I'm not sure what you asked, but I think you might want to check (s/with-gen)
2017:07:06 18:02:17 wilkerlucio then you can set any generator you want to your spec, no mater the predicate
2017:07:06 18:02:37 peeja Right, I mean I want to register that as the generator to use everywhere (if it's not overridden)
2017:07:06 18:04:01 wilkerlucio then it will inherit the generator
2017:07:06 18:05:54 wilkerlucio @hlship on the and case, the next predicate will receive the conformed version from the previous one
2017:07:06 18:06:23 hlship (s/explain ::identifier :foo)
java.lang.ClassCastException: clojure.lang.MapEntry cannot be cast to clojure.lang.Named
`
I'm guessing that the entire map conformed by s/or (e.g., {:keyword :foo}) is being passed to graphql-identifier? (which expects a String, Keyword, or Symbol).
So, what's the correct way to deal with this situation? I'd rather not do the identifier check first. I guess I could change graphql-identifier? to expect the conformed result from the s/or?
2017:07:06 18:06:43 wilkerlucio so, your graphql-identifier? will receive something like {:keyword :bla} or {:symbol some-sym}
2017:07:06 18:06:56 peeja @wilkerlucio Yeah, that makes sense. I was hoping to do it straight on the predicate because it looks like a nicer API, but if that's not how it's meant to work, then that works.
2017:07:06 18:08:08 hlship It seems like it gets the Map$Entry, e.g, [:symbol foo].
2017:07:06 18:08:54 wilkerlucio @peeja if you look at the sources for the spec generators, it uses a map to convert from symbol -> generator, I guess we can suggest this to be replaced with a multi-method making it extensible from the outside
2017:07:06 18:50:28 peeja Is there any way to use with-gen without blowing away generator overrides?
2017:07:06 19:01:18 peeja It seems like it can't, since it doesn't have access to the overrides like gen* does
2017:07:06 19:01:49 peeja Are we meant to define our own macros a la keys/`map-spec-impl` etc?
2017:07:07 17:15:49 english I’ve been working on a Ruby port of spec: https://github.com/english/speculation
Implementation-wise it’s got all the important stuff: data validation, instrumentation and generative testing. I’m currently working on improving the docs and writing up some guides.
If there are any Rubyists here jealous of clojure’s spec, I’d been keen to hear your thoughts and feedback!
2017:07:09 09:21:07 jimmy english: hey this is amazing. It's a good lib for me to train some junior Ruby dev before converting them to Clojure completely 😄
2017:07:09 16:58:29 english Thanks! The api is pretty much the same so transitioning should be straightforward
2017:07:07 19:53:09 ghadi What is the farthest along library with making nice experiences for spec-based errors?
2017:07:08 02:07:20 xiongtx Is the file/line/col of spec definitions stored anywhere, as they are for vars when the latter are read?
It’d be nice to have that information so we can jump to a spec’s definition.
2017:07:08 06:04:33 richiardiandrea @xiongtx there has been some plan for that if I remember correctly, unfortunately I cannot find the reference (I think it was exactly in this channel)
2017:07:08 17:44:00 jsselman How can I use the result of clojure.spec.test/check to see if a test failed? The doc seems to hint at checking the value of the :failure key, but that value is false in my test despite there being a clear check error
2017:07:08 17:47:16 gfredericks @jsselman is it false or [false]?
2017:07:08 17:47:55 jsselman println shows it as false
2017:07:08 17:48:09 gfredericks hmm
2017:07:08 17:48:09 jsselman I should mention this is ClojureScript not Clojure
2017:07:08 17:48:41 gfredericks what's the full return?
2017:07:08 17:51:03 jsselman the function call is just (stest/check ``sut/normalize)`
2017:07:08 17:51:36 jsselman arg how do i insert a literal backtick
2017:07:08 17:51:48 gfredericks the value under [:clojure.test.check/ret :result] should work
2017:07:08 17:52:11 gfredericks I have no idea what the failure key is meant to mean
2017:07:08 17:52:17 jsselman the value there is false, does that indicate the test failed?
2017:07:08 17:52:22 gfredericks yes
2017:07:08 17:52:23 jsselman i guess i can quickly check that
2017:07:08 17:56:39 jsselman @gfredericks that does work, thanks!
2017:07:08 17:59:25 jsselman The spec guide shows an example using
2017:07:09 14:12:43 mpenet Couldn't it be made more efficient by conforming 1 arg at a time instead of creating a s/cat at invoke time? That would also allow to avoid & args?
2017:07:09 14:13:39 mpenet Ah but you do this to pattern match depending on arg type for same arg length. Nevermind
2017:07:09 14:14:53 mpenet Cool stuff
2017:07:09 23:07:25 olivergeorge hey wow. that goes beyond typing the defns. interesting stuff.
2017:07:09 23:09:07 olivergeorge I've been looking at porting some ocaml code. they rely heavily on functions with match statements which looks a lot like this. (I've been using core.match)
2017:07:10 05:53:52 anticrisis that part makes sense, but I'm not connecting it to the use of (s/spec) to change the behavior of the s/* operator when defining a function spec
2017:07:10 05:54:41 anticrisis I'm scrutinizing the output of s/exercise with and without s/spec right now
2017:07:10 05:58:53 anticrisis To me, the conformed output of (s/exercise (s/cat :t (s/spec ::test))) and (s/exercise (s/cat :t ::test)) look the same
2017:07:10 06:01:55 anticrisis Thanks for tip about s/specwith regexp ops and s/exercise, they point me in the right direction, even if I don't completely understand it. Many thanks.
2017:07:10 06:25:33 anticrisis I still think there's something inconsistent going on, either because I'm misusing this simple s/* form, or because something's broken. Here's a short walkthrough to demonstrate:
2017:07:10 06:25:54 anticrisis
(s/def ::test (s/cat :k keyword? :rest (s/* pos-int?)))
(s/conform ::test [:foo 1 2 3 4])
;; => {:k :foo, :rest [1 2 3 4]}
(s/conform (s/spec ::test) [:foo 1 2 3 4])
;; => {:k :foo, :rest [1 2 3 4]}
(s/conform (s/cat :t ::test) [:foo 1 2 3 4])
;; => {:t {:k :foo, :rest [1 2 3 4]}}
(s/conform (s/cat :t (s/spec ::test)) [:foo 1 2 3 4])
;; => :clojure.spec.alpha/invalid
(defn test-fn [test] nil)
(s/fdef test-fn :args (s/cat :t ::test) :ret nil?)
(stest/instrument `test-fn)
(test-fn [:foo 1 2 3 4])
;; => exception
(s/fdef test-fn :args (s/cat :t (s/spec ::test)) :ret nil?)
(stest/instrument `test-fn)
(test-fn [:foo 1 2 3 4])
;; => nil
;; Note that in order to get test-fn to pass instrumentation,
;; its :args must use the same form which conform considers invalid
2017:07:10 06:46:21 seancorfield The :args in s/fdef is the sequence of arguments.
2017:07:10 06:50:31 seancorfield For your first s/fdef -- :args (s/cat :t ::test) -- your test-fn would have to be (defn test-fn [kw & nums] nil) -- first argument :k, a keyword, and the :rest of the arguments are numbers.
2017:07:10 06:50:40 seancorfield does that help @anticrisis ?
2017:07:10 06:52:03 anticrisis I see what you're saying... but I thought :args (s/cat :x ::whatever) meant "the arguments are a list of one element, labeled 'x', which can be conformed with spec '::whatever'
2017:07:10 06:57:24 anticrisis Your example of fdef'ing [kw & nums] is helpful to tell me I've been thinking about this all wrong. I would have fdef'd it like this: :args (s/cat :k keyword? :nums (s/* pos-int?)) -- which you're telling me is the same as :args ::test-- which it is, but those particular neurons in my brain apparently weren't connected.
2017:07:10 07:04:22 anticrisis For the record, I'd like to point out that nesting data structures is confusing. Is there a version of clojure I can use that doesn't do that? Thanks. 🙂
2017:07:10 07:14:48 seancorfield If you want actual sequences (and nested sequences) then use coll-of etc, not cat 😸
2017:07:10 07:39:23 hkjels How can I stub a function? Typically I’d like to require that on-click is a function, but it fails when I later exercise it since theres no generator for it
2017:07:10 07:40:29 hkjels Creating a custom generator would be overkill I think
2017:07:10 11:12:20 gfredericks @hkjels would a custom generator of (gen/return (constantly nil)) be overkill?
2017:07:10 11:41:17 hkjels @gfredericks ohh, hehe. no
2017:07:10 11:41:19 hkjels 🙂
2017:07:10 11:41:26 hkjels thanks!
2017:07:10 13:13:39 rickmoynihan Hmmm… I’ve just made an observation about specs which I’m wondering if others could validate for me…
Basically I have a layered architecture where we do something like this: (->> input transform-1 transform-2 transform-3) and I’ve just realised that in layout terms it seems better to organise the input specs alongside the transforms, rather than the :ret specs.
It also seems to more closely align with things like instrument checking :args rather than :rets
Has anyone else noticed this?
2017:07:10 13:55:57 rickmoynihan What is the most idiomatic way to spec something to be a single value?
I’m guessing:
(s/def ::single-value #{:foo})
2017:07:10 13:58:00 mpenet yep, you get free gen with that one
2017:07:10 14:51:41 rickmoynihan mpenet: yeah the free gen is a big motivator 🙂
2017:07:10 13:58:26 mpenet in theory (constantly :foo) would work for validation but no gen
2017:07:10 14:01:03 gfredericks (constantly :foo) would work? That seems identical to any? to me
2017:07:10 14:01:19 gfredericks it's a predicate that always returns truthy
2017:07:10 14:01:24 mpenet right
2017:07:10 14:01:33 mpenet bad example
2017:07:10 14:01:43 gfredericks #(= % :foo) I guess?
2017:07:10 14:01:51 mpenet yes or #(identical? :foo %)
2017:07:10 14:02:07 gfredericks just for keywords
2017:07:10 14:02:13 mpenet yup
2017:07:10 14:02:40 mpenet but anyway, the set solutions is better in most case
2017:07:10 14:03:20 gfredericks I can imagine people stumbling on #{nil} relatively often
2017:07:10 14:04:17 alexmiller use nil? then
2017:07:10 14:04:19 mpenet well I guess you want nil? in that case
2017:07:10 14:04:37 gfredericks yep
2017:07:10 14:04:55 gfredericks I guess that sort of error isn't likely to go unnoticed
2017:07:10 18:43:35 bbrinck Excuse the lack of documentation or sane organization
2017:07:10 18:46:49 bbrinck Has anyone found a use case for specing a map with coll-of? I see that you can create a spec like (s/def :foo/map (s/coll-of int? :kind map? :into {})) but I can’t see why this would be useful since everything but the empty map would always fail.
2017:07:10 18:48:18 bbrinck I suppose you could maybe replace int? in that example with a spec for the key/value pairs, but in that case, map-of seems superior.
2017:07:10 18:48:41 alexmiller there are some cases where it is useful to spec a map as a sequence of entry tuples
2017:07:10 18:49:17 alexmiller “hybrid” maps (combination of s/keys and s/map-of styles) is one case
2017:07:10 20:09:36 souenzzo alexmiller: When I look to the result of (s/conform ::binding-form it remember me about ast's ... may in the future, can spec/conform be used as analyzer?
2017:07:10 20:18:30 alexmiller If you use that in the general sense, then yes absolutely - it’s a great tool for turning s-expressions into a tree of maps describing a DSL syntax
2017:07:10 18:54:05 bbrinck @alexmiller Ah, I see. You’re using a tuple as the “inner” spec. That makes sense. Thanks for the info!
2017:07:10 18:54:27 alexmiller that’s actually how s/map-of and s/every-kv are implemented
2017:07:10 18:57:35 bbrinck Interesting. Your example was instructive. I can see how oring together several tuples would let me define different valid key/value pairs
2017:07:10 18:58:42 bbrinck Basically, it lets me create a clear relationship between the spec for certain keys and the specs for their values. But combine them all into one big map spec
2017:07:10 18:59:37 bbrinck “valid key/value pairs” Sorry, I meant “valid key/value pairs of specs”
2017:07:10 20:03:14 lfn3 Is fspec meant to handle cases where the provided function throws? It’s causing spec errors when instrumented.
2017:07:10 20:05:23 lfn3 I’ve got a relatively minimal repro:
2017:07:10 20:06:12 lfn3 That yields: In: [0] val: (true) fails at: [:args :f] predicate: (apply fn), scary exception! when the test is run
2017:07:10 20:24:04 alexmiller when you have an fdef with an fspec arg, the instrumented var will check the fspec by generating from its args spec and invoking the function to validate it conforms to the spec
2017:07:10 20:24:27 alexmiller in this case it will generate boolean values and invoke f
2017:07:10 20:25:41 alexmiller specs cover non-exceptional use (as those are the exceptions :) so I think a more accurate fspec args spec in this case would use false?, not boolean?. it would then invoke f with only false and not trigger the exception
2017:07:10 21:38:58 lfn3 Right I guess I just wasn't expecting the fspec'd arg to be invoked? Wouldn't it be better to use with-gen so the spec still captures the allowed range of values?
2017:07:10 22:28:20 alexmiller well the idea is that you said this function takes a function that responds a certain way - the only way to validate that is to invoke it
2017:07:10 22:33:04 lfn3 Yeah agreed. I don’t think that was really a question, I was just kinda surprised. I would have assumed that the fn would be instrumented and validated when it was invoked rather than when it was passed into another function. Might be something worth mentioning in the docs, if it isn’t already?
2017:07:10 22:45:27 alexmiller This has been brought up several times but I haven't had a chance to discuss it with Rich
2017:07:10 23:14:05 lfn3 Ok. Thanks for the clarification and explanation in any case, it’s much appreciated.
2017:07:11 01:46:02 danielcompton I'm trying to use spec.test/check to check all my functions, but I'm getting a couldn't satisfy such-that predicate exception. Is there any way to figure out which spec wasn't able to be satisfied?
2017:07:11 02:03:29 seancorfield I don't know of any way to debug that -- other than to s/exercise your specs as you write them...
2017:07:11 02:04:02 seancorfield You could call s/registry and go through all the keys in that (which are specs) I guess...
2017:07:11 02:04:27 seancorfield ...I've just gotten kinda paranoid about exercising each spec in the REPL as I write it.
2017:07:11 02:26:18 bfabry I think there's a request open to include the spec in that message. iterating the registry sounds like a good option in the meantime
2017:07:11 07:39:30 danielcompton @bfabry do you know which issue it is? I took a look in JIRA but couldn't see anything
2017:07:11 08:39:03 hkjels How do I express, many strings and/or many vectors? And I don’t care which one it is, so I don’t need any branching
2017:07:11 09:12:16 rickmoynihan @danielcompton: I’ve had that same issue…. Supposedly the latest version of test.check exposes more information on that, infact I updated my test.check to the latest version which provides a bit more info, but still doesn’t tell you the spec that failed (as it knows nothing of spec).
2017:07:11 09:15:40 danielcompton 😞
2017:07:11 09:31:30 rickmoynihan finding them is a bit of a PITA, lucky for you bisecting specs with exercise is logarithmic 🙂
2017:07:11 09:31:56 rickmoynihan but better to exercise them as you write them bottom up like seancorfield suggests
2017:07:11 11:12:13 gfredericks @danielcompton @rickmoynihan @bfabry what's needed is to find the spot[s] where clojure.spec calls gen/such-that and get it to pass that debugging info along; I have no idea how difficult that would be, might be super easy
2017:07:11 11:12:48 gfredericks the test.check change is that you can now pass an option (`:ex-fn`) that lets you customize the exception
2017:07:11 15:14:08 rickmoynihan Some of my spec generators seem to grow to pretty big values under the default 1000 generations used by clojure.spec.test.alpha/check. I don’t know how big they’re getting in numbers of items, but they’re OOMing. Is there an easy way to make specs flatten out at a certain size, but keep iterating over random generations?
2017:07:11 15:16:01 rickmoynihan obvs I could add an extra constraint e.g. #(> 1000 (count %)) to maps etc… but it feels like it’d be better done in the map/sequence generators
2017:07:11 15:23:01 gfredericks @rickmoynihan under default configurations your tests should max out their size at 200 trials and cycle back to small at that point;
there should be a way to restrict the size further than that
2017:07:11 15:23:54 rickmoynihan hmmm… not sure why I’m seeing so much heap then
2017:07:11 15:24:20 gfredericks the underlying problem here relates to collection sizing, which we have an open test.check issue for
2017:07:11 15:24:34 gfredericks but the workaround is to explicitly limit sizing via the option I just pointed to
2017:07:11 15:25:25 rickmoynihan yeah you can pass those through via (check ,,, {:clojure.spec.test.check {:max-size 10}})
2017:07:11 15:25:29 gfredericks All I was saying with 200 vs 1000 is that you should probably see the same issues with only 200 trials
2017:07:11 15:25:33 gfredericks I'd try 50 for starters
2017:07:11 15:27:01 rickmoynihan 100 for num-tests seems to work but more than that GC seems to dominate the run time
2017:07:11 15:28:17 rickmoynihan will try reducing max-size
2017:07:11 15:29:12 gfredericks num-tests can be as high as you want as long as max-size is low enough
2017:07:11 15:29:37 gfredericks max-size defaults to 200, and the sizes used throughout the run are (cycle (range max-size))
2017:07:11 15:29:45 rickmoynihan cool
2017:07:11 15:30:11 gfredericks so num-tests can "fix" sizing issues only when num-tests < max-size
2017:07:11 15:31:36 rickmoynihan makes sense
2017:07:11 15:32:02 rickmoynihan cool. That max-size trick at 50 seems to work nicely.
2017:07:11 15:33:00 rickmoynihan @gfredericks: so does :max-size 50 mean maps have a maximum of 50 pairs, strings have a maximum of 50 chars, vectors have a maximum of 50 items etc?
2017:07:11 15:33:12 rickmoynihan or is it a weight/multiplier?
2017:07:11 15:33:49 gfredericks that tends to be true, but there's no true definition of size; it's an abstract thing that could theoretically be interpreted differently by any given generator
2017:07:11 15:34:20 rickmoynihan ok that’s what I thought
2017:07:11 15:34:34 gfredericks e.g., you can easily defy that by generating vectors with a specified length of 75
2017:07:11 15:34:42 rickmoynihan 🙂
2017:07:11 15:34:59 rickmoynihan but is it true of the default generators?
2017:07:11 15:35:03 gfredericks yeah
2017:07:11 15:35:43 gfredericks presumably the fix for all this will be that lower-down nested structures will be substantially smaller
2017:07:11 15:35:55 gfredericks but that wouldn't contradict what you just said
2017:07:11 15:39:20 rickmoynihan lowering it to 50 seems to work nicely… it also means I can run more generations in the same time which seems like the right trade off for my data.
2017:07:11 16:47:12 lwhorton does anyone know a way to express “a map of keys of spec A or B, where A key points to spec A0 and B key points to spec B0”?
2017:07:12 01:04:55 danielcompton I just realised that if you're using a tools.namespace based workflow, the spec registry isn't cleared when you refresh. Is there a recommended way to "clean the registry" safely?
2017:07:12 01:05:41 danielcompton I suspect that if I just reset! the registry I might be missing internal/clojure.core.specs specs?
2017:07:12 01:05:52 twashing Question. This will break with Clojure telling me that “No such var: eid”
2017:07:12 10:11:34 joost-diepenmaat if I have to I can copy & paste and adapt that code but it would be nice to have a supported library
2017:07:12 11:50:46 gfredericks @joost-diepenmaat if you paste it into https://github.com/gfredericks/schpec and make a PR, I'll release it
not sure if that counts as "supported" or not
2017:07:12 14:23:26 jpmonettas hi everyone! I'm trying to understand how to match :clojure.spec.alpha/problems inside :clojure.spec.alpha/value in explain-data result independent of the specs that were applied and I'm having some trouble
2017:07:12 14:27:19 jpmonettas so the thing is that seeing a problem in [::k 1] you don't know if it means, it's the value of that key, or it's the element at pos 1 of a seq in the value of that key
2017:07:12 14:27:23 jpmonettas does it make sense?
2017:07:12 14:41:17 bronsa you're right in your analysis, it's been discussed before and it's logged in that ticket
2017:07:12 14:43:45 jpmonettas great, thx a lot @bronsa
2017:07:12 16:58:27 urbank So I'm wondering, is spec with s/def + s/valid? supposed to replace predicate functions? Otherwise I find it hard to decide what should be a spec and what should just be a function
2017:07:12 17:04:42 urbank Oh, I suppose that isn't really a good question. More pertinent would be whether new predicates should be written as anon functions in s/def or should s/def refer to a named predicate function
2017:07:13 10:10:24 rickmoynihan depends… sometimes the predicate function is generic and reusable but the s/def can give it a more specific name/abstraction.
e.g.
(s/def ::user-id integer?)(s/def ::post-id integer?)
2017:07:13 10:10:57 rickmoynihan othertimes the predicate is too specific to be reusable, so just keep it anonymous.
2017:07:13 10:11:13 rickmoynihan (No idea if this is best-practice btw… but it’s what I’ve been doing)
2017:07:12 17:30:04 assoc-in I am wondering how I would write a spec for a nested map. I have something like this
{:1 {:user-id 1 :username "bob"}
:2 {:user-id 2 :username "jim"}}
Where I have a (s/def ::user (s/keys :req-un [::user-id ::username])).
I know how to do this if I had a vector of users by using (s/coll-of ::user), but how would I approach this with having the map keys being the ::user-id spec and the values of each map being the ::user?
2017:07:12 17:58:21 assoc-in @bfabry Ah yes I didn't see that looks like just what I needed. Thanks!
2017:07:12 17:58:56 bfabry np
2017:07:13 00:14:43 bbrinck Several people on this channel have asked about pretty-printing spec failures. I’ve just released Expound, a library that formats clojure.spec errors https://github.com/bhb/expound
2017:07:13 07:07:53 richiardiandrea bbrinck: great great great thanks!
2017:07:13 00:15:05 bbrinck I’ve got a lot more features planned, but hopefully this first release is useful to people on this channel
2017:07:13 00:15:42 bbrinck ^- cc @ghadi
2017:07:13 00:17:34 bbrinck cc @samueldev (who also asked about this awhile back)
2017:07:13 01:20:32 samueldev Awesome @bbrinck I'm going to try this out tomorrow morning!
2017:07:13 06:13:07 naomarik How can I constrain a multispec to a specific key? With the event example in spec guide I’ve tried this: (gen/sample (s/gen :event/event {:event/type #(s/gen #{:event/search})})) which doesn’t work
2017:07:13 06:28:00 naomarik Solved it using gen/fmap to constrain the type, wondering if more succinct solution is available though.
2017:07:13 10:13:29 rickmoynihan @bbrinck: awesome!! I’ve been wanting something like this for a long time… Another thing that might be useful is integrating this into prone for rendering spec errors in stack traces over http (in dev):
https://github.com/magnars/prone
2017:07:15 22:17:17 joshjones I can replicate the instrument stack overflow -- however, for the check, it's just creating huge maps. So, best to run check with some constraints IMO usually:
2017:07:16 22:27:41 lwhorton what’s the proper way to spec protocols? i’m assuming I can just spec some client fn instead of the protocol? but I’m having a conceptual issue about how to have multiple implementations of the protocol with this approach:
;; protocols.cljs
(defprotocol IMyProto
(-foo [this v]))
;; client.cljs
(defn foo [this v]
(protocols/-foo this v))
;; impl-A.cljs
(defrecord AImpl []
IMyProto
(-foo [this v] ...))
(defn foo [this v] (-foo this v)) ;; this is the fn I would instrument for A types
;; impl-B.cljs
(defrecord BImpl []
IMyProto
(-foo [this v] ...))
(defn foo [this v] (-foo this v)) ;; this is the fn I would instrument for B types
in client.cljs I want to be able to call protocols/foo instead of prototols/-foo, because I can instrument (defn foo...). But this breaks the purpose of a protocol - how do I know which implementation to invoke from the client a-impl/foo or b-impl/foo?
What’s the proper way to organize these guys so I can stest/check 'foo?
2017:07:16 22:37:55 lwhorton I suppose I could turn it on its head so that client calls always invoke (protocols/foo this v), and each impl points to an internal function IMyProto (foo [this v] (-foo this v))? But this doesn’t seem friendly, particularly once the records start having fields and if any protocol fn needs to refer back to those fields.
2017:07:16 22:46:43 lwhorton *or perhaps I should re-apply s/fdef to the protocol’s foo when I actually want to run my checks? (s/fdef protocols/foo ::args (s/cat :type ::a-type), then re-def with ::b-type?
2017:07:16 23:21:51 seancorfield Protocols are open for extension -- so you can't write a spec for all possible argument types. I think you could write a multi-spec for foo -- where the defmulti discriminator function returns the type of the first argument?
2017:07:17 00:06:35 danielcompton @lwhorton I don't think you can spec protocols, the recommendation I got at the time was to create a regular function in front of the protocol function and spec that
2017:07:17 00:18:52 lwhorton congratulations by the way. i hope you have a heck of a time with the kiddo. I’‘ll certainly miss The REPL.
2017:07:17 02:55:53 danielcompton I won't be gone for too long 🙂
2017:07:17 00:08:27 lwhorton yea .. that’s what I’m rolling with. I probably wasn’t clear, but in my question I’m trying to address the issue with trying to test.check multiple “regular functions in front of the protocol function” for each possible argument type
2017:07:17 00:09:03 lwhorton as @seancorfield mentions, they’re open for extension so you can’t cover every base. that being said, i’m only trying to cover my particular bases.
2017:07:17 09:11:48 rickmoynihan Worth remembering also that you can spec that the value your wrapping fn receives satisfies? the protocol. You’ll need to write a custom generator to generate conforming values for each of the implementations you care about though. You don’t get that for free like you do with multi-spec.
2017:07:17 09:15:28 rickmoynihan This said if your implementations had a common construction interface… e.g. they all took a string as a single argument to their constructor, you could easily write a dynamic generator by inspecting the protocol i.e. by using (keys (:impls foo.bar/Protocol))
2017:07:17 09:15:54 rickmoynihan then mapping the implementation classes to constructors/generators
2017:07:17 11:40:21 samueldev does anybody have any specs around HTML5 filetypes?
2017:07:17 11:40:50 samueldev I have a fn I'd like to spec but am currently omitting spec'ing one particular argument, which I'm expecting to be an HTML5 File/Blob
2017:07:17 11:53:28 samueldev @bbrinck hey ben, do I have to do anything special to make expound work in CLJS?
2017:07:17 11:59:38 samueldev nvm I didn't realize the transition from cljs.spec -> cljs.spec.alpha
2017:07:17 11:59:43 samueldev had happened
2017:07:17 11:59:46 samueldev all good now 🙂
2017:07:17 14:29:59 bbrinck @samueldev I’ll add that to the README
2017:07:17 14:32:43 bbrinck A user has also reported a issue on CLJS 1.9.671 (I had been testing on CLJS 1.9.542). Just FYI: https://github.com/bhb/expound/issues/3
2017:07:17 15:08:00 samueldev @bbrinck using the lib and converted all my spec explains to yours this morning; quite pleased 🙂
2017:07:17 15:09:19 samueldev its aim seems to be to output developer-friendly error messages; have you considered the use case of going all the way and having it output non-developer-friendly error messages, that can be presented to an end-user?
for example if I'm doing some simple validation on a POST route in my API to create an entity, and they omit a required key, an output-string provided along the lines of "Missing required key: _____"
2017:07:17 15:09:55 samueldev as it stands now, I'm using regular spec explains and iterating over the problems and generating these strings
2017:07:17 15:10:43 samueldev (with some hard-coded mappings from something like a str? predicate failing --> a string being generated along the lines of "Value provided for key ____ must be a string."
2017:07:17 15:11:09 samueldev this is something I've considered lib-ifying for my own sake but could instead PR it into expound perhaps, if you see value in it
2017:07:17 16:05:27 bbrinck Yes, in the future, I want to have good defaults for a number of contexts: test failures, REPL, end-user, etc
2017:07:17 16:06:15 bbrinck That won’t replace the current error message, but it’ll either be another namespace or perhaps some way to configure expound
2017:07:17 16:06:57 bbrinck But more generally, I want to extract out a set of helper functions that work on clojure.spec.problems such that you can trivially build your own custom string representation without dealing with all the details of spec
2017:07:17 16:09:23 bbrinck That way, if you don’t like the default “user-facing” error message, you can peek underneath and see how its built and build your own.
2017:07:17 16:10:26 bbrinck @samueldev If you want to create a GH issue with an example of a spec, a value, and the error string you’d like to see for a user-facing error, that would be very helpful!
2017:07:17 15:17:17 adamfrey for a spec that can potentially generate huge values (like giant nested maps with long keywords), what is the most efficient way to get a small values (for playing around with at the repl)
2017:07:17 15:21:19 schmee for some reason, my spec errors are printed without newline, which makes them completely unreadable:
:error-while-loading kleinheit.datomic.impl
#error {
:cause "Call to clojure.core/refer-clojure did not conform to spec:\nIn: [2 1] val: :as fails at: [:args :exclude :op :quoted-spec :spec] predicate: #{:exclude}\nIn: [2 1] va......
2017:07:17 15:21:28 schmee any idea what might cause this?
2017:07:17 16:15:19 bbrinck @schmee It looks like you’re printing out the entire error record. Is it possible to just get the :cause and print that?
2017:07:17 16:15:40 schmee I hope so 😄
2017:07:17 16:15:51 schmee I don’t get why it prints the whole error record in the first place
2017:07:17 16:16:06 bbrinck Sorry, what is “it”?
2017:07:17 16:16:23 schmee lein repl🙂
2017:07:17 16:17:24 bbrinck Ah, I see. Can you perhaps write a wrapper function that calls the function in a try/catch?
2017:07:17 16:21:23 bbrinck I have not tested this, but you could
2017:07:17 16:21:43 bbrinck @schmee Or, after you see an error like that, could you just do something like (println (:cause (ex-data *e))) to print out the cause of the last error?
2017:07:17 16:22:10 schmee I suppose so, but I would prefer if I could get the REPL to always do that for me
2017:07:17 16:22:11 bbrinck (just put that on the REPL after the error has been caused)
2017:07:17 16:24:40 bbrinck @schmee I haven’t tried this, but it looks like lein let’s you configure the error handler https://github.com/technomancy/leiningen/blob/master/sample.project.clj#L357-L370. Note that by only printing the cause, you’ll presumably be losing other contextual information about the error, which may or may not be useful.
2017:07:17 16:25:12 bbrinck In any case, this isn’t really related to spec, so you may have more luck tracking down an answer in #leiningen . Good luck!
2017:07:17 16:25:35 schmee ahh, now that’s what I’m looking for, thanks! 😄
2017:07:17 17:34:43 schmee is there a “recursive” version of describe, that resolves non-fn specs?
2017:07:17 17:39:18 alexmiller not currently
2017:07:17 17:49:42 schmee what’s the easiest way to constrain a s/cat into a vector?
2017:07:17 17:50:06 schmee (s/and vector? (s/cat ...)) seem to fail to generate data with exercise
2017:07:17 17:51:48 schmee here’s how I’m doing it now, just curious if there is a better way:
2017:07:17 18:23:15 bbrinck Or, perhaps I should say “regex specs that are also vectors”
2017:07:17 18:43:18 gfredericks I feel like in some platonic sense the regex specs fit seqs much more than vectors
but that doesn't mean it wouldn't be terribly useful to have regexes for vectors
2017:07:17 18:43:30 gfredericks probably mostly/entirely for macros?
2017:07:17 18:53:32 bbrinck I’ve also had use cases when using conform/unform on vectors
2017:07:17 18:54:47 bbrinck We have cases where we like the succinctness of using vectors e.g. something like hiccup: [:p {} "hi"]. When we manipulate it, it’s handy to have named parts by using a regex spec, but then we can unform it back to the vector form
2017:07:17 18:55:22 bbrinck Right now we have to unform, then convert the list to a vector
2017:07:17 18:57:28 schmee yeah, a catv would be nice
2017:07:17 18:57:35 schmee is there a JIRA or something for this?
2017:07:18 19:46:44 plins hello everyone, how can i validate a month inside a string?
is the regexp approach the best?
it should be a string containing 2 digits ranging from “01” to “12"
2017:07:18 19:49:58 gfredericks it's certainly easy as a regex #"0[1-9]|1[0-2]"
2017:07:18 19:51:00 plins thx
2017:07:18 22:37:06 jcf
(into #{} (map #(format "%02d" %)) (range 1 13))
You could build a set of the strings as the count is small too. 🙂
2017:07:18 22:39:50 jcf Then with clojure.spec you can do something like (s/def ::month-number-str (into #{} (map #(format "%02d" %)) (range 1 13))).
And/or just write it out like so:
(s/def month-number-str
#{"01" "02" "03" ...}
2017:07:18 23:08:45 creese I'd like to generate a spec dynamically, where the name of the spec is returned from a function. Is there any prior art on this?
2017:07:19 09:10:06 schmee cool, I’ll try it out later today!
2017:07:19 10:47:48 danielneal I'm still new with spec. Is it considered unidiomatic to use conformers to coerce, as in the following. What would be the idiomatic approach
2017:07:19 14:32:01 danielneal @ikitommi what do you do for the coercing bit
2017:07:20 06:03:07 ikitommi @danieleneal spec-tools but I'm biased as co-authoring the lib ;) haven't seen anything else for it, really. Cognitect people have said many times that runtime transformations are not is scope of spec :(
2017:07:19 15:28:49 tcoupland Got a bit of a spec design and validation question: We've got a few messages flying around and are working on adding spec's for them now (i know, wrong way round), the messages generally look like this:
The trouble is the payload key being reused across all the the different :type fields, making writing a spec for :payload very tricky as it lacks the context of the :type value. I'm thinking we should have gone for a flat structure:
A multi spec would then have something to go on. Just wondering if there's anything I've missed that would let us write a decent :payload spec? Feels like the current format just doesn't work with spec very well at the moment.
2017:07:19 15:47:42 jcf @tcoupland if you're using a multimethod with your multi-spec you can implemtent arbitrarily complex dispatch logic.
2017:07:19 15:48:44 jcf You can get at what's in the :payload, and transform it, use or to pick from multiple options.
2017:07:19 15:49:10 jcf Of course, if your data looks really similar that might be tricky, but then how specific does your spec really need to be? 🙂
2017:07:19 15:53:47 tcoupland 🙂
2017:07:19 15:55:02 tcoupland yeah, i'm worried that the dispatch function would get a bit nuts, also it wldn't really solve the problem of attaching the type to a specific payload
2017:07:19 15:58:27 jcf Time for version "2.0.0" 😉
2017:07:19 16:00:31 nwjsmith I was fiddling with some spec tooling last night, and there is something I'd like to express that is difficult in spec. I'd like to be able to express "this is a collection of As for some comparable type A".
2017:07:19 16:03:00 nwjsmith I don't know much about types, but I think in Haskell this can be expressed as Comparable a => List a. It would be useful in situations like spec-ing the args of clojure.core/sort. Especially since vectors are comparable
2017:07:19 16:03:54 nwjsmith That way you could express "this is a collection of As or a collection of vectors of As for some comparable scalar A"
2017:07:19 16:18:49 andrewmcveigh Can't you write:
(s/coll-of #(instance? Comparable %))
2017:07:19 16:18:51 andrewmcveigh ?
2017:07:19 16:19:25 andrewmcveigh Seeing how spec is working on values, you don't really need type constraints
2017:07:19 16:20:30 nwjsmith Yes, totally. It's on the data/test-generation side that this becomes a little awkward.
2017:07:19 16:21:02 andrewmcveigh Ah, OK. Then yes I see your point 😉
2017:07:19 16:21:35 nwjsmith I'd like my cake and I'd like to eat it too 🙂
2017:07:19 16:25:06 andrewmcveigh I think that (s/coll-of (s/and any? #(instance? Comparable %))) is closest in meaning to Haskell's Ord a => List a
2017:07:19 16:25:21 andrewmcveigh Probably will get super slow though
2017:07:20 06:05:14 jimmy hi guys, I'm in a situation that we have two kind of specs:
# "correct" spec, this one is used to do validation, instrument
and # "test" spec, this one is mainly to use in gen test. Since when doing test, sometimes I want a function to handle all kind of input instead of the "correct" one generated by "correct" spec.
How do you guys manage these in terms of code organization ( "test" spec in test namespace, "correct" spec in the same ns of the function def ) or ... ? I would love to hear your idea on this.
2017:07:20 06:06:15 seancorfield @nxqd I would have a single spec but override the generator.
2017:07:20 06:07:20 seancorfield You want the same "correct" spec in both production code and test code -- but it's reasonable to have a generator that produces simpler data. You do not want the other way around.
2017:07:20 06:07:44 seancorfield Perhaps you're not explaining what you're really trying to do?
2017:07:20 06:14:42 seancorfield If you're explicitly doing validation inside the function, then the arguments it takes are a different spec, by definition, since the function takes a broader range of data -- it's expecting a broader range of data so that it can validate a small range of data and do something different and observable for the "invalid" data.
2017:07:20 06:17:08 seancorfield i.e., (if (s/valid? ::some-spec input-data) (do-good-stuff input-data) (do-something-else input-data)) -- the function is anticipating input-data that does not conform to ::some-spec and it is expected, defined, testable behavior that the function does something else -- and the function accepts, as part of its core production spec input data that is beyond just ::some-spec. Does that make sense @nxqd ?
2017:07:20 06:23:23 jimmy yeah you are correct. The approach I'm doing atm is basically the same as you mentioned above. what I have in mind is to design it like this:
code-ns
(s/fdef ... :args (s/cat :args ::args))
(defn ... )
test-ns
(s/fdef ... :args (s/cat :args ::args)) ;; the only different of this ::args is the gen function as you said.
;; register this fdef instead of the one above and run tests.
;; since the real implementation will still use the "correct" spec, there is no problem regarding valid? in function.
Is there any better solution to swap different gen fn in different env, so we don't have to create another test ns.
2017:07:20 06:24:26 seancorfield No, what I'm saying is the function itself has one spec that is the same for test and production. It may be a different spec from what it checks (validates) inside the function.
2017:07:20 06:25:21 seancorfield A generator for a spec cannot generate data that fails to conform to the spec, so the generator may generate a subset of acceptable data, but not a superset.
2017:07:20 06:26:43 jimmy hmm, interesting. I got your point now
2017:07:20 06:28:16 seancorfield Consider an API function that accepts arbitrary strings as arguments but then conforms the arguments to numeric values -- the function argument spec is basically string? and then it will have a separate spec for conforming numeric strings to numbers.
2017:07:20 06:29:25 seancorfield Since you want the arguments to function to be mostly numeric, you need a custom generator that mostly generates numbers and calls str on them, but sometimes generates arbitrary strings (which will fail the other spec) and cause the function to do whatever error handling you expect.
2017:07:20 06:30:29 jimmy agree. this clears my mind. What is your naming convention for fdef args spec and separate spec?
2017:07:20 06:30:48 jimmy
(s/def :fdef/arg )
and (s/def :correct/arg ) ?
2017:07:20 06:31:47 seancorfield It depends on the business domain. In this case, I'd probably have a :domain/spec for the conforming spec and a :api/spec for the function argument spec.
2017:07:20 06:32:06 jimmy great ! Thanks a lot. Have a nice day sir 😄
2017:07:20 06:32:32 seancorfield Glad to help!
2017:07:20 17:39:07 phreed Can I get some help with writing a spec?
I want a vector of of alternating types, similar to a tuple spec but repeating where the types of the first and last are the same. The following is invalid but hopefully suggests what I am looking for.
(s/def ::graph-path (s/tuple-ish ::node ::edge ::node ::edge ... ::node))
2017:07:20 18:02:04 phreed In think I figured it out...
(s/def ::graph-path (s/cat :h ::node :r (s/* (s/cat :e ::edge :n ::node)))
2017:07:21 09:25:50 ikitommi @danieleneal @alex - gave a shot at the CLJ-2116: https://github.com/clojure/spec.alpha/pull/1. Idea is to support a optional conforming callback for conform & explain, allowing specs to be separated from conforming. Would be used like this:
2017:07:21 10:50:45 scaturr Are there any known issues with clojure.test.check and multi arity functions?
2017:07:21 10:52:07 scaturr for some reason when I try to run (st/check make-action) my repl just hangs forever
2017:07:21 10:53:02 scaturr same thing out of the repl too - lein test just hangs (running generative tests via clojure.test.check in my normal test suite)
2017:07:21 10:54:55 scaturr I did verify that I could generate data for all the (s/def) expressions individually - that seems to work
2017:07:21 11:12:04 gfredericks @scaturr did you try generating (s/cat :type ::type :error? ::error? :payload (s/? ::payload)) as well?
2017:07:21 11:13:05 scaturr I did not - I will try that now
2017:07:21 11:14:00 scaturr yes that worked 😕
2017:07:21 11:14:08 gfredericks I'm trying this out too
2017:07:21 11:15:16 gfredericks can't reproduce; check ran fine for me
2017:07:21 11:15:26 scaturr I changed ::payload back to map? for that example
2017:07:21 11:15:33 scaturr instead of string? - though both produce the error
2017:07:21 11:16:08 scaturr what version of test.check are you running?
2017:07:21 11:16:49 scaturr thank you for your help by the way 🙂
2017:07:21 11:20:00 gfredericks well I'd certainly like to know what the difference is, but I don't think it matters enough to look into 😕
2017:07:21 11:20:11 gfredericks but if you figure it out, let me know 🙂
2017:07:21 11:20:16 scaturr will do!
2017:07:21 13:45:21 alexmiller @ikitommi noted. I'm not going to do anything with it until I get a chance to ask Rich about it
2017:07:21 16:22:05 wilkerlucio hello people, for those looking at how to do coercion with specs, spec-coerce leverages your spec definitions to coerce value types 🙂https://github.com/wilkerlucio/spec-coerce
2017:07:22 01:21:22 danielcompton Looks neat, I see there is "Coercion overrides map to specify contextual coercions" on the TODOs, will that let you specify different contexts like a JSON payload or form encoding, e.t.c.?
2017:07:22 03:04:53 wilkerlucio @danielcompton thanks for the interest 🙂 for that I was thinking more like when you do generator overrides, like you do on specs, I think the main use case would be to set a custom parser for a different date/time format depending on the source
2017:07:22 05:01:33 thedavidmeister hey, is there a way to and additional predicates onto existing keywords when used in spec/keys for a new def?
2017:07:22 05:02:35 thedavidmeister e.g. i have :db/id which can be any int? but when i have a :db/id in a (spec/def :item/new ...) i also want to ensure that it is the specific int -1
2017:07:22 05:55:07 alexmiller No, but you can s/and additional constraints over the s/keys
2017:07:22 06:11:26 thedavidmeister @alexmiller like this?
2017:07:22 06:14:05 thedavidmeister but seems to be what i want when i run exercise, thanks 🙂
2017:07:22 06:22:41 alexmiller I would try to avoid the comp valid? … and instead state it as an additional spec or predicate
2017:07:22 06:23:32 alexmiller you can also use s/merge instead of s/and to combine two map specs together
2017:07:22 06:23:47 alexmiller one general, and one more specific
2017:07:22 06:24:52 alexmiller s/keys already checks that the value for the key conforms to the spec, so the extra check in your ::item is duplicating that
2017:07:22 06:28:02 thedavidmeister @alexmiller ah, i'm pretty new to this, could you give me a simple example of that?
2017:07:22 06:29:35 thedavidmeister (spec/def :db/id int?) this is :db/id
2017:07:22 06:29:47 thedavidmeister so that's any int
2017:07:22 06:30:05 thedavidmeister but the item id has to be either a positive int or new-item-id (which is -1)
2017:07:22 06:30:12 thedavidmeister it can't be just any negative int
2017:07:22 06:30:20 alexmiller I think you mostly have it above already
2017:07:22 06:30:33 alexmiller your ::id looks good
2017:07:22 06:31:03 alexmiller and then just (spec/def ::item (spec/keys :req [::id])) is sufficient
2017:07:22 06:31:37 alexmiller spec/keys checks that the map has the required keys and that every registered key spec has a value that conforms to that spec
2017:07:22 06:32:49 thedavidmeister except that when i do exercise it generates maps with ::id instead of :db/id
2017:07:22 06:33:44 thedavidmeister i can't actually pass it ::id because the key :db/id is coming from upstream
2017:07:22 06:45:03 thedavidmeister so, related question
2017:07:22 06:45:03 alexmiller sorry, it was unclear whether those were the same or different things
2017:07:22 06:45:09 thedavidmeister oh well yeah
2017:07:22 06:45:18 thedavidmeister i've got something related
2017:07:22 06:45:23 thedavidmeister i have a :project/id
2017:07:22 06:45:32 thedavidmeister and i want to reference it with :item/project
2017:07:22 06:45:54 thedavidmeister they have the same predicate so i thought i could do (spec/def :item/project :project/id)
2017:07:22 06:45:59 thedavidmeister but it doesn't like that
2017:07:22 06:46:11 alexmiller should work fine - what’s not working?
2017:07:22 06:46:38 thedavidmeister Unable to resolve spec: :project/id
2017:07:22 06:47:01 alexmiller could you give a larger example that produces that?
2017:07:22 06:48:32 thedavidmeister but when :project/id is in a different ns (that i required) it gives the error
2017:07:22 06:48:51 alexmiller can you give that example?
2017:07:22 06:49:44 alexmiller there is a known issue right now with ordering of something like that such that the aliased spec needs to be defined first
2017:07:22 06:49:56 alexmiller are you running into something like that?
2017:07:22 06:50:05 thedavidmeister potentially
2017:07:22 06:50:13 thedavidmeister i'm stopping and starting my test runner now
2017:07:22 06:50:19 thedavidmeister maybe it got confused with me changing things
2017:07:23 08:33:21 schmee this is not needed for s/and, since there is no branching there (all predicates must be true)
2017:07:23 08:35:06 schmee checking that something only contains a certain set of keys is sort of against specs pricinples, since key sets are supposed to be open
2017:07:23 08:35:33 schmee of course you can do that anyway by using a custom predicate fn
2017:07:23 09:35:19 thedavidmeister @schmee i have a map that contains some keys that represent "machine data" like ids/references and some that represent "user data" which is what the user actually typed in
2017:07:23 09:35:52 thedavidmeister when an item has only machine data left in it i want to be able to clean it out of my db
2017:07:23 09:36:11 thedavidmeister because that item no longer represents anything useful to the user
2017:07:23 09:36:20 thedavidmeister so i want a spec that can tell when only machine keys are left
2017:07:23 09:36:46 thedavidmeister so it is open when adding, but closed when making a decision about whether to "clean up"
2017:07:23 09:38:36 schmee that can be done with a predicate fn spec
2017:07:23 09:39:02 schmee but for instance s/keys has no way to say “these keys are not allowed”
2017:07:23 09:40:02 thedavidmeister it's not that they're not allowed
2017:07:23 09:40:04 thedavidmeister i mean they are
2017:07:23 09:40:13 thedavidmeister but that's not the right way to think about it in context
2017:07:23 09:42:47 thedavidmeister i'm trying to detect something not restrict it
2017:07:23 09:49:09 thedavidmeister it's like
2017:07:23 09:49:51 thedavidmeister (if (spec/valid? :item/item--machine-only) (do ...) (do ...))
2017:07:23 09:50:25 thedavidmeister @schmee the extra keys are totally allowed but i do want to treat items that have only machine keys a bit differently
2017:07:23 11:14:08 schmee perhaps something like this would work?
2017:07:23 11:14:35 schmee then you could validate thing separately when needed but still have a spec for the combination
2017:07:23 12:21:52 thedavidmeister @schmee the problem is that ::machine-keys is valid when there are human keys too
2017:07:23 12:22:52 thedavidmeister and human keys is valid for machine keys
2017:07:23 12:36:24 schmee then do something like (and (s/valid? ::machine-keys foo) (not (s/valid? ::human-keys foo))
2017:07:24 00:15:33 thedavidmeister @schmee but then :f would trigger
2017:07:24 00:15:45 thedavidmeister i actually do want my keys to be open
2017:07:24 00:16:27 thedavidmeister i don't want to have to stipulate every human key, and none of them are required, they're all optional, it's just important that there is at least one
2017:07:23 10:52:58 thedavidmeister @schmee also, when you're doing spec/or and using keywords it still seems odd to force k/v
2017:07:23 10:53:18 thedavidmeister e.g. (spec/or :foo/bar :foo/bar :a/b :a/b)
2017:07:23 11:11:51 schmee the first keyword there is just a tag
2017:07:23 11:12:30 schmee spec makes you name all the branches so that you can use that in error messages etc.
2017:07:23 12:17:26 thedavidmeister @schmee but if the branch is just a keyword that is already registered it already has a tag, so it could use that for error messages
2017:07:23 12:17:42 thedavidmeister hmmm how is it possible that
2017:07:23 12:18:04 thedavidmeister (valid? ::foo a) and (valid? ::bar a) are both true
2017:07:23 12:18:23 thedavidmeister but (valid? (spec/and ::foo ::bar) a) is false?
2017:07:23 12:37:31 schmee it would be weird if there was a different syntax for registered keywords, better to have a uniform interface and accept the extra typing
2017:07:23 12:38:01 schmee you can make a macro that does what you want
2017:07:23 13:10:41 thedavidmeister @schmee mmk, any idea why spec/and can be false when each of the individual items are true?
2017:07:23 13:11:52 schmee well, in your example you test that a is valid for ::foo and b is valid for ::bar
2017:07:23 13:12:09 schmee then you test if ::foo and ::bar is true for b
2017:07:23 13:12:31 schmee so you’re testing two different things
2017:07:23 13:12:35 schmee the first doesn’t imply the second
2017:07:23 13:22:49 thedavidmeister @schmee typo, it's all supposed to be a
2017:07:23 13:31:34 thedavidmeister but does it matter?
2017:07:23 13:31:44 thedavidmeister how can (and true true) be false?
2017:07:23 13:36:20 schmee that is a good question
2017:07:23 13:46:01 thedavidmeister interestingly, {:item/project #uuid "4e2db038-f64d-4805-95cf-720e9e7f1419", :db/id 1, :item/title "85", :item/list-id {:db/id 10}} was generated by exercise for the spec/and
2017:07:23 13:48:40 thedavidmeister wait, that's a lie 😛
2017:07:23 13:48:58 thedavidmeister i generated it with slightly different code
2017:07:23 13:54:46 thedavidmeister hmmm, so if i do exercise for :item/item--exists or :item/item--user-input independently i can generate :item/list-id
2017:07:23 13:54:51 thedavidmeister but together i cannot
2017:07:24 19:39:40 devth continually bitten by not knowing if instrumentation is on during interactive dev (e.g. turn it on, then redefine a symbol - now it's off) and tests for a given symbol. i think it's on, then find out i had an invalid spec in a lib for weeks because the upstream lib turned on instrumentation and threw an exception. 🤔
2017:07:24 19:43:50 caio mutable shared state is the root of all evil 🙂
2017:07:24 19:40:12 devth i think i want a single jvm flag to turn it on globally
2017:07:24 19:47:35 wilkerlucio @devth a flag would be complicated, if you think about how instrumentation works, it actually overrides your var, much like a monkey patch in Ruby if you are familiar with, so it would have to re-trigger on every def situation. a simpler solution would be to find a way to handle it on your editor, make it automatically call s/instrument on file save or something
2017:07:24 19:48:11 devth familiar with how it works and monkey patching. that puts the onus of using this thing correctly on users, and the many tools/editors they use
2017:07:24 19:50:14 wilkerlucio yup
2017:07:24 19:50:52 devth so i'm not satisfied with that solution 😛
2017:07:24 19:52:53 devth i kinda like how schema had their own s/defn. that would provide the means to flip a global switch.
2017:07:24 19:53:11 devth yes i know i'm free to build my own wrapper / macro / lib thing
2017:07:24 19:53:33 devth but first have to answer "should i build my own wrapper marcro lib thing?"
2017:07:24 22:11:45 wilkerlucio the main issue on having a global toggle would be that it requires a conditional test overhead, imagine if you have to had this overhead on each function call to check if the parameters should be validated or not
2017:07:24 22:12:08 wilkerlucio that can be quite expensive, the way instrumentation works right now adds no overhead at all when you are not using it
2017:07:24 22:12:47 wilkerlucio currently can be annoying to use, but editors and other features on top of it can solve this issue, while having the global check maybe not have a way to solve from the user code base
2017:07:24 23:41:44 devth with a macro it could be zero cost
2017:07:24 19:57:40 tetriscodes Hello, I’m trying to gen some JSON for the Netflix Eureka API and it has @‘s in the keys
2017:07:24 20:18:44 moxaj hopefully you don't have too many references to the enabled spec ^^
2017:07:24 20:32:37 tetriscodes still no enabled key
2017:07:24 20:32:55 tetriscodes spec/keys takes keys that are registered in the catalog as a spec
2017:07:24 20:34:44 avi hi all 👋 anyone happen to know why clojure.spec.gen doesn’t include let, and if there’s an idiomatic approach to doing what clojure.test.check.generators/let does?
2017:07:24 20:36:44 tetriscodes How are you using it?
2017:07:24 20:37:25 avi
(def ^:private gen-datetime
(tc-gen/let [year (gen/large-integer* {:min 2017 :max 2117}) ; we should be so lucky
month (gen/large-integer* {:min 1 :max 12})
day (gen/large-integer* {:min 1 :max 28}) ; damn you February
hour (gen/large-integer* {:min 0 :max 23})
minute (gen/large-integer* {:min 0 :max 59})
second (gen/large-integer* {:min 0 :max 59})]
(str (string/join "/" [year month day])
" "
(string/join ":" [hour minute second]))))
2017:07:24 20:38:58 avi hmmm maybe there’s an altogether better approach here… I should probably just generate a unix timestamp integer within a certain range, then use clj-time to convert it into the desired string format…
2017:07:24 20:39:06 avi I’m still curious about let though
2017:07:24 20:42:26 tetriscodes you could use spec/with-gen to make your clause a regex and then have your generator create the date formatted as a string and then use clojure.spec.gen/sample to generate data.
2017:07:24 20:44:07 avi Thanks! Not sure I 100% follow but I’ll try to parse that (mentally) and eventually report back. Thank you!
2017:07:24 20:44:32 tetriscodes Looks like you are trying to generate data
2017:07:24 20:45:37 tetriscodes So writing the generator can create some random data for you, or the library test.chuck has a string-from-regex function that you can use but it may be harder to get the ranges on your numbers.
2017:07:24 20:45:55 avi aha, string-from-regex sounds promising, I didn’t know about that!
I keep getting errors like this when I run stest/check on the namespace this is in:
java.util.concurrent.ExecutionException: clojure.lang.ArityException: Wrong number of args (1) passed to: events/fn--4703
clojure.lang.ArityException: Wrong number of args (1) passed to: events/fn--4703
Not sure what’s happening, because when I define these specs in the REPL they work just fine with gen/generate …
2017:07:24 23:47:24 avi ah crap I figured it out… the spec should have been #(instance? % InputStream)
2017:07:24 23:47:28 avi picard-facepalm
2017:07:24 23:48:20 avi or, rather (partial instance? InputStream)
2017:07:24 23:48:25 avi picard-facepalmpicard-facepalm
2017:07:25 02:21:27 jpmonettas @bbrinck I remember yo were working on a lib for formatting clojure.spec errors, I've been also experimenting with the same but with GUIs
2017:07:25 12:54:44 misha btw, what are more suitable use cases for s/cat as opposed to s/tuple?
2017:07:25 12:58:20 misha I think s/cat is suitable when you need to destructure seq into map, for example for clojure.pprint/print-table
2017:07:25 13:17:24 schmee @misha use s/? for variable length tuples
2017:07:25 13:18:19 schmee s/cat requires you to tag the elements which can be useful for conforming while s/tuple doesn’t
2017:07:25 13:19:17 misha @schmee s/?s/+ expect homogeneous elements as far as I can tell reading docs.
I, on the other hand, need to spec a seq where particular elements are of specific types
2017:07:25 13:19:46 schmee not sure what that means, can you give an example?
2017:07:25 13:20:41 misha
(defmacro +
"Returns a regex op that matches one or more values matching
pred. Produces a vector of matches"
[pred-form]
2017:07:25 13:22:03 misha it'd be ok to use it to spec, say, seq of keywords.
but I need to specify, seq of
[int keywords type-a type-b bool] or
[int keywords type-a type-b] or
[int keywords type-a]
2017:07:25 13:22:30 misha (basically each element can have its own spec)
2017:07:25 13:23:06 misha I end up with or and tuple
2017:07:25 13:25:14 misha yeah, I might replace tuple with cat if I will require seq-to-map destructuring.
2017:07:25 13:26:28 misha but in your example you need to wrap it in the s/or and add more "arities", because it'll fail validation on shorter seqs with Insufficient input
2017:07:25 13:27:27 misha that's odd. how did I get "Insufficient input" then?
2017:07:25 13:27:53 schmee can you post your spec?
2017:07:25 13:28:11 misha I wiped it out already opieop
2017:07:25 13:28:35 misha ah, lol, you wrapped each subspec in (s/?)
2017:07:25 13:29:04 misha your spec probably will accept missing-element-specs too.
2017:07:25 13:29:15 misha like [e a t]
2017:07:25 13:29:47 misha which are not the droids I am looking for
2017:07:25 13:30:21 schmee dang, you got me on that one
2017:07:25 13:32:39 misha the only thing I don't like about or - those :6-tuple labels always feel like a hack.
It's like I put effort into designing/naming specs, but those labels? just barf some random name out to make compiler stop complaining.
2017:07:25 13:33:49 misha (or rather about me using or and similar specs)
2017:07:25 13:36:23 schmee well, it’s great for some things, like when writing parsers and you want to dispatch on the tags
2017:07:25 13:36:23 schmee well, it’s great for some things, like when writing parsers and you want to dispatch on the tags
2017:07:25 13:38:07 hmaurer can you give an example of that please? I am interested
2017:07:25 13:40:15 misha when you conform, and then pass conformed value to any multimethod
2017:07:25 13:45:21 schmee I did a toy macro to create maps which has some special syntax for not including nil values
2017:07:25 13:45:40 schmee first I wrote regular clojure code to parse it, but then I thought, why not do it with spec?
2017:07:25 13:45:57 schmee so the conformed input looks like this:
2017:07:25 13:46:44 schmee so you can see the :any, :maybe and :pred tags in there which come from s/cat specs
2017:07:25 13:47:01 schmee which I then use to dispatch and parse to the appropriate form
2017:07:25 13:48:59 schmee slack is acting up on me, excuse the double posts
2017:07:25 13:51:37 hmaurer @schmee it’s acting up on me too; probably a server issue
2017:07:25 13:51:59 hmaurer thanks for the explanation 🙂
2017:07:25 13:36:35 schmee but in your case I agree it doesn’t add much value
2017:07:25 13:38:16 misha I just have not had enough experience with spec to be actually using those tags much "later" in the code, so I don't really have naming intuition developed yet.
2017:07:25 14:23:00 misha are there any apparent downsides of reusing spec names as dispatch keys in s/or/`s/cat`?
2017:07:25 15:50:04 alexmiller there is a currently undocumented s/nonconforming that you could wrap around the s/or for that
2017:07:25 16:08:23 misha the exact problem I have in the s/or above is "Assert failed: spec/or expects k1 p1 k2 p2..., where ks are keywords", and I used booleans hoping to get this kind of coercion
2017:07:25 16:33:29 alexmiller oh sorry, I didn’t even read your spec! the ks need to be keywords. And sets of falsey values won’t work (as they will return falsey values even when there’s a match)
2017:07:25 16:38:42 alexmiller hard to suggest an exact rewrite without knowing your goals re conforming
2017:07:25 16:38:43 misha I wrote a conformer for that:
2017:07:25 16:39:29 alexmiller you don’t need a conformer
2017:07:25 16:40:08 alexmiller the problem with conformers is that you are doing a lossy conversion that throws away the original value, and making that decision for all future users of your spec
2017:07:25 16:41:43 misha this is true. I can return [true :fsm/internal] though, right? and write an unformer for such values
2017:07:25 16:42:37 alexmiller I would spec it as (s/def ::ft/internal? (s/or :b (s/nilable boolean?) :k #{:fsm/external :fsm/internal}))
2017:07:25 16:42:59 alexmiller the tagged result tells you how to coerce the value if needed
2017:07:25 16:43:17 misha I just wanted to avoid dispatching on arbitrary keyword later, instead of doing something like:
2017:07:25 16:44:05 alexmiller I would at the very least write the spec as I have it above, then write a second spec that applies a conformer to the result of the first if needed
2017:07:25 16:44:18 misha yeah, and so I'd need to
(->> huge-map ... ::ft/internal? coerce-internal)
2017:07:25 16:44:29 alexmiller yeah
2017:07:25 16:46:54 misha just going through the options I have atm. Baked in conformer is indeed "bleh" comparing to just using coerce-fn there. I just don't really know how many client call sites there will be, don't want to forget calling coercion somewhere.
2017:07:25 20:26:36 avi 👋 Hi all! I’m new to spec and I’ve got a fairly complex fdef that’s taking a very long time to run… ~30 seconds. Are there any tips, articles, best practices, etc on debugging this sort of thing? Thanks!
2017:07:25 20:27:12 avi (I can try to reduce it down to a gist if anyone’s interested in seeing my slow ugly code)
2017:07:25 20:45:51 english @aviflax a gist might be useful. also, what’s taking a long time to run? is it stest/check?
2017:07:25 20:47:03 avi Ah, yes, it is stest/check — thanks!
2017:07:25 20:47:13 avi I will try to put together a gist that isolates the problem
2017:07:26 01:29:13 alexmiller @aviflax there are a couple known issues related to s/coll-of and the other collection specs. if you posted your args spec, I might be able to suggest something
2017:07:26 13:59:05 avi @alexmiller great, thank you! I’m working on a gist now — almost done, but having a snag with tools.deps.alpha. Where/how should I report this snag?
2017:07:26 15:49:57 alexmiller @aviflax The jira link for tools.deps is on the readme page
2017:07:26 15:55:34 avi Got it — will report there. Thanks!
2017:07:26 16:47:05 alexmiller Two things:
1) there is a known issue when using s/coll-of with :kind but without :into (https://dev.clojure.org/jira/browse/CLJ-2103) - this should be fixed soon, but adding your own :into clause will work around it. This could easily be enough.
2) using s/coll-of with a :gen-max option can also help constrain the size of generated collections
2017:07:26 16:48:24 alexmiller actually, #2 may be a non-issue for you since you’re using :count
2017:07:26 16:48:52 alexmiller so I’d recommend in line 20ish, adding :into []
2017:07:26 18:29:44 avi @alexmiller will do — thanks so much!
2017:07:26 19:08:15 gfredericks would a conj talk on test.check be useful? if so, what angle exactly? usage patterns? implementation details? something else?
I haven't really considered this before because I figured that reid had already talked about it a few years back and it hadn't changed much since then, but maybe that's not true and/or not a good reason
2017:07:26 19:30:44 avi @alexmiller I added the :into but it doesn’t seem to have made a difference ¯\(ツ)/¯
2017:07:26 19:35:17 avi adding :gen-max doesn’t seem to make a difference either 😞
2017:07:26 23:01:18 alexmiller Well on the upside, you can rule those out as the problem :)
2017:07:27 00:11:29 avi Should I open a JIRA ticket?
2017:07:27 00:18:03 avi I’ve shrunk down the example just a bit by removing the CSV parsing… it’s still slow 😓
2017:07:26 20:00:32 misha @gfredericks I'd listen about approaching to property based testing, something in between cookbook and best practices, to improve test-design intuition (what ever that means)
2017:07:26 20:01:30 gfredericks @misha cool; I think that's more or less what I meant by "usage patterns", maybe
2017:07:26 20:01:54 misha maybe kappa
2017:07:26 20:10:18 wilkerlucio @gfredericks a fresher way to go about it might be using spec as reference, how to use and extend generators for specs, and how to write tests using those
2017:07:26 20:11:22 gfredericks yeah, intersecting with spec would probably be more relevant. potentially harder for me since I haven't used spec very much
2017:07:26 20:12:11 wilkerlucio you can use that as a reason to get more into spec too 🙂
2017:07:26 20:12:29 gfredericks right 😄
2017:07:26 20:32:03 danielcompton @gfredericks the hardest thing I've found with property based testing is figuring out useful properties that can be calculated, without reimplementing the original function
2017:07:26 20:32:55 gfredericks yeah I think that's common. that would fit better as a pure test.check patterns talk
2017:07:26 22:19:11 dadair is there a way to do nested/consecutive multi-specs? e.g., {:type :a}, then also expect :subtype :b, and now multi-spec based off :subtype?
2017:07:26 23:00:34 alexmiller You can have a multispec return a spec that is another multispec or you can dispatch on both at the same time using a more complicated dispatch function
2017:07:26 23:02:38 alexmiller @gfredericks a talk on building more complex generators would be very useful and relevant to both spec and test.check
2017:07:26 23:05:39 gfredericks @alexmiller so limiting the scope to generators, enabling a deeper exploration?
2017:07:26 23:05:58 alexmiller I think there is plenty to talk about
2017:07:26 23:06:32 alexmiller Property test patterns is good too but less relevant re spec and has somewhat been done
2017:07:26 23:08:12 gfredericks okay cool; that happens to also be probably the most enjoyable for me to put together
2017:07:26 23:20:02 dadair @alexmiller do you have an example of what a multispec returning a multispec would look like? I've tried that and didn't seem to be able to get it to work
2017:07:27 13:22:43 misha @alexmiller @gfredericks I'd listen about "highjacking" specs with generators, and generators strategies in general, e.g. for cases, where you need to define some root rules, and all downstream generators need to inherit it.
Which is, probably, the same issue/situation, as "I wrote a lib, and need to either hardcode some resource to use it implicitly (bad, but easy), or need every function to accept it as an argument (good, but messy verbose)". Think taxi-driver's browser arg, component vs. mount, etc.
One use case would be testing datomic/datascript, where on the one hand you have all the specs about db/`tx-report` structure, but on the other, when you test actual transaction, – an extra layer of testing the data flow is required, like "my temp ids appear in tx-report, and transacted data appears in db". Such "flow" specs can be described, and tested generatively, but it is somewhat messy, and is hard to keep de-complected from "structure" specs.
2017:07:27 13:24:39 gfredericks @misha is this a situation where you're doing a higher level test and want to model some external system?
2017:07:27 13:27:30 misha the closest to real life I got – is datascript testing, which I described. It probably is "higher level testing" from some pov, but feels like essential for any lib a larger than few util functions
2017:07:27 13:29:23 misha You might perceive datascript as being external to the app code, yes. Not sure if I answered your question
2017:07:27 13:31:44 gfredericks Is "flow" vs "structure" partially about one call vs multiple calls?
2017:07:27 13:33:00 misha yes, though, those multiple calls could be a single fn.
2017:07:27 13:33:22 gfredericks Right; okay cool that makes sense
2017:07:27 13:39:44 misha ok, how'd you go about testing datscript/datomic's transact!?
I'd have specs for db, tx-report.
I'd see, if structure conforms the specs.
But then I'd need to test that, all temp-ids are permanent, that mapped temp-ids - are the same and for same entities, as in submitted data.
That extra data in new db - is in fact submitted data.
I feel, like it can be done on "generators" level, and not in just custom test code, which does not leverage, say, spec.
Because after I'm done testing datascript, I want to model it (lol now I understand your question opieop) to enable testing app code build on top of it, and I would not be able to do this with custom test code, which does not leverage generators
2017:07:27 13:42:31 gfredericks @misha so when you say "custom test code" you're thinking of example-based tests, not something like vanilla test.check properties?
2017:07:27 13:42:48 misha yeah, I think so.
2017:07:27 13:44:41 misha I think the question can be narrowed down to "how to use system-wide seed data, and set of rules, which affect all(most) of the generators"
2017:07:27 13:46:52 misha I don't have an intuition developed in this area much, so I might be talking gibberish here and there. That's why I'd listen to something structured on the topic opieop
2017:07:27 13:48:27 gfredericks yeah, it sounds like you need a higher level overview; which is somewhat at odds with alex's more focused talk idea, but it might be that going over certain generator patterns would illuminate other aspects as a side effect
2017:07:27 13:59:18 misha anything helps, and generative testing is so huge, but "under-appreciated" from talks perspective.
Everyone's just "here, you can generate strings, ints, and booleans. Boom! Off you go."
2017:07:27 21:50:32 misha ~alternative to closures and dynamic vars
2017:07:27 22:08:07 gfredericks @misha this is related to test.check generators somehow?
2017:07:27 22:30:59 misha @gfredericks no, but this might be useful in a highjacking generators with seed data. Seemed relevant to what I wrote earlier today...
2017:07:27 22:35:44 gfredericks @misha I'm having trouble figuring out what you mean by hijacking a generator
2017:07:27 22:43:49 misha I am having trouble describing it opieop
2017:07:27 22:47:22 misha I had a problem with my spec usage, where I wanted to have a spec for, say datascript entity. This spec contains datascript id spec, which can specify either id-before-transaction (actual id or temp id), or id-after-transaction (actual id only). So I have this nested spec, at the bottom of which there is an id-spec. For some tests I wanted that id-spec to be id-before-transaction, and for the others – id-after-transaction.
2017:07:27 22:49:08 bfabry ok I can't find a gif to demonstrate it. but imo that sounds like a sign that you have two different things that should have different names
2017:07:27 22:52:12 misha @bfabry that's sort of true, but I doubt, that redefining entire entity spec with only id spec change – is THE solution.
2017:07:27 22:52:26 seancorfield What about s/merge?
2017:07:27 22:53:23 seancorfield Define a datascript-core spec and then have datascript, datascript-before-transaction, and datascript-after-transaction for the three cases where you 1) don't care, 2 & 3) do care.
2017:07:27 22:53:40 bfabry ^
2017:07:27 22:54:03 seancorfield (or possibly you only need datascript and transacted-datascript since the ID in the first one is either anyway)
2017:07:27 22:54:30 seancorfield Given that you have two ID specs, presumably there are situations where you care which type of ID you have?
2017:07:27 22:54:59 seancorfield (if you don't care, you don't need two specs -- you're over-specifying!)
2017:07:27 22:56:17 misha @seancorfield in this example, you are might be 100% right. But I just want to illustrate a situation, where you might have a composed spec more than 2 levels deep, and you want to replace a leaf sub-spec or a generator for it.
2017:07:27 22:57:26 misha in that case I cared about pre/post transact ids, hence the "story"
2017:07:27 22:58:47 misha all of it is to describe "what I'd like to hear about in test.check conj talk" opieop
2017:07:28 17:26:21 bfabry @alexmiller is there anywhere to look to find out which core functions are spec'd / will be spec'd?
2017:07:28 17:26:48 bfabry actually there's an obvious answer to that, just grep for fdef in master, doh
2017:07:28 20:04:45 noprompt does anyone know of existing code which produces fully resolved spec descriptions with respect to the namespace from whence they came?
2017:07:28 20:05:57 noprompt e.g. (spec/describe some-spec) => (clojure.spec.alpha/coll-of foo.bar/bean-sprout?) not (`(coll-of bean-sprout?)`)
2017:07:28 20:06:35 noprompt i’m asking before i embark on the journey.
2017:07:28 20:29:24 jpmonettas if I understood correctly (spec/form some-spec) is what you are looking for
2017:07:28 20:39:44 wilkerlucio @noprompt I did some stuff that does reflection on the specs, I would be interesting to know if you can figure that out. afaik, this can be tricky, because to correctly resolve that you need to know the namespace definitions for the place where the spec was defined (which can be different from the spec namespace itself), so I'm not sure if that is even possible, given that you might not be able to disambiguate the non-qualified names (or even the qualified, considering aliases can change). but anyway, please let me know if you figure it
2017:07:28 20:42:32 jpmonettas @noprompt @wilkerlucio but isn't (spec/form ...) just that?
2017:07:28 20:47:04 wilkerlucio @jpmonettas that returns the original form as data, but it doesn't give you the context about, so imagine that I have a spec like (s/def ::some-spec my-pred?), just by reading the form, how can you tell from where my-pred? comes from?
2017:07:28 20:48:18 wilkerlucio maybe by getting the var from the function reference, might be possible there, but then I have no idea on how to do that on CLJS
2017:07:28 20:51:02 jpmonettas @wilkerlucio sorry still don't get it, s/form returns everything qualified, I think the idea is you can serialize, deserialize specs
2017:07:28 20:52:25 wilkerlucio @jpmonettas ha, you are right, hahha, man, I was assuming it didn't because of what noprompt said, but I did the test now and you are right, it returns with the full name 😅
…if I don’t want to do something gross like a semantic-defying s/or, is my only option to make have two phone_number specs in separate namespaces?
2017:07:30 00:30:04 camdez And, assuming that is right, any advice on where those additional namespaces live? Do you actually go with a proliferation of tiny files? Or create the namespaces without creating files? And then do you alias those namespaces or just go with something like this?:
2017:07:30 01:16:32 bfabry standard disclaimer that if you can figure a way to make your app use namespaced keywords using :req and :opt rather than :req-un and :opt-un you'll probably have a better time in the long run
2017:07:30 02:22:04 camdez Thanks, @bfabry.
A couple small notes for posterity after experimenting a bit…
1. The argument order for alias is actually the other way: (alias 'phone_numbers 'foo.customers.phone_numbers)
2. The namespace actually needs to exist before you can alias it (one could (create-ns 'foo.customers.phone_numbers)), even though the namespaced keywords used for the spec registry don’t necessarily need to exist.
I appreciate the disclaimer but I’m trying to wrap a spec around an API response I don’t control, so it is what it is.
2017:07:30 02:24:22 camdez This all makes me wonder even more how people are actually doing things. Does your spec registry tend to match actual namespaces? Are those in the same namespaces as your code? Or a parallel set of namespaces? Inquiring minds want to know. 😛
2017:07:30 02:27:37 camdez IMHO Spec seems broad enough in its scope to require not just how can I but how should I documentation.
2017:07:30 04:35:56 seancorfield @camdez We have a combination of keywords where the namespace matches a code namespace and others where they just use a unique single-segment prefix.
2017:07:30 04:42:02 camdez @seancorfield Thanks! What’s the deciding factor between the former and the latter?
2017:07:30 04:42:59 seancorfield I think the recommended approach for alias currently, if you want to introduce a new ns is (alias 'alias (create-ns 'the.full.name)) FYI.
2017:07:30 04:43:38 seancorfield @camdez It Depends(tm) 🙂
2017:07:30 04:46:16 seancorfield For specs that are internal to a ns, the obvious choice is just to use ::name and it'll be in that ns. For specs that are intended to be used across nses, then you need a "unique enough" name for the intended usage. So if that's a subsystem of your own application, you can just use a simple qualifier. For example, we use wsbilling for World Singles Billing. You could use a prefix that matched a DB table, for example, or a component in your app.
2017:07:30 04:47:47 camdez @seancorfield Thanks. Good explanation.
2017:07:30 04:47:54 seancorfield For specs that are intended to be more global, then the prefix also needs to be more global.
2017:07:30 04:52:27 seancorfield I think it's a bit early yet with clojure.spec for "best practices" to have truly settled down.
2017:07:30 04:57:23 seancorfield We've been using it heavily in production for quite a while. We've defined the data model for a few systems with it and we use those for both test generation of data and validation of some inputs. We also have a series of coercing specs defined around one of our REST APIs: we s/conform the raw parameters and get either ::s/invalid or the valid parameters conformed to longs, doubles, strings, keywords etc.
2017:07:30 05:17:40 camdez @seancorfield I definitely agree RE best practices but it feels like such a buffet that we definitely need to be having the conversations about what might become best practices.
2017:07:30 05:18:29 camdez Despite some valiant documentation efforts (largely around what’s possible) I think it’s still hard for a spec beginner to know where to start.
2017:07:30 06:27:31 seancorfield Yes, definitely hard. I actually submitted a talk for one of the Clojure conferences about our use of spec, but after talking to Alex Miller, I pulled it because I wasn't sure we were doing things a good way (I've since changed my mind again 🙂 )
2017:07:30 06:28:12 seancorfield (to be fair, I had a few reasons for pulling the talk but that was the main one)
2017:07:30 06:29:09 seancorfield I'm looking forward to hearing other people's spec talks at future Clojure conferences!
2017:07:30 08:19:02 matan Hi all, are we approaching a release of clojure 1.9 with the finalized spec in it?
I still see in the docs a mention of :clojure.spec.alpha/invalid, will I have to change my code to use a differently named value later on or is it just out-of-date documentation?
https://clojure.org/guides/spec
2017:07:30 08:19:04 matan Thanks!
2017:07:30 15:10:10 camdez @matan I’m not speaking with any authority here, but I don’t believe the docs are out of date (see latest here: https://github.com/clojure/spec.alpha). I believe you will have to use a different named value later, but if you’re requiring spec with an alias (e.g. (:require [clojure.spec.alpha :as s])) and using that alias to refer to the keyword (e.g. :s/invalid) then you’ll only need to change the require statement later.
2017:07:30 15:12:52 camdez I’m assuming that code living under an alpha namespace is not subject to the standard guidelines about accretion and breakage (indeed, that’s likely the exact meaning of the alpha namespace), and thus clojure.spec.alpha will become clojure.spec in a non-backwards compatible way.
2017:07:30 18:11:13 stathissideris this is how to generate params for a function based on its spec:
2017:07:31 05:51:51 olivergeorge I'd like to check some javascript data based on a regular spec which uses s/keys and s/coll-of.
2017:07:31 05:52:21 olivergeorge I know it's not in scope for clojure-spec but perhaps someone has written something which does this?
2017:07:31 05:52:59 olivergeorge
(s/assert-obj ::typical-spec #js {:a [1 2 3]})
2017:07:31 06:31:14 ikitommi @camdez if you are dealing with deeply nested legacy models and only want to validate those in the api layer - after which you only transform them to conform your own application specs - you could check out data-specs (in spec-tools lib). Builds on top of clojure.spec. It’s *not* a good/best practise, but food for thought anyway. Something like:
2017:07:31 06:32:34 ikitommi a sample bare-bones http api with it: https://github.com/ikitommi/business, added a :tags (a set of keywords) to demonstrate the automatic coercion (string->keyword & vector->set here)
2017:07:31 12:26:24 camdez Thanks! I’ve used Plumatic (née Prismatic) Schema quite a bit, so this schema-resembles-data approach is quite familiar. But I’ve been using keyword prefixes divorced from the extant namespaces and it’s working pretty well for me, if a tad verbose.
2017:07:31 16:20:57 bfabry @olivergeorge simplest thing that comes to mind is to just js->clj and run spec over the result
2017:07:31 17:50:47 misha @bfabry @olivergeorge js->clj and clj->js are not symmetric, beware
2017:07:31 19:19:30 plins hello everyone, im trying to spec a map where all keys are optional but you should have at least one of them, is this achievable?
2017:08:01 04:55:05 athos @plins I think (s/keys :req-un [(or ::a ::b)]) would be a smarter solution
2017:08:01 10:39:17 misha @athos is this even legal?
2017:08:01 10:41:32 misha it is!
2017:08:01 10:43:08 athos @misha Of course! s/keys docstring refers to this feature.
2017:08:01 10:44:57 misha not sure how I missed it then. anyway: wow, nice
2017:08:01 11:33:28 misha is "mutual exclusiveness" and/or "orthogonality" of a set of the predicates is even approachable, let alone provable automatically (e.g. during testing with spec)? Does it imply complete exhaustion of the possible input combinations?
2017:08:01 11:42:33 misha Context is:
In Harel's charts (or UML state diagrams), state can have multiple transitions to mutually exclusive states, via the same trigger.
What ensures that system ends up in a valid state, is a combination of guards (predicates) on those transitions, which should guarantee mutual exclusiveness of the guard outcome. I want to try to automatically test whether guards on a set of transitions are mutually exclusive. e.g. :
Current state is :A. Some event triggers transition :a. Machine has to switch to either :B or :C, but not both.
I want to make sure guards (`pos-int?`, and neg-int? in this case) are mutually exclusive. Automatically. At compile time.
2017:08:01 11:44:13 misha Is it even possible? Or the best I can do is to just issue a "dude, make sure [:A :B :guard] and [:A :C :guard] are mutually-exclusive" warning?
2017:08:01 11:44:16 gfredericks there's no general solution to that, but you could write a test.check property that asserts it
2017:08:01 11:44:37 gfredericks if you have a meaningful generator for the state
2017:08:01 11:44:55 gfredericks or whatever it is that's getting passed to the guards
2017:08:01 11:45:38 misha @gfredericks the complication with this, is the huge arity of the guards: they need to accept app-db, whole-machine, and event trigger with its args harold
2017:08:01 11:46:56 misha where for machine I may supply generator, but app-db and trigger args structure, contents, and specs are entirely up to user
2017:08:01 11:48:34 gfredericks sounds tough 🙂
you can validate at compile time of course
2017:08:01 11:48:48 misha and for even as simple as a single string arg, exhausting 2 arbitrary predicates (e.g. with regexps inside) is doubtful, I think(?)
2017:08:01 11:49:14 gfredericks but for a turing-complete language you can't in general check if two predicates are mutually exclusive
2017:08:01 11:49:30 gfredericks regexes might be doable, depending on the details
2017:08:01 11:49:38 gfredericks not worth it though
2017:08:01 11:49:53 gfredericks just a theoretical curiosity
2017:08:01 11:49:56 misha I feel... relief 😄
2017:08:01 11:53:41 misha what if the predicate's source code is available? Is there something to, say, reverse-engineer things like (select-keys app-db [:foo :bar]) to reduce relevant input scope? Or at least to deduce, that guards, in fact, are orthogonal (which is useful too, but I forgot why just now)?
2017:08:01 11:54:11 gfredericks definitely for special cases
2017:08:01 11:54:37 gfredericks but the general case impossibility I mentioned earlier applies to source or compiled or whatever else
2017:08:01 11:55:35 misha it'd be useful to cover at least some basic cases, because most of (UI) FSMs would actually use pretty basic guards
2017:08:01 11:59:56 misha (There are at least 2 implementation options: 1) pseudo states, where you define a fork in the transition as a separate state, and just ask to for cond/case guard there. 2) duplicate transitions with separate boolean guards, which (implicitly) need to be mutually-exclusive.
I am exploring (2) at the moment, to make literal FSM definition less verbose.)
2017:08:01 12:32:24 mfikes The solution to my s/cat in s/cat question yesterday was, of course, s/spec:
2017:08:01 15:19:54 jpmonettas if someone finds it useful, just released a version of https://github.com/jpmonettas/inspectable with clojure and clojuresript support, also with an enhanced browser that shows the spec together with a sample
2017:08:02 11:40:33 Niclas Could someone help me understand why for the given example fn
2017:08:02 12:48:16 schmee unless you have a transient collection, for which contains? doesn’t work
2017:08:02 13:39:13 matan @camdez thanks for last week's answer about clojure.spec.alpha/invalid in the context of spec's code stability 🙂
2017:08:02 13:39:59 camdez @matan You’re very welcome. I hope it was helpful.
2017:08:02 13:46:56 matan @camdez yes, it certainly was!
2017:08:02 13:47:12 matan @camdez will I bump into a lot of stuff coming from its "alpha" namespace when using spec today?
2017:08:02 13:47:58 camdez @matan I don’t think you’ll run into much.
2017:08:02 15:04:45 cddr Am I correct in thinking that spec would be the wrong tool to assert relationships between keys in a map.
For example, lets say we're representing a bill split between n folks, can/should you leverage spec to assert that the sum of amounts under the payees key is greater than or equal to the top-level amount?
If not spec, is there some other library that allows you to organize validations like these in a way that lends support to generating good error messages
2017:08:02 15:08:16 mpenet seems fine to me, could be a (s/and (s/keys ...) sum-amounts>=total?)
2017:08:02 15:13:24 mpenet then it's just pattern matching on the explain to return something more "flat" for error messages (depending on your usage), which you can do with/without spec (you could build it with a spec errors conformer that maps to your precise error types)
2017:08:02 15:15:26 ghadi agree with @mpenet. i don't think it's the wrong tool at all. just a predicate!
2017:08:02 15:25:28 cddr Doh! It never occurred to me that (s/keys) could be anded.
2017:08:02 19:48:54 ghadi on the contrary, I really don't think you need a library to describe a datastructure -- this is literally the rationale for clojure.spec
2017:08:02 19:49:10 ghadi what cddr wants is a predicate that applies to a collection
2017:08:02 19:49:48 ghadi Similar to how fspec provides the :fn spec that relates a function's :args and :ret
2017:08:02 19:49:48 caio you're trying to define an invariant about a data structure, not a contract. I like the idea of separating those concepts (thus using a library to define invariants looks like a good idea)
2017:08:02 19:50:36 caio but yeah, that's just being pedantic, so for a simple structure like this, I agree adding a new dependency is probably an overkill
2017:08:02 19:51:34 cddr I definitely prefer using the standard lib when it supports what I'm trying to do. Cheers for the suggestion though. Looks interesting 🙂
2017:08:02 19:53:17 bfabry I think invariant is just a library for defining data structure predicates. if you have to write enough/complicated enough data structure predicates I think it would be useful to use with spec. similar to how specter is useful if you've got to do enough deeply nested data manipulation
2017:08:02 19:53:47 caio @cddr yeah, it is! I'm probably biased towards it because I used it and liked it a lot hahaha. In my case, I had to define invariants on a custom markup language
2017:08:02 19:54:34 caio so spec wasn't going to help there
2017:08:02 21:16:26 rgdelato is it possible to define a spec for map keys without defing a separate spec for each key? I guess what I have in my head is something like:
2017:08:03 01:37:16 bbrinck I’m trying to make a patch to spec.alpha, but I’m running into problems testing my changes (with provided tests) and building a local jar (so I can include in my own projects for further testing). I’m used to doing everything with lein, so I’m less familiar with “bare” clojure development. Does anyone know of a good guide? My googling is failing me.
2017:08:03 01:40:07 gfredericks mvn test maybe?
2017:08:03 01:47:43 bbrinck @gfredericks yes, that works! thank you so much. I’m realizing I never bothered to learn much about the underlying java foundations before jumping straight into lein for managing clj projects.
2017:08:03 16:27:03 bmaddy Does anyone see what I'm doing wrong here?
I would expect (foo 1) to throw an exception with a message about how (int? :bar) is false.
2017:08:03 16:29:13 mpenet you need to call check
2017:08:03 16:29:17 bfabry @bmaddy instrument does not check :ret specs
2017:08:03 16:29:19 mpenet instrument only checks args (sadly)
2017:08:03 16:30:04 mpenet I think everyone hits that one once. Not really intuitive I guess
2017:08:03 16:30:53 mpenet arguably it can be considered a naming issue at least
2017:08:03 16:31:17 bmaddy Interesting.... Is there a reason for instrument not looking at :ret or is it just not implemented?
2017:08:03 16:32:01 mpenet it's intentional
2017:08:03 16:32:38 mpenet I think it's to allow granularity with gen overrides and such, in theory bundling the 2 in a single function isn't really difficult, it's just not provided
2017:08:03 16:35:02 bmaddy Ok. Thanks for the help you two!
2017:08:03 17:28:57 matan Just went through the spec guide today... and looking forward to heavily using it for better code and better testing workflow. How do you control whether to instrument, and whether to run test code reliant chiefly on stest.check, using leiningen?
What would a leiningen workflow look like?
2017:08:03 17:29:31 bmaddy Interesting, thanks @misha. Those comments were very helpful.
2017:08:03 18:44:22 seancorfield @matan We're mostly using spec for defining and validating data structures, rather than testing. I've suggested to people that they call instrument either in their test fixtures or directly in each test namespace as appropriate (to instrument functions in the namespace under test).
2017:08:03 18:45:13 seancorfield I think there are lots of ways to go about using clojure.spec in tests -- and no "best practices" have arisen yet because it's still alpha and early days.
2017:08:03 18:46:26 seancorfield As Rich and others have said, generative testing -- via test.check and clojure.spec.test/check -- should probably be viewed as separate from your "unit testing", since it can take a while and you want "unit tests" to run very quickly (for fast feedback).
2017:08:03 18:47:28 seancorfield So it's probably a good idea to separate those out into "tests" that don't run as a normal part of your "unit" test suite, but can be run via a separate Leiningen/Boot task (so they can still be run automatically as needed).
2017:08:03 18:48:40 seancorfield I will say that with clojure.java.jdbc, which has optional specs, the "unit tests" run much, much slower with instrumentation in place so even that probably needs to be something you optionally enable for testing.
2017:08:03 18:50:53 gfredericks I wish there was a good way of specifying different profiles for test.check properties
2017:08:03 18:51:17 gfredericks you want to do different things with it at different times
2017:08:03 18:52:31 seancorfield For example, clojure.java.jdbc tests on Clojure 1.8 take about 30 seconds (user; 12.5s real) whereas tests on Clojure 1.9 with instrumentation of all java.jdbc functions take 1 minute 30 seconds (user; 1 minute real). So that's a huge overhead.
2017:08:03 18:56:03 royalaid @seancorfield thanks for clarifying that! I have been trying to add specs to an internal tool at where I work and seems like a massive uphill battle to get them to work in the way that I was expecting. Having the generative tests be a separate thing all together make so much more sense
2017:08:03 20:08:51 seancorfield We use s/conform a lot for validating (and coercing) input to our REST API and also for user input validation, where we can pick apart s/explain-data to generate informative error messages. That stuff's all in production.
2017:08:03 20:10:09 seancorfield We use test.check and stest/check for a small handful of what would otherwise be "unit tests" where they don't introduce a drag on our tests. We use s/exercise and rand-nth to get random, conforming input in some of our tests.
2017:08:03 20:11:01 seancorfield So far we've generally kept instrument and stest/check primarily for manual, isolated test runs. We'll probably integrate that based on environment variables or Boot task flags at some point.
2017:08:03 20:12:37 seancorfield @royalaid One thing to be careful of: don't try to spec everything! Use spec where it provides the most leverage, at system boundaries, and for key APIs, and/or key arguments to those APIs. One of the really nice aspects of spec (compared to, say, Typed Clojure) is that you really can opt-in one piece at a time, where you need it most.
2017:08:03 21:24:28 andrewboltachev Hello. Is it possible to specify a complex spec inline (w/o s/defing something)?
2017:08:03 21:30:19 seancorfield @andrewboltachev You mean directly in a s/conform or s/valid? call, for example? Sure, specs are "just" predicates.
2017:08:03 21:31:05 andrewboltachev @seancorfield To be specific, I want to validate a map
2017:08:03 21:31:41 andrewboltachev but e.g. (s/keys :req-un [::a]) refers to a symbol defined in (some) ns
2017:08:03 21:32:01 seancorfield For s/keys you must s/def the keys if you want them validated.
2017:08:03 21:32:01 andrewboltachev i.e. it takes both name and value from it
2017:08:03 21:32:35 seancorfield If you read the spec rationale, it explains why you can't specify both key names and value "types" together.
2017:08:03 21:32:39 andrewboltachev got it, thanks!
looks anyway slightly opinionated solution for me 😉
2017:08:03 21:32:53 seancorfield At least it's the correct opinion 🙂
2017:08:04 06:13:54 matan There's not really something like that, a "correct opinion" for how to provide an API for humans 🙂
2017:08:03 21:35:16 misha d
2017:08:04 02:59:00 bbrinck Besides core.specs.alpha, does anyone know of an open-source project with a fair number of specs?
2017:08:04 06:11:41 matan Thanks for yesterday's short discussion of spec.check and all of that. I guess I'd use custom leiningen tasks and all those related leiningen features, to fiddle my workflow. As for test duration, a quick glimpse reveals that the number of generated inputs/runs is a parameter, and I'd gladly use slightly longer-running tests while controlling it
2017:08:04 14:46:22 yenda Is there a way with spec to validate maps like {:a 1 :b {:c 2}} were a is optional and c is only required when a is not present
2017:08:04 17:23:17 alexmiller you can s/and a custom predicate to make any check you like
2017:08:04 17:23:37 alexmiller something like (s/and (s/keys :opt-un [::a ::b]) #(if (:a %) (contains-key? (:b %) :c) true)) (can be improved, but you get the idea)
2017:08:04 17:24:17 alexmiller I flipped the case on :a but you know
2017:08:04 23:53:59 leongrapenthin Is there a way to prevent spec from checking fspeced lambdas with generative testing when instrumentation is enabled? It's not helpful when those functions are Callbacks that do side effects
2017:08:04 23:58:02 leongrapenthin I'd really like a knob to disable validation of fspeced lambdas through generative testing
2017:08:04 23:58:22 leongrapenthin Augmentation with a wrapper that checks the fspeced lambda during invocation against the fspec would be fine.
2017:08:04 23:58:43 leongrapenthin Would be great, even more so
2017:08:05 00:34:06 leongrapenthin @wilkerlucio Not sure thats what I'm looking for. My case is like this: A global functions :args are speced like this (s/cat ::config (s/keys :req [::on-click-save])), then there is (s/def ::on-click-save (s/fspec :args (s/cat :cust ::customer)))
2017:08:05 00:34:57 leongrapenthin Now when I invoke the global function with {::on-click-save (fn [cust] (persist-on-server cust))}, on-click-save will be invoked a hundred times with generated ::customers by specs validation code
2017:08:05 00:35:26 wilkerlucio ah, true, that's for generators
2017:08:05 00:36:08 wilkerlucio well, not sure if there is an API for that, but something you could do is to change the spec registry directly, maybe that's a bad idea, but I guess it could work
2017:08:05 00:36:16 leongrapenthin Yeah sure
2017:08:05 00:36:29 wilkerlucio its just a map in an atom, nothing fancy
2017:08:05 12:59:06 leongrapenthin @wilkerlucio it appears that setting s/fspec-iterations is a workaround, but it disables it for all
2017:08:05 12:59:18 leongrapenthin setting it to 0
2017:08:06 16:40:58 scaturr How would s/conform work in an http context? Like how would I conform JSON payloads for specs defined with namespaced keys? Like if I have a spec for ::entity/id instead of just :id?
2017:08:07 19:14:19 bfabry seems like that should work, does it not?
2017:08:07 19:16:49 souenzzo works 😄. But i think that it's not on docs. I dig into code...
2017:08:08 02:00:49 didibus Can I spec a map in a way where I want it to have at-least one of two keys. So a valid map would be: {:x :y} {:x} or {:y}, but {} would not be valid?
2017:08:08 14:58:08 mbarbieri I'm playing for the first time with spec. I want to generate a spec from a contract map and test if another map is conforming.
As first step I would like that all keys in the map are present in the contract map.
But i read the contract from a json file, converting to unqualified keywords, so
(s/keys :opt-un (keys contract-map))
gives me Assert failed: all keys must be namespace-qualified keywords
Do I have to convert them to qualified? Or is it a better way to handle it?
2017:08:08 15:01:43 misha @mbarbieri :opt-un expects coll of qualified keys (which might point to their specs), -un part indicates that data will have no namespaces in keys.
2017:08:08 15:04:22 mbarbieri @misha ok data will have no namespaces, but not even the keys I'm passing in the spec will have, hence the failure. If I'm not wrong, that exception happen because my code will evaluate in something like:
2017:08:08 15:04:41 mbarbieri
(s/keys :opt-un [:a :b :c])
2017:08:08 15:06:03 misha it is ok for data to have no namespaces, but in spec declaration you need to supply namespaces, so later you can assign actual specs to those qualified keys, and s/keys will check not only keys presence, but will check values against those specs as well
2017:08:08 15:08:37 misha you need to supply qualified keys, yes. how exactly - depends on your use case.
If you are want to generate spec once and put in the source file – just come up with a namespace and list those manually.
If you need to generate that spec anew dynamically every time – that'd seem odd to me
(replaced string with number to illustrate my confusion with 1 as path inside 9(?))
2017:08:08 15:11:22 misha what 1 in :in [:foo/bar 1] is supposed to point to?
2017:08:08 15:11:51 misha how do I actually use this :in path? Which data structure do I apply it to?
2017:08:08 15:12:16 mbarbieri @misha ok thanks
2017:08:08 15:13:33 misha
from
:in - the key path through a nested data val to the failing value. In this example, the top-level value is the one that is failing so this is essentially an empty path and is omitted.
2017:08:08 18:06:37 spinningtopsofdoom I think (s/fdef foo :args (s/cat)) is correct. s/cat constructs a sequence on 0 - N specs (like it's regex namesake) so no specs given to it should mean zero arguments expected
2017:08:08 18:39:13 kenny Is there a best practice you guys are following regarding enabling of instrumentation? I am finding that many times if I don't enable or forget to enable it, my functions will silently fail and I receive an obtuse error message. Normally I would have an assertion at the beginning of the function to produce a clear error message but I figured spec instrumentation would replace this.
2017:08:08 18:43:41 souenzzo subscribe
2017:08:10 12:48:08 souenzzo Bump!
2017:08:10 13:44:52 souenzzo Checkout (clojure.spec.test.alpha/instrumentable-syms) and (stest/instrument)
2017:08:08 20:30:37 souenzzo What is the current status of clojure.spec.specs.alpha? (just curious/studding)
2017:08:08 22:06:44 mrkaspa is there a library to convert clojure.spec/explain-data to something friendly to respond in a json api? I was using the struct library I want something similar
2017:08:08 22:12:55 olivergeorge Has anyone written a spec function coverage tool? e.g. report the percentage of functions with a spec defined by namespace or project.
2017:08:10 13:52:08 souenzzo It's easy to write...
Checkout clojure.spec.test.alpha/instrumentable-syms and clojure.repl/dir-fn
2017:08:10 23:17:53 olivergeorge Thank you I'll try those
2017:08:11 01:50:12 olivergeorge Thanks for the help. I'm writing clojurescript which isn't quite as self aware but I had success on a few fronts.
2017:08:11 01:50:27 olivergeorge Simple case is checking a namespace.
2017:08:11 01:51:03 olivergeorge The "total coverage" option was possible but messy. I had to use cljs.analyzer.api to know what namespace and interns existed.
2017:08:09 01:15:04 olivergeorge @mrkaspa clojure.spec provides a basis for that sort of thing but leaves other tools to craft user friendly messages.
2017:08:09 01:15:10 olivergeorge There are others.
2017:08:09 01:15:40 olivergeorge The other JSON api consideration is that clojure.spec deals in clojure data structures (maps, sequences...) not javascript objects and arrays.
2017:08:09 01:16:29 olivergeorge You can do something like js->clj to translate the data into a format clojure.spec likes
2017:08:09 01:17:43 olivergeorge The error messages might be a little confusing where they say "this keyword is missing" when a JS consumer might expect "this property is missing"
2017:08:09 10:21:25 ikitommi @mrkaspa I'm returning everything (problems + extra info of the cause) to clients, specs can be serialized via s/form. Not sure how usefull this is thou. Spec-tools has utility for adding human readable :reason for specs (like struct does).
2017:08:09 12:03:57 misha how can I properly delete the spec? (undo s/def, actually, undo s/fdef)
2017:08:09 12:08:29 misha use case is:
first, I get some function symbols from config, resolve them vars, and see if those are actually functions.
next, I want to enforce common function signature on those, so I want to dynamically do (s/def 'some-handler :lib.specs/handler).
This is all cool, but I want to clean specs registry from those dynamically set specs on some tear down event (some reload in REPL, or what not).
2017:08:09 12:11:18 misha (or at least to know I don't have to do cleaning up at all, because spec registry thing was designed to not worry about it.)
2017:08:09 12:16:27 misha actually, I control the call site, and can ensure args are ok, but does fspec actually check the arity of a function? or does it check args, and if a function (not args) does not conform to the spec – it will blow up, but not in the spec-way, but in usual invalid arity one?
2017:08:09 12:39:06 misha so the best UX I can provide, is to make "handler" fspec available, so user could instrument handlers during development at will.
2017:08:09 13:37:46 misha Generating queue, am I doing it right?
2017:08:09 14:01:28 bbrinck @misha Looks OK to me. I would have written :into PersistentQueue/EMPTY (no parens), but in practice, it doesn’t seem to make a difference
2017:08:09 14:41:12 misha @bbrinck you might be right (originally I had a (q) cljc function call there, edited for brevity). Still need to see how it'd work in cljs.
2017:08:09 14:50:55 misha to create spec for keyword with arbitrary namespace, I need to (create-ns ...) first, right (if there is no file for that ns in a project)?
2017:08:09 15:00:15 bbrinck @misha In CLJS you can do :kind #queue []
2017:08:09 15:01:19 bbrinck @misha I’m not sure if this works for your use case, but specs just need to be namespaced, but that namespace doesn’t have to be a clojure namespace. e.g. (s/def :user/name string?) is totally valid.
2017:08:09 15:02:32 bbrinck The jury is still out on the pros/cons, but I’ve tended to build my specs based on my logical domain, not my Clojure namespaces. I never get to use the :: syntax, but the plus side is that when I print out my specs (or get spec failures), things are a bit more succinct
2017:08:09 15:03:36 bbrinck For the example above, namespacing further by app or company helps avoid conflicts e.g. :my-app.user/name
2017:08:09 15:52:36 misha @bbrinck ah, I went step further, and it complained about aliased namespace, which I need to create before aliasing.
So if I want to use ::u/name, I need to (create-ns 'user) and (alias 'u 'user) first.
2017:08:09 17:43:18 alexmiller that’s totally fine (and something Rich has ideas about ways to improve specifically re working with keyword namespace aliases)
2017:08:09 17:45:29 misha @alexmiller any news on unregistering specs?
2017:08:09 17:45:47 alexmiller yeah, there’s a ticket out there, should be in next spec batch
2017:08:09 17:45:57 alexmiller plan is to have (s/def ::foo nil) do this
2017:08:09 17:56:02 didibus Question: I'm not sure I understand what fmap is for. When would I want to use it. Its one of the few functions that has no doc.
2017:08:09 17:58:33 alexmiller all of the gen namespace functions are dynamically loaded and thus don’t have doc, but they are just aliases for the equivalent function in clojure.test.check.generators
2017:08:09 17:58:59 alexmiller fmap is used to construct a generator based on applying an arbitrary function to some other generator
2017:08:09 22:22:43 ghadi You may want to add that test case if it's substantially different
2017:08:09 22:23:01 ghadi (I haven't looked closely, just aware of the bug)
2017:08:09 22:46:55 dspiteself Thanks
2017:08:09 23:17:37 didibus @alexmiller Thanks. I meant test.check doesn't have a doc for fmap. I get it now though.
2017:08:09 23:26:41 didibus Question: Is there a way I can specify that a spec can be one of a set of specs? Such as (s/def ::user-info #{::name ::address}). Where (s/def ::name (s/keys [::first ::last])) and (s/def ::address [::street ::city ::country]))
2017:08:10 00:37:59 gfredericks @didibus the alpha versions of test.check have a docstring for fmap
2017:08:10 00:43:11 bfabry @didibus I don't think you want multi-spec, that's for multimethods. you just want s/or I think
2017:08:10 00:44:04 didibus Hum, what would multi-spec do that s/or wouldn't?
2017:08:10 00:47:46 bfabry requires a defined multimethod by the looks
2017:08:10 00:48:29 didibus I guess its just that multi-spec is open for extension
2017:08:10 00:49:02 didibus So maybe useful for libraries?
2017:08:10 00:49:29 bfabry seems to require a tag of some sort on your entity
2017:08:10 00:49:37 bfabry which you don't have in your original example?
2017:08:10 00:50:48 didibus Oh right, hum. I guess I see. I think multi-spec is more what I'm imagining in my head, since I'm thinking of it as an inheritance of some sort. Like a shape can be a block square or cube.
2017:08:10 00:51:23 didibus But, I guess I don't really care about the type name, what's that called, when you have types defines by there shape and not a name?
2017:08:10 00:52:21 bfabry in static typing land I believe it's called "structural typing", but I've been writing in dynamic languages for like 8 years at this point
2017:08:10 00:52:27 bfabry so not the person to ask 🙂
2017:08:10 00:53:10 didibus haha, anyways, thanks for the alternative, I'll think about which one works best for what I'm trying to do
2017:08:10 02:46:01 dspiteself @alexmiller I submitted a patch on https://dev.clojure.org/jira/browse/CLJ-2003 . I tried to find a spec.alpha JIRA project. Am I correct to submit the patch to Clojure?
2017:08:10 12:07:01 alexmiller Yep!
2017:08:10 06:50:24 mpenet question of style: when you spec (deeply) nested maps inside a namespace. do you create tons of new deep ns for keys specs corresponding to the map shape ex: ::foo.bar.baz.etc/* or do you use a more flat naming scheme (like inner classes in java) so ::foo$bar$baz/* (spec-tools seems to do something like this for the Schema like api I think)?
2017:08:10 12:09:16 alexmiller I would definitely not do the latter and would potentially not even do the former but instead use the same namespace for many of the levels
2017:08:10 12:21:03 mpenet how would you do that when you spec a json api for instance, there often are duplicates in key names
2017:08:10 12:25:57 misha I try to keep all specs in a single file, at least in active design/development phase, so I end up with something like above (or your 1st approach, @U050SC7SV )
2017:08:10 12:26:09 mpenet yup, involves a lof of create-ns juggling
2017:08:10 12:26:42 mpenet same, I tend to create foo.clj foo_specs.clj pairs
2017:08:10 12:27:13 misha not, if you are willing to use long qualified keywords, instead of aliases.
2017:08:10 12:28:43 mpenet hence the create-ns stuff
2017:08:10 12:28:59 misha I still did not calibrate my preference of aliases over long kws. Converted 300 lines of specs yesterday to aliases – can't understand shit now dafuq
2017:08:10 12:29:42 mpenet I don't abuse aliases (I never create them manually)
2017:08:10 12:30:24 misha now quick, whose ids are those?
::fcb/id and ::fcs/idkappa
2017:08:10 12:30:27 mpenet I just create-ns under the current ns (via a macro) and add dotted childs foo.clj -> :foo/*:foo.bar/*foo.bar.baz/*
2017:08:10 12:33:23 misha I might revert all of the aliases, and settle down with just long qualified kws. Will have to spend ~same amount of time to comprehend same spec, but at least I will not need to jump around the file in process
2017:08:10 12:34:32 mpenet if you do it "locally" aliases are not really necessary you can use ::bar.baz.prout within 'foo ns for foo.bar.baz.prout etc
2017:08:10 12:34:57 mpenet it's what I have found to be the less horrible to deal with that stuff, and semantically it's kinda clean
2017:08:10 12:35:19 misha on the other hand, those namespaces are not that long.
I'd not like to have 1kloc of specs for, say, ring.middleware, where nss are 100 chars long harold
2017:08:10 12:36:23 misha let me try that. (sorry, for highjacking thread, and driving chances of @alexmiller reading all of it to zero opieop )
2017:08:10 12:40:11 misha basically, I just need longer alias names
2017:08:10 07:19:29 mpenet (I usually go with the former)
2017:08:10 15:11:44 misha does not really feel "reuse", man harold
2017:08:10 17:03:55 misha @souenzzo interesting, but I will still require to copy/paste dispatch keys by hand
2017:08:10 19:43:36 didibus Question: What versioning scheme does clojure.spec uses. Specifically, what part of the version do indicates non-backward compatibility if it changes? "1.9.0-alpha17". Is it the number following alpha? Is it the minor 0 version, or the 9 or the 1?
2017:08:10 19:49:35 souenzzo inside -alphaXX, any change can break i think
2017:08:10 20:15:24 seancorfield Also note that Spec is currently clojure.spec.alpha and has a different version number to Clojure itself, as do the also separate core specs.
2017:08:10 20:23:21 didibus Oh I see: 0.1.17
2017:08:10 20:24:06 didibus So, the way they have it now, is there a strict notion of breaking change on the version? Like 0.2 would be breaking, but not 0.1.18 ?
2017:08:10 20:32:11 bfabry I would say while it's still labelled alpha any release could be breaking, as it's more just a "new build"
2017:08:10 21:36:37 seancorfield I get the impression -- from talking to Rich and Alex and others, and from what they've said in public talks at conferences -- that "The Clojure Way" doesn't really pay much attention to version numbers... After all, "major" releases of Clojure are all 1.x so far, and nearly all the Contrib libraries are 0.x.y but are production-ready...
2017:08:10 21:37:23 seancorfield I suspect we'll see a bigger push -- across the whole community -- for non-breaking future releases while the artifact/namespace name is unchanged.
2017:08:10 21:37:36 seancorfield For example, Rich's Spec-ulation talk.
2017:08:10 21:38:51 seancorfield I've struggled with this with clojure.java.jdbc because I've had several releases that have been breaking ones because the API took a long time to drift toward "best practices" as they've evolved over the last 5-6 years I've been maintaining it.
2017:08:10 21:39:49 didibus Its true, but all Clojure versions have historically maintained backwards compatibility. Also, while Rich Hickey doesn't seem to like versions, he does like clearly indicating when things break backward compatibility, though he suggest a whole new name instead of a version bump. At least that's what I got from his talk.
2017:08:10 21:40:32 seancorfield At one point, I pushed the entire API into a "deprecated" namespace, allowing for a simple cross-application edit for users, if they wanted to upgrade without changing all their calls. I think the new way means we'll get "versions" encoded in the namespace names instead, at least until we figure out a better approach.
2017:08:10 21:40:35 didibus Not really an aversion for versions, just that we could do much better I guess
2017:08:10 21:41:13 seancorfield At least with clojure.spec.alpha, you know it's subject to change and can (and has!) cause(d) code breakage.
2017:08:10 21:41:45 seancorfield Once it's ready to become stable, it'll get renamed back to clojure.spec (and folks can continue using clojure.spec.alpha until they're ready to switch).
2017:08:10 21:42:41 didibus Ya, I'm assuming this now. I think its the right assumption for spec when in alpha
2017:08:10 21:42:47 didibus thanks
2017:08:10 21:44:48 seancorfield We've just had a long discussion about this with clj-time because I'd originally pushed to switch its implementation from Joda Time to Java Time. With discussions over about a year, it became clear that wouldn't be possible while retaining the clj-time/clj-time coordinate due to possible conflicts in versions across multiple libraries that depend on clj-time -- so it's sort of become the poster child for "don't break code when you only change the version".
2017:08:10 21:48:47 didibus Hum, ya
2017:08:10 21:50:03 didibus Well, in fact, my reason for asking was that at my company, we internally maintain our own package repo. And our version scheme actually goes like this: Any.Number.Affects.Backwards.Compatibility. And we drop all part of a version that doesn't affect backwards compatibility.
2017:08:10 21:50:24 bfabry lol, I appreciate you not breaking clj-time sean, that would definitely give me a headache
2017:08:10 21:50:53 didibus In that way, conflicts auto-resolve to the newest version that does not break backwards compatibility. But for that, you have to tell the build tool what numbers in the library versioning scheme is the backward breaking one.
2017:08:10 21:52:11 didibus So, I think its actually more in line with Rick's vision. Just freely upgrade things, but don't break people. And if you need to break people, then and only then, just make a fork as a whole new name. So sure, that could be clj-time_v2, but the version is now only a way to indicate incompatibility, and nothing else.
2017:08:10 21:55:21 didibus P.S.: Aren't you stuck using joda-time until clojure drops supports for Java 1.6 and 1.7?
2017:08:10 22:57:11 seancorfield My solution to this at work was to introduce clojure.java-time and start switching code over to use that instead (of clj-time and date-clj which we were previously using).
2017:08:10 22:58:16 seancorfield Given the presence of clojure.java-time, I'm not very inclined to create a "somewhat API-compatible version of clj-time" based on Java Time -- I'd rather support clojure.java-time instead.
2017:08:11 00:11:31 didibus Ah, didn't know about clojure.java-time. Looks good. Though what I'd like to see is a common clojure/clojurescript API for date and time.
2017:08:11 00:16:01 seancorfield Well, there's cljs-time which mirrors clj-time, and in the clojure.java-time repo there's an issue proposing a cljs API that matches...
2017:08:11 00:18:33 didibus Oh, okay ya, I guess that's good enough.
2017:08:11 05:51:19 tianshu Can I add spec to a function before I define the function? like @spec in elixir. And is it possible to have a variable T to represent some type like generic types?
2017:08:11 05:56:46 seancorfield @doglooksgood Not sure what you mean about "generic types" in the context of Clojure?
2017:08:11 05:57:31 seancorfield as for declaring a spec for a function before its defn, yes, that definitely possible...
2017:08:11 06:09:12 tianshu my mistake, I was thought that I have to write spec for function after that is defined.
for my first question, maybe I shouldn't call it a generic type. I want to know something that can help me writing a spec for functions like map.
2017:08:11 06:17:28 seancorfield map is a pretty hard function to write a spec for -- especially since the one argument version returns a transducer 🙂
2017:08:11 06:18:33 seancorfield You don't need to spec everything. Just spec at the boundaries of your subsystems and/or APIs.
2017:08:11 06:20:10 tianshu it seems impossible for writing spec for a function that receive T and returns T?
2017:08:11 16:14:30 alexmiller correct - specs are not parameterized
2017:08:11 17:17:15 didibus I think it doesn't make sense to have parameterized specs. Because they're predicate based, the parameters would be non intuitive. For each one you'd have to pass in a pred, but you wouldn't know how that pred would combine with the spec. And each spec could do it differently. That said, I'm not sure, it's a good idea. I'm trying to think if there's useful real use cases for it, nothing jumps at me right away, can you suggest one?
2017:08:11 18:49:04 alexmiller well, we’re not going to do it, so I’m not going to invent a reason why :)
2017:08:11 06:20:34 seancorfield I think spec is much more valuable for spec'ing data structures than functions, to be honest, but that might be the domain I'm working in (where I'm using spec mostly to describe what API parameters should be, so s/conform and s/invalid? are being called).
2017:08:11 06:20:56 seancorfield So you want the type of the return to match the type of the first argument?
2017:08:11 06:21:32 tianshu yes
2017:08:11 06:21:52 tianshu so I should use :fn in fdef?
2017:08:11 06:21:58 seancorfield You can spec that with :fn to check that yes.
2017:08:11 06:22:58 seancorfield Do you really want identical types tho'? What sort of function are you talking about?
2017:08:11 06:25:54 seancorfield For example, map takes a function and a collection and returns a lazy sequence so the types don't match there -- but I guess you could check non-empty collection/sequence members being the same type? But you can't do much for empty collection/sequence. Would you want to check the whole input/output? What about lazy sequences not being entirely consumed, like (take 5 (map inc (range)))?
2017:08:11 06:26:15 seancorfield You need to be careful that the spec doesn't realize the whole sequence.
2017:08:11 06:43:57 misha @doglooksgood :fn will not be checked by instrumentation. you will need you run tests explicitly.
2017:08:11 06:55:55 tianshu I got a library for this
2017:08:11 06:56:52 tianshu But I think it will be nice to have some spec inference in editor. but without type variable, it seems to be difficult.
2017:08:11 17:25:00 didibus @doglooksgood I'd give a look at https://github.com/arohner/spectrum It gives everything you spec a type based on the spec, and then uses that to validate types at compile time. Maybe you could use it to infer things in an editor.
2017:08:11 19:16:57 arohner I’m slowly working on inference of untyped fns
2017:08:11 19:29:44 arohner also, spectrum isn’t really usable for production use yet
2017:08:11 19:59:46 souenzzo Functions with the signature like +: (fn [& args])...
How to declare theirs specs? (s/fdef my-fn :args (s/coll-of integer?)) ?
(defn ^:skip-wiki nonconforming
"takes a spec and returns a spec that has the same properties except
'conform' returns the original (not the conformed) value. Note, will specize regex ops."
[spec]
2017:08:12 21:38:25 alexmiller As a note here, this was intentionally left out of the docs (the “skip-wiki” meta) because Rich is still mulling whether to include it. Some time has passed since we last talked about it, but at this point I think we should either keep it, or we should have a nonconforming version of s/or (which is the really common case where you want it).
2017:08:13 16:30:48 alexmiller in this case, the data structure does conform to the spec
2017:08:13 16:31:00 alexmiller because s/keys supports open maps
2017:08:13 16:31:56 alexmiller so, essentially no. there is something here Rich is considering adding but not sure if and when that will happen
2017:08:14 06:31:56 misha the use case is to re-use ui-approved-spec in a select-keys to not leak any sensitive or just extra information about entity, exactly because of s/keys's support of open maps
2017:08:14 08:47:58 misha @ikitommi will check it out, thanks
2017:08:14 12:15:57 hmaurer Hi! I have a quick question. I have a spec for a map, which defines a set of required and a set of optional keys (`(s/keys :req [...] :opt [...])`). I would like to generate a version of that spec suitable for “partial updates”. By that I mean that the spec should pass if a required key is missing, but it should not pass if the required key has value nil.
2017:08:14 12:16:02 hmaurer Is that possible?
2017:08:14 12:17:00 hmaurer For a real-world use-case, think of a CRUD scenario. When creating an entity I need to validate the presence of some attributes, but when updating I only need to make sure that an attribute marked as required isn’t set to nil
2017:08:14 12:38:58 alexmiller The vals for the required keys (actually all keys) in s/keys will be verified according to the spec for that key. If the spec does not allow nil, it is not valid.
2017:08:14 12:41:39 alexmiller If the key is required, then its absence will cause validation to fail as required keys are … required. :)
2017:08:14 16:17:18 hmaurer Hi again… I am getting this error whenever I run the “doc” command (and others) in the REPL: java.lang.NoClassDefFoundError: clojure/spec/alpha$get_spec. Any ideas?
2017:08:14 16:18:13 bronsa there's already a ticket + patch on jira
2017:08:14 16:37:26 hmaurer ah, thanks @bronsa
2017:08:15 01:55:49 bbrinck For anyone that uses Orchestra to instrument :fn and :ret specs, Expound 0.2.0 now supports pretty error messages for Orchestra errors. https://github.com/bhb/expound#using-orchestra
2017:08:15 10:37:11 hmaurer (I’ve added the error as a comment on the gist)
2017:08:15 10:37:53 hmaurer I think this is because s/keys is a macro, but I am not sure how to get it to do what I want
2017:08:15 10:37:59 hmaurer (dynamically build that spec from the map)
2017:08:15 10:59:33 moxaj @hmaurer left a comment on the gist
2017:08:15 11:01:00 hmaurer @moxaj thank you! So I assume I cannot / should not define specs at runtime? e.g. I can’t define specs based on my database schema after opening a connection?
2017:08:15 11:01:27 moxaj eval is another option
2017:08:15 11:02:08 hmaurer ah, I’ll try that. Is it an outright terrible idea to define specs at runtime though?
2017:08:15 13:31:36 alexmiller no
2017:08:15 11:10:26 urbank Does anyone here define specs in the same file as the functions? Above the functions like haskell type annotations
2017:08:15 11:16:47 souenzzo I just started to write my specs like that.
Not sure if is the best practice or not.
2017:08:15 11:17:03 gfredericks I usually do
2017:08:15 11:31:13 urbank The second answer here gives a good arguments
2017:08:15 13:02:28 misha I lean towards: data specs in a separate file(s), function specs – next to actual implementation. (Yet to see how it'll turn out in longer run)
2017:08:15 13:08:26 urbank @U051HUZLD Right, that makes sense to. Because functions might share data specs, right?
2017:08:15 13:13:39 misha @U489U338R yes, and, functions are more "implementation detail"-y than data, I think. But I'd start with everything in one file, and see what's up later in project.
2017:08:15 18:58:16 didibus Ya, data spec in their own namespace gives you a clean domain model. And fn specs I put below the functions.
2017:08:15 19:13:22 seancorfield Most of our specs are for data structures and they go in whatever is the appropriate namespace for those data structures -- which is often a separate ns but sometimes is a ns with some "utility" functions specific to the data structure. Where we spec functions we generally put the spec right above the function (like a type annotation).
2017:08:15 19:13:54 seancorfield But, as noted, if you want your code -- a library -- to be usable by Clojure < 1.9 then you have to put specs in a separate and completely optional namespace.
2017:08:15 19:14:18 seancorfield See, for example, how clojure.java.jdbc handles this (specs in a separate namespace on their own -- even for functions).
2017:08:15 14:03:46 stathissideris sure, but I wanted to be able to be strict conditionally
2017:08:15 14:47:23 gfredericks oh sorry, I didn't look closely
2017:08:15 12:41:42 hmaurer Re-iterating my earlier question, does anyone have a big objection to defining specs at runtime (with eval), based, .e.g, on the app’s database schema? I am a beginner so I am not sure if it is “good taste/practice” in the clojure world
2017:08:15 13:27:12 misha it possible, but keep in mind, you cannot yet retract specs from registry as of now @hmaurer
2017:08:15 13:28:17 misha so if your specs will change during application run time - make sure you migrate those appropriately
2017:08:15 13:32:18 alexmiller well, you can if you reach in and munge the registry atom :) but that’s coming soon.
2017:08:15 13:32:59 misha I meant in official way opieop
2017:08:15 13:33:07 alexmiller @hmaurer I think dynamic generation of specs is fine
2017:08:15 16:29:41 scaturr I am having a hard time understanding s/keys. I keep my specs organized in separate namespaces - i.e something like entity.clj and entity/spec.clj. I have the following used to spec a generic entity map
2017:08:15 16:30:40 scaturr I define ::id to make importing it elsewhere easier, and then use it in ::entity/id because its my understanding that is how the :req field in ::entity is able to conform that specific key?
2017:08:15 16:30:59 scaturr is that correct? am I doing this in a weird way?
2017:08:15 16:33:05 scaturr mainly so some other “entity spec” can merge in ::entity
2017:08:15 16:33:36 scaturr which implies todos would need an ::entity/id key
2017:08:15 19:01:33 didibus I'm not sure I follow your logic. It seems to me your overcomplicating it.
2017:08:15 19:15:11 didibus If conceptually the id of entity is supposed to be the same as the id of todo, the you s/def :some-ns/id. And in your spec for entity you say keys req [:some-ns/id]. And do tje same in your todo spec.
2017:08:15 19:39:28 didibus Basically s/keys describes an associative structure which when used with :req says that the structure must have as keys the given list of namespaced keywords. If the keyword has a spec defined, it will be used to validate and conform the value of the key.
2017:08:16 15:31:50 scaturr The todo example does use the :some-ns/id approach
2017:08:16 15:32:22 scaturr But if I were to define (s/def ::entity/id ...) how can I then import that elsewhere?
2017:08:16 15:32:31 scaturr This might just be a misunderstanding of clojure O_o
2017:08:16 15:33:19 scaturr or even if you use :as - since its already namespaced in the spec - is there a way to get at it?
2017:08:16 15:33:28 scaturr thank you for the reply by the way 🙂
2017:08:16 16:43:42 didibus You don't need to refer anything, just require it.
(:require [todos.entity.spec])
Specs are always defined globally. All that needs to happen is your app must eval the s/def form before you use it. Require will do that.
2017:08:16 16:44:34 didibus Also, ::entity/id doesn't make sense. It would be :entity/id
2017:08:16 16:47:47 didibus So specs are namespaced. They're globally accessible from anywhere after they've been deffed, but they need to be prefixed with a namespace. This helps reduce name clash.
2017:08:16 16:49:45 didibus So if you are in namespace todos.entity.spec and you s/def a spec, and you use the double colon ::, then you created :todos.entity.spec/your-spec
2017:08:16 16:51:11 didibus Now if you are in say todos.app namespace, you can require todos.entity.spec and refer to the spec as :todos.entity.spec/your-spec
2017:08:16 16:54:10 didibus If what you want it :entity/id instead of :your-ns/is you have two options
2017:08:16 16:56:47 didibus Either define the spec like that, so don't use the double colon, you can (s/def :entity/id int?) for example
2017:08:16 16:58:22 didibus This will be make it less portable though, since the chance of name clash are higher.
2017:08:16 17:01:05 didibus Or you can define it prefixed with the full namespace using ::, and then when you use it elsewhere, you can alias the namespace as entity.
2017:08:16 17:15:36 didibus `(ns ns1)
(s/def ::id int?)`
Then in ns2
`(ns ns2
(:require [ns1 :as entity])
(s/def ::user :entity/id)`
2017:08:16 17:18:06 didibus As far as I know, you can't refer a spec yet. But you can kinda manually do it if you want
2017:08:15 18:06:32 csm So I have a cat spec that contains a UUID for one field, and another field is a coll-of maps, and each map has a :guid->UUID; the requirement is that one of the maps contain the UUID. I can express this in a spec, but I’m not sure how to make a generator for such a scenario
2017:08:15 19:07:14 csm clojure.spec.gen.alpha/uuid is sufficient for generating UUIDs, the problem is ensuring the first UUID appears in the second collection
2017:08:15 21:15:51 hmaurer Hi. Is there a straightforward way to serialize the result of explain-data to JSON? I am getting an error on trying to serialize spec predicate functions
2017:08:15 22:14:30 didibus Can you serialize to edn instead? That generally works better for Clojure.
2017:08:15 22:53:57 souenzzo cheshire do it without extra settings
2017:08:16 10:41:39 hmaurer @U2J4FRT2T it looks like it cannot encode functions (`:pred` key in the spec problems)
2017:08:16 10:42:13 hmaurer @U0K064KQV I am consuming it from a JS frontend so EDN does not seem ideal
2017:08:16 11:34:30 misha depends on what you plan to do with predicates.
2017:08:16 15:51:28 hmaurer @U051HUZLD not much, I just want a string so I can know why validation failed for a specific field
2017:08:16 15:53:29 misha then (map #(update % :../pred pr-str)) or something, prior to json-serialization
2017:08:16 16:00:32 dergutemoritz Or just use s/explain-str instead of s/explain-data
2017:08:16 16:01:04 dergutemoritz Or would that be too opaque? 🙂
2017:08:16 16:35:01 didibus I always wished cheshire offered custom encoders.
2017:08:16 16:56:39 hmaurer @U06GVE6NR too opaque; I need to know which fields contain errors 🙂
2017:08:16 16:57:23 hmaurer @U051HUZLD I didn’t know the function pr-str; thank you!
2017:08:16 21:27:06 didibus Question: Is there a way I can find the spec of a data-structure at runtime? I know data-structure are not tagged with their spec, but can you like run a structure through all specs and get the spec it validates back?
2017:08:16 21:29:06 gfredericks I bet it'll pass any?
2017:08:16 21:29:14 gfredericks though I guess that's not in the registry
2017:08:16 21:29:40 gfredericks but there could be a lot of keywords in the registry that just have any? as their spec
2017:08:16 21:30:09 didibus Well, I guess it would need to be smarter, like return the strictess spec that it validates or something. Not sure how that could work
2017:08:16 21:30:35 joshjones what specifically do you want to accomplish?
2017:08:16 21:31:30 gfredericks specs like (s/keys :opt [...]) would validate unrelated maps
2017:08:16 21:31:33 didibus I want conditional logic based on the "type" of my data.
2017:08:16 21:32:11 joshjones have you considered using metadata?
2017:08:16 21:32:43 didibus I thought about it, and then I wondered if there's a way to have spec validate that some data-stucture has particular meta on it
2017:08:16 21:33:44 joshjones spec can validate most anything … if you can write a function that tests whether or not something is “valid”, you can use it with spec
2017:08:16 21:34:24 didibus Hum, ya I guess that's true, forgot its just predicates. I was looking for some prefab like s/meta, but I can just build my own
2017:08:16 21:34:26 gfredericks I'd just make it explicit if you can
{:type :something, ...the rest of the data...}
2017:08:16 21:35:09 didibus Ya, I thought I might need to go down that path. It just forces everything to now operate over the data in a special way though.
2017:08:16 21:35:42 didibus I'll try tagging my data with the spec keyword
2017:08:16 21:35:46 didibus in their meta
2017:08:16 21:35:50 didibus see if that works
2017:08:16 21:36:18 joshjones i think @gfredericks meant not using metadata at all, but just using a map with a :type keyword. it seems to me you don’t may not really need spec, or metadata, but maybe just a good ol’ multimethod
2017:08:16 21:36:31 didibus I always worry about using meta like this, because I feel a lot of functions in Clojure don't handle meta very well, like if they modify the data they can return a copy with the meta lost
2017:08:16 21:38:07 didibus @joshjones Ya, I got it. But then everything is a map. Say I have a set, I can't union it directly anymore for example, I have to unwrap and re-wrap manually. Maybe there's a monad I can use to help with this though
2017:08:16 21:39:14 joshjones you don’t need to make it a map to use a multimethod to implement different logic (at the function level) based on your data — but i don’t have any specifics so i’m not quite sure what you really want
2017:08:16 21:40:19 joshjones if you can write a function that can determine what “type” of data a particular input is, that’s your dispatch function — but again, maybe you don’t even want function-level conditional logic, so just guessing
2017:08:16 21:46:01 didibus My problem is I guess the structure and values of my data aren't unique enough. So really I want my data-structures to have a name or a tag of some sort. So say I could distinguish between two vector of ints, because one is an age vector, and one is a height vector.
2017:08:16 21:46:28 didibus Then based on if the vector is of age or of height, I want to do something different
2017:08:16 21:48:32 didibus adding a metadata tag to my vectors would be one way, but like I said, I know some functions over vectors don't preserve meta. I could wrap my vectors in a map and have {:tag :age :value [12 23 34]}, but then for every operation over the vector I have to first unwrap the vector, pass it to the function expecting a vector, then wrap it back again.
2017:08:16 21:50:01 didibus For some reason, spec always fools me in thinking it can do this, because of the name you give to specs, I assume that data of that spec will have that name on them too, but its not the case.
2017:08:16 21:52:17 didibus It would be great if all data-structure could have a pointer to a spec, like that's what I would want. So then given a vector, you could do (spec [1 2 3]) and it would return the spec for [1 2 3]. Unfortunately, it works the other way around.
2017:08:16 21:56:27 seancorfield Your problem there is that your raw data (a vector of numbers) isn't meaningful on its own...
2017:08:16 21:56:55 seancorfield (as you said "...not unique enough...")
2017:08:16 21:58:06 seancorfield Metadata never seems to be a powerful enough solution -- except where it is attached to a Var or to code (that a macro can process). Metadata on values seems outside of the problem space imagined in its design...
2017:08:16 22:01:11 didibus Hum, that's a good rule of thumb for metadata.
2017:08:16 22:04:30 joshjones based purely on data alone, how would you know if a vector is a vector of ages, or heights? my guess is, you don’t really know, so you need to tag it as such somehow.
even if you did what you imagined, (spec [1 2 3]), the spec that you’re utilizing has to know somehow what you say the “type” is … the simplest solution is to use a map. there’s overhead in any solution, whether that’s explicitly declaring a type when it’s created and tailoring functions to only operate on those types, or whether it’s doing what you’ve suggested. you can’t get around the overhead of differentiating data
2017:08:16 22:09:32 didibus I guess I'm not sure what's the best way right now to do that tagging. Basically, I want to add semantic meaning on my data-structures, but I'd like to retain the ability to use all of clojure's functions that operate over that structure. You would think metadata is exactly for this, but not really. Like doing ^:age [1 2 3] would be very natural. So you've got data structured in a vector, and you're giving it semantic meaning by saying it represents the concept of age. So what are other options I have?
2017:08:16 22:18:47 joshjones what is the actual source of these vectors? where are they coming from?
2017:08:16 22:22:45 joshjones sometimes people will have a function return different shapes/types of data, and the advice is usually to have a function return a consistent shape/type. in the same way, if your data comes from a source which can itself be split such that when you receive data, you already know what it looks like, this is an option.
for example, at some source level you must know whether this data represents ages or heights. so, depending on your design and requirements, you may be able to ask for only data that represents ages, and then operate with the knowledge that you have ages. ditto for heights.
2017:08:16 22:53:38 seancorfield If you have a TaggedValue record that has a tag and a value then you could have functions that accepted functions and turned them into tag-preserving operations (and could verify consistency of tags in multiple arguments too).
2017:08:17 19:48:52 alexmiller fyi, there is already tagged-literal and tagged-literal? that use clojure.lang.TaggedLiteral. Those instances also respond to :tag and :value keyword lookups.
2017:08:17 19:49:32 alexmiller and print as tagged literals
2017:08:17 21:32:51 seancorfield Responds to :tag and :form -- I tried :value and got nil. Definitely useful. I did not know about that @U064X3EF3 Thank you!
2017:08:17 21:36:49 alexmiller it’s used by reader conditionals in the case where you read and keep the conditionals, but don’t have a tagged literal to represent a tagged literal version. it would ideally be the fallback reader if you didn’t find a reader rather than the error you get now - you can install that yourself though.
2017:08:16 22:54:18 seancorfield But you're straying off into a hundred different types (with one function each) instead of one type with a hundred functions. So it's beginning to sound non-idiomatic.
2017:08:17 01:55:13 didibus Question: Is there a place on a spec you can add a doc-string, so that it is printed when the spec fails?
2017:08:17 01:55:46 didibus I would like to have literate english prose explain what the spec predicates are asserting.
2017:08:17 03:19:04 seancorfield I believe there's a JIRA issue open to consider docstrings for specs -- but it wouldn't be printed (by default) when a spec failed.
2017:08:17 04:50:29 mpenet There s a jira for metadata as well
2017:08:17 04:59:26 seancorfield I think better third-party "explain" functions are the right way to go here.
2017:08:17 05:07:59 seancorfield (but I think that each application is going to need its own customization so I don't know how much generic goodness we're going to get here)
2017:08:17 11:51:17 misha is there builtin predicate for exceptions?
2017:08:17 17:12:24 souenzzo I think that "spec is about what you function can do. You shouldn't specify what your function cant do."
(under quote cos it's about what I understand when I see the core team talking about spec)
2017:08:17 11:57:47 misha
(s/def ::throwable #(instance? Throwable %))
?
2017:08:17 14:06:53 scaturr I asked this a bit ago but I have yet to stumble upon a solution 🙂 - is there a way to specify a function of arity 1? If i have a function that takes another function as an argument - and that function argument is expected to take a single argument?
2017:08:17 14:52:31 scaturr @gfredericks I’ll give that a whirl. Thank you!
2017:08:17 15:03:42 samueldev how can I check if something is a spec?
2017:08:17 15:03:51 samueldev I am writing a DSL and one of the values needs to be a clojure.spec
2017:08:17 19:43:18 alexmiller s/spec?
2017:08:17 16:36:56 samueldev I'm opting for (s/get-spec) for my problem
2017:08:17 17:58:35 joshmiller Is there an equivalent to s/describe for fdef‘ed fns?
2017:08:17 18:16:08 souenzzo "(s/describe `rules.core/rules-in-eid)"works for me
2017:08:17 18:19:20 joshmiller Oh, duh, I didn’t quote the fn name so it evaluated it. Thanks!
2017:08:17 19:44:04 alexmiller also note that (doc rules.core/rules-in-eid) will print the function specs
2017:08:17 19:44:21 alexmiller as part of the docstring
2017:08:17 19:44:43 alexmiller doc also knows about specs so (doc ::foo) works for data specs
2017:08:17 20:00:30 joshmiller Oh awesome, didn’t know about that either. Thanks!
2017:08:17 21:48:50 souenzzo (s/cat :foo string? :bar (s/spec (s/cat :barbar string?))) is it right? there is a shorter way? I'm not sure if s/spec + s/cat is right
["foo string" ["barbar string"]] << sample
2017:08:17 21:50:36 joshjones 👍:skin-tone-2: ^^
2017:08:17 21:52:51 bfabry @souenzzo the second half could be (s/tuple string?) if you are specifically expecting a vector of length 1
2017:08:17 21:56:00 souenzzo But tuple is some kind of anonymous
Thinking in docs (in my case, the main propose), give a name for the elements of the tuple is better...
2017:08:17 21:58:57 souenzzo (in the real case, it's a tuple 3 elements where each is a different tuple of 3 elements)
2017:08:17 22:09:09 bfabry aye, it's very dependent on your use case. with that much nesting I'd say names could definitely be helpful
2017:08:17 21:53:55 bfabry (without the need for wrapping in s/spec)
2017:08:18 10:08:15 lmergen what would be the idiomatic way to override certain spec generators just for my test cases ? it feels a bit icky to be doing this in my source namespaces
2017:08:18 10:08:41 lmergen i’m thinking of doing a couple of s/with-gen‘s in my test namespace to update the existing spec definitions, would this work ?
2017:08:18 10:11:58 lmergen oh, i just noticed s/gen accepts a gen overrides map
2017:08:18 10:12:07 lmergen and that answers my question 🙂
2017:08:19 01:48:24 potetm So, I would like to do something like #"[abc]{0,9}d?" in a spec regular expression.
2017:08:19 01:49:41 potetm Is there a recommended way of doing a {0,9} quantifier?
2017:08:19 01:50:11 potetm Or do I just have to check the conformed value in a separate predicate?
2017:08:19 03:51:11 ghadi spec/& is what you want @potetm
2017:08:19 03:54:57 potetm Thanks @ghadi!
2017:08:19 07:35:41 misha how do I conform strings with spec? I don't? Or should I just split string into some data structure first, and then proceed with spec?
an example would be a csv file (let's pretend there is no parser for csv)
2017:08:19 09:07:07 seantempesta Is there a tool to generate documentation based on Spec’ed functions?
2017:08:19 14:10:08 pwrflx hi! is it possible to share specs between clojure and clojurescript?
2017:08:19 14:41:02 scaturr it sure is!
2017:08:19 14:41:13 scaturr as long as the underlying predicates work on both platforms
2017:08:19 14:42:03 scaturr you can use reader conditionals to include the cljs version
2017:08:20 16:17:38 souenzzo How to fdef a multimethod?
I will need my-mm and my-mm', right?
Is there some standard pattern? namming pattern?
2017:08:20 20:33:58 didibus @misha Hum, I swear I thought I had tried it, oh well, you're right
2017:08:21 00:45:28 alexmiller @souenzzo multimethods are not currently support with spec fdef
2017:08:21 13:31:50 mgrbyte @alexmiller Does ☝️ also hold for protocols? currently trying to spec a protocol fn that dispatches on type with s/or and not succeeding...
2017:08:21 13:37:10 ghadi i think so @mgrbyte
2017:08:21 14:17:31 alexmiller yes
2017:08:21 14:18:22 alexmiller functions that can’t be instrumented right now include: multimethods, protocols, inline functions, and primitive type hinted functions
2017:08:21 14:18:33 alexmiller some of those are fixable, some aren’t (easily)
2017:08:21 15:25:08 mgrbyte attempting to spec a fn that has one argument, but can accept either a string or keyword, my attempt:
2017:08:22 10:58:29 cddr Is there any way to leverage spec while writing "interpreters/transpilers" like hiccup which seem to be typically implemented by writing a defmethod for each operator.
One disadvantage I've noticed about having these things as data is that when authoring the data, you don't get niceties like editor completion that you would get if it was actual code.
2017:08:22 13:30:13 cddr Oh. I see this was discussed literally yesterday. Sorry. Are multi-methods one of the cases that aren't easily fixable?
2017:08:22 15:08:21 alexmiller I think it’s possible to make them work, but haven’t worked on doing so.
2017:08:24 07:35:03 cddr It occurred to me today that for my use-case, I think I can work around it with a macro that expands to a (defmethod, defn) pair and then write fspecs against the defn.
2017:08:24 14:07:01 alexmiller yes, that should work
2017:08:22 14:19:59 yonatanel Is it customary to spec string length with coll-of characters with :count?
2017:08:22 15:09:09 alexmiller In most cases, I wouldn’t use coll-of for strings as that implies anything that is a sequential view of characters (lists, vectors, seqs)
2017:08:22 15:09:47 alexmiller prob better to do something like (s/and string? #(= 3 (count %)))
2017:08:22 18:27:41 souenzzo @alexmiller is almost possible* specify strings/routing with spec
For example:
(s/or :nums (s/coll-of #{"1" "2" "3" "4"}) :not-nums (s/coll-of #{"a" "e" "i" "o" "u"}))
* with almost possible I mean: you can do with collections of chars.
There is plans to do (s/string string-spec) or something like?
2017:08:22 18:29:15 alexmiller no plans currently
2017:08:22 18:30:15 alexmiller we already have string regex matching - use that
2017:08:22 18:30:39 alexmiller spec’s regex stuff will never be anywhere near as fast or as full-featured as the excellent regex engine already built into java
2017:08:22 18:31:18 alexmiller if you want gen, Gary’s regex stuff in test.chuck is pretty cool
2017:08:22 19:11:13 seancorfield Yeah, a big +1 for test.chuck's regex generator -- we love that!
2017:08:22 20:16:19 souenzzo I thought about using the specifications to do string routing.
Trying to do something better than bidi, I realized spec would do that.
Bidi gives you a data/dsl form do describe strings [[:foo :bar :my-path1] [:foo :my-path0]]
Then, bidi "conform" this string and returns the conformed [:my-path1 {:foo "username" :bar "project-name"}]
It's very close to spec (s/or :my-path (s/cat :foo string-wo-bar? :bar string-wo-bar?))...
2017:08:22 20:24:16 dealy Hi, I'm just getting started w/spec. I just added clojure 1.9 to my project.clj and immediately get a huge spec error at startup. Is it usually difficult to upgrade a project from 1.8 to 1.9?
2017:08:22 20:26:36 hiredman kind of hard to say, in general, anything that spec complains about 1.9 is also broken in 1.8, 1.8 just isn't checking
2017:08:22 20:29:51 dealy It seems to be failing on my require statement, the error msg doesn't hlep much: In: [2] val: (quote :as) fails at: [:args :exclude :op :spec] predicate: #{:exclude}
2017:08:22 20:30:01 dealy it repeats that error several times
2017:08:22 20:30:21 dealy its on the first line of the first file it tries to compile
2017:08:22 20:30:47 seancorfield What does your require look like?
2017:08:22 20:32:09 dealy its about 21 lines, should I post it here?
2017:08:22 20:32:27 seancorfield Use a snippet (via the + button on the left)
2017:08:22 20:33:07 hiredman are you using core.async I vaguely recall there being a gnarly bug that looks something like that if you aren't on the latest or so release
2017:08:22 20:35:22 seancorfield Interesting -- core.async isn't listed on that page above so I guess it had been updated/fixed before that page was created -- good catch @hiredman !
2017:08:22 21:02:01 seancorfield Yeah, I suspect a lot of people working with core.async in any way are adding it as an exclusion and pulling in a new version directly.
2017:08:22 20:35:26 hiredman so that is an example of a similar issue, fixed by bumping core.async versions
2017:08:22 20:36:34 dealy well that helped some, at least the program went further before spewing errors, I didn't realize that there were gonna be these kinds of problems just upgrading to 1.9 ugh
2017:08:22 21:02:17 alexmiller 1.9 has specs on core that will find errors that were silently ignored in the past. so, it’s good! but also frustrating at times. many of the most popular libs have been fixed up long ago.
2017:08:22 20:51:56 dealy any idea what is causing this: Error refreshing environment: java.lang.NoSuchMethodError: com.google.common.base.Preconditions.checkState(ZLjava/lang/String;Ljava/lang/Object;)V, compiling:(closure.clj:100:1)
I'm using cljs 1.9.908
2017:08:22 20:57:41 hiredman you should check your build tool for conflicting versions of whatever com.google.common comes from
2017:08:22 21:11:13 dealy it was guava, upgrading to the latest cljs caused a conflict I guess, its working now
2017:08:22 23:43:07 olivergeorge It might be mild OCD but I find it quite inconvenient to come up with qualified keywords when writing function specs.
2017:08:22 23:43:37 olivergeorge I think what I want is a more adhoc/flexible way to create aliases.
2017:08:22 23:46:48 alexmiller Likely coming in the future (but not 1.9)
2017:08:22 23:44:23 olivergeorge This is a ring handler. If I want to spec several handlers in one namespace the ::params key needs to be unique.
2017:08:22 23:44:47 olivergeorge So I end up wanting qualified keywords with a namespace unique to the function symbol
2017:08:22 23:45:01 olivergeorge e.g. ::oauth-success/params
2017:08:22 23:45:38 olivergeorge I can do it long hand. :my-app.handlers.oauth-success/params
2017:08:22 23:46:15 olivergeorge In CLJ I can declare the alias (but not CLJS) and then do ::oauth-success/params but even that is pretty heavy handed.
2017:08:23 01:24:04 didibus Am I right in thinking that the generator for (s/keys :req [(or ::a ::b ::c)]) will always return a map with all keys? I think that will leave out potential edge cases when doing generative testing.
2017:08:23 02:08:28 alexmiller there is a pending enhancement to fix that
2017:08:23 03:27:47 didibus Question: I've got a spec which is a map-of string? to itself. It works, but sometimes the generator takes like 5 seconds or more to generate a single example. Is there a way I can limit the number of recursive generation it does?
2017:08:23 10:06:34 martinklepsch I’m getting errors when starting my repl: clojure.lang.ExceptionInfo: Unable to resolve spec: :app.spec/thing
2017:08:23 10:07:28 martinklepsch I require my app.spec namespace in my main namespace so I’m thinking it should be loaded before other namespaces depending on it.
2017:08:23 10:11:27 rod Do you have a repo you can link to martinklepsch?
2017:08:23 10:11:46 martinklepsch @rod unfortunately not but I may have just found the issue
2017:08:23 10:12:17 martinklepsch another namespace except core required app.spec resulting in it (+ some others) being loaded before the spec namespace was loaded
2017:08:23 17:20:55 jrychter I just realized that multi-specs use the raw data, not conformed data, for their multimethods. I didn't expect that, but now that I think about it, it seems it's the only way to avoid dependency issues. Is this expected, or am I doing something silly? To clarify what I'm talking about:
The defmethod has to be defined on "meta", not :meta, because that's what the database returns and that's what the value is before conforming.
2017:08:23 17:26:43 alexmiller that seems like the expected behavior to me
2017:08:23 17:27:31 alexmiller multi-spec chooses a spec based on the data
2017:08:23 17:27:51 alexmiller note that the multimethod does not need to be based on a keyword though, it can be an arbitrary function
2017:08:23 17:45:51 jrychter Yes, after thinking about this for a moment, I realized that it would be unreasonable to expect anything different. And yet it surprised me, I grew used to thinking of conform as a conversion layer between database representation (JSON in my case) and native Clojure data. I expect the "conversion" to happen first, before the method gets selected.
2017:08:23 17:46:43 jrychter I think I read somewhere that s/conform isn't really intended for this role, but I find it amazingly useful.
2017:08:24 15:42:38 joshjones you could put the class of the promise elsewhere if you wanted to avoid creating a promise each time you ran the predicate
2017:08:24 15:47:40 alexmiller I’m not sure there’s much value in checking just that something is a promise. I presume what is of interest is that it’s a promise that returns something that matches a spec
2017:08:24 15:47:59 alexmiller however, checking for that would require you to block waiting for the promise to be delivered
2017:08:24 15:48:33 alexmiller so I think you instead want to validate the spec at the point where you are already retrieving the value
2017:08:24 15:48:44 alexmiller or where you are delivering the value
2017:08:24 16:09:47 jpmonettas yeah I'm trying to spec a function return that returns a promise with an int?
2017:08:24 16:11:08 jpmonettas since it's a :ret I'm not worried about instrument blocking on deref
2017:08:24 16:11:44 jpmonettas I'm more interested in the doc aspect
2017:08:24 16:35:27 seancorfield Several times I've wanted to be able to spec "this should be an atom containing an X" so I feel the lack @jpmonettas -- but as @alexmiller says, a lot of the times where you would want this, spec would need to deref the container to check the value inside and that's not the correct semantics (it would block).
2017:08:24 17:12:35 alexmiller No, that's different. Atom deref is not blocking
2017:08:24 17:16:11 alexmiller We might add atom-of eventually
2017:08:24 17:17:48 seancorfield I meant more in the general "derefable" situation -- but you're right that each type of derefable is different.
2017:08:24 17:19:12 alexmiller All of the Clojure reference types do nonblocking read
2017:08:24 17:23:12 seancorfield Good point. Future/Promise are the outliers.
2017:08:24 17:27:01 gfredericks and delay
2017:08:24 17:36:45 alexmiller none of Future/Promise/delay are reference types
2017:08:24 17:39:28 alexmiller that is, by “reference type” I mean, IRef (not IDeref)
2017:08:24 16:36:24 seancorfield It would make me want to have the function return int, and instead call it in a future (which has the advantage that the caller controls both the wrapping and the dereference).
2017:08:24 16:36:49 seancorfield (since you don't mind the checker blocking on a deref to check the value anyway)
2017:08:24 16:43:32 mpenet You can put a validator on the atom with a valid? check i guess
2017:08:24 16:43:44 mpenet Wont help with gen tho
2017:08:24 16:59:46 mpenet Golang is interesting for chan args, you can say "this arg is a chan of int that can only be take!en in that scope" etc
2017:08:24 17:00:25 mpenet Chan args have a concept of "direction"
2017:08:24 17:18:01 alexmiller As do core.async chans via the port protocols
2017:08:24 17:18:18 mpenet Yes it s very similar
2017:08:24 17:19:11 mpenet xforms on chans might make specing them more tricky tho
2017:08:25 05:32:25 thedavidmeister when i use spec/exercise for things like string? and pos-int? i mostly get blank or 1 character strings and numbers 1 and 2
2017:08:25 05:32:31 thedavidmeister is there a way to get it to "try harder"
2017:08:25 05:33:16 thedavidmeister and give me longer strings more often, with more variety, like emojis and non english alphanumeric characteres
2017:08:25 05:33:38 thedavidmeister and give me random ints anywhere in the full range of available positive integers?
2017:08:25 05:34:13 thedavidmeister i really like the idea of using spec to help me generate tests, but it doesn't give me a lot of confidence to know that most of my tests are just passing "" and 1
2017:08:25 05:41:54 thedavidmeister oh hmmm
2017:08:25 05:42:27 thedavidmeister i just realised that if i (spec/exercise string? 1) i almost always get ""
2017:08:25 05:42:44 thedavidmeister but if i (spec/exercise string? 100) then the later strings are much longer
2017:08:25 05:44:20 thedavidmeister so i could do something like (ffirst (shuffle (spec/exercise pos-int? 100))) but this seems inefficient...
2017:08:25 05:48:14 seancorfield @thedavidmeister I'm a bit puzzled about how you're running tests here?
2017:08:25 05:48:57 seancorfield Are you using spec.test's check on functions? Or test.check with properties etc?
2017:08:25 05:53:37 thedavidmeister uh, nothing like that atm
2017:08:25 05:53:50 thedavidmeister i've just been working on creating specs for my existing codebase
2017:08:25 05:54:02 thedavidmeister and i've got existing tests that i've handrolled examples for
2017:08:25 05:54:17 thedavidmeister but i thought i could use spec/exercise as a "drop in" replacement for my examples
2017:08:25 05:54:54 seancorfield We have a couple of places where we do (rand-nth (map first (s/exercise ::my-spec 100))) which is an equivalent to what you have so it's a reasonable approach for getting a specific random piece of test data.
2017:08:25 05:56:00 thedavidmeister ok cool
2017:08:25 05:56:07 thedavidmeister just checking that i'm not crazy 🙂
2017:08:25 05:56:08 seancorfield I suspect rand-nth is going to be faster than first of shuffle (and (first (rand-nth ...))` is probably the fastest).
2017:08:25 05:56:18 thedavidmeister is there a way to get it to be lazy?
2017:08:25 05:56:47 seancorfield Lazy how? You only want one value and you want a random one out of 100...
2017:08:25 05:57:09 thedavidmeister yes but it's clearly doing something different for the 100th to the 1st item that it generates
2017:08:25 05:57:49 thedavidmeister i want the logic of the 100th iteration
2017:08:25 05:57:57 thedavidmeister but not the 99 values that come before it
2017:08:25 05:58:27 seancorfield Since we're talking about tests I wouldn't be too concerned about efficiency.
2017:08:25 05:58:36 thedavidmeister i know...
2017:08:25 05:58:40 thedavidmeister it just seems so wasteful
2017:08:25 05:58:54 thedavidmeister i'm not normally "that guy" but...
2017:08:25 06:00:19 seancorfield FWIW, (first (rand-nth (s/exercise string? 100))) takes between 6 and 30ms on my low-powered laptop
2017:08:25 06:00:33 thedavidmeister ok
2017:08:25 06:00:38 thedavidmeister well the other half of my q
2017:08:25 06:00:49 thedavidmeister is there a way to get string? to be less alphanumeric-y?
2017:08:25 06:01:05 seancorfield What do you want it to be?
2017:08:25 06:01:14 thedavidmeister i sort of expected utf-8
2017:08:25 06:01:28 thedavidmeister so :poop: emojis and japanese kanji, etc.
2017:08:25 06:01:58 seancorfield Some of the built-in generators are a bit conservative. Take a look at test.chuck and its regex generator.
2017:08:25 06:02:53 seancorfield We have a crazy email address regex and use test.chuck to produce random email addresses and it generates absolutely wild UTF-8 strings 🙂
2017:08:25 06:04:06 seancorfield We also have a wild password spec (with complex rules about UTF-8 character classes and length etc), but for testing we override to use a much simpler ASCII regex generator.
2017:08:25 06:04:07 thedavidmeister so how do i plug that into spec?
2017:08:25 06:04:18 seancorfield We're testing different things there.
2017:08:25 06:04:43 seancorfield s/exercise lets you specify overrides, or you can use s/with-gen to provide a custom generator for any spec.
2017:08:25 06:05:28 thedavidmeister ok cool, so can i do like (s/with-gen string? (partial string-from-regex #".*")
2017:08:25 06:05:31 thedavidmeister or is that not right?
2017:08:25 06:05:55 seancorfield It needs to be a zero arity function that returns a generator.
2017:08:25 06:09:03 seancorfield We wrap test.chuck so we don't drag it in for production code but it can come in during testing so we have
(defn fn-string-from-regex
"Return a function that produces a generator for the given
regular expression string."
[regex]
(fn []
(require '[com.gfredericks.test.chuck.generators :as xgen])
(let [string-from-regex (resolve 'xgen/string-from-regex)]
(string-from-regex regex))))
2017:08:25 06:09:49 seancorfield (`password-valid?` is a gnarly predicate)
2017:08:25 06:10:18 thedavidmeister ok cool
2017:08:25 06:10:35 thedavidmeister and then once you've done that, exercise will know what to do with ::password
2017:08:25 06:12:06 seancorfield Yup, it will generate based on that regex.
2017:08:25 06:12:34 thedavidmeister that sounds much more thorough than alphanumeric strings
2017:08:25 06:12:36 seancorfield Which will satisfy our rules (`password-valid?`) but much more easily -- and still readable.
2017:08:25 06:14:04 thedavidmeister soz, i have another question while i go through all this
2017:08:25 06:14:14 thedavidmeister is there a predicate for date/times?
2017:08:25 06:14:21 gklijs I needed to use it to be able to even generate maps containing an email
2017:08:25 06:15:41 thedavidmeister @gklijs yeah i'm running into limitations with the default string? generator pretty fast too
2017:08:25 06:22:29 seancorfield @thedavidmeister We have a set of specs and generators for date/times but it's a fair bit of code because we actually need to coerce from strings to dates, and we want to generate formatted dates.
2017:08:25 06:23:00 seancorfield But for basic stuff you can just use inst?
2017:08:25 06:24:02 thedavidmeister oh yeah, i'm slowly working through moving between js dates, goog dates, joda dates and iso8601 strings
2017:08:25 06:24:13 thedavidmeister just wanted something basic for now though 🙂
2017:08:25 06:24:29 thedavidmeister ah inst? looks great
2017:08:25 06:25:23 thedavidmeister although hmmm
2017:08:25 06:25:59 thedavidmeister it seems to strongly prefer 1970-01-01
2017:08:25 06:26:49 thedavidmeister even if i do (first (rand-nth (spec/exercise inst? 1000))) it rarely moves away from 1970
2017:08:25 06:27:14 seancorfield Yeah, the inst? generator is pretty conservative.
2017:08:25 06:27:33 thedavidmeister hmmm
2017:08:25 06:27:38 thedavidmeister know of anything like that regex lib?
2017:08:25 06:28:09 thedavidmeister i feel like even using a random int as a timestamp would be better 😕
2017:08:25 06:29:08 gklijs maybe with dates you want one based on the current date and add/substract some random year/month/seconds etc?
2017:08:25 06:30:31 seancorfield Hence our age range spec / generator 🙂
2017:08:25 06:31:01 thedavidmeister yeah
2017:08:25 06:31:13 thedavidmeister i think i'm going to go for a jog and think about how i can adapt to my stuff 🙂
2017:08:25 06:33:05 seancorfield One thing I've found important with clojure.spec is to be careful not to over-specify things and to override generators to produce more "sane" data (for your problem domain) -- generating just a small subset of possible conforming values sometimes.
2017:08:25 06:34:18 thedavidmeister i think that will be ok for me atm
2017:08:25 06:34:29 thedavidmeister as i'm replacing hard-coded values in tests anyway
2017:08:25 06:34:42 thedavidmeister it will be an incremental process
2017:08:25 06:34:52 thedavidmeister as i gradually introduce more complexity of what is specced
2017:08:25 09:19:51 thegeez @thedavidmeister spec generator have the notion of a size parameter (100 here). However, this is not a minimum length: (clojure.test.check.generators/generate (clojure.spec.alpha/gen string?) 100) and clojure.test.check.generators/resize
2017:08:25 09:21:26 thedavidmeister @thegeez wow that is great
2017:08:25 09:21:29 thedavidmeister thanks!
2017:08:25 15:47:19 martinklepsch how do you express that all keys of a map should be keywords? I want to spec a structure that looks like this:
2017:08:25 15:51:45 martinklepsch @seancorfield thanks, I’ll look into map-of
2017:08:25 18:19:57 ikitommi @martinklepsch map-of is the right answer, but your pseudo looks much like the thing that the data-specs are eating 😉 (and generating map-ofs & friends)
2017:08:25 18:24:30 martinklepsch @ikitommi ah that indeed seems like a nice utility!
2017:08:25 19:46:29 eraserhd So I just discovered that (s/def :foo/bar :baz/quux) doesn't work if :baz/quux hasn't already been defined. Is this new?
2017:08:25 19:46:48 eraserhd Is it just s/keys that defers lookup?
2017:08:25 19:48:02 eraserhd And if so, does that mean that I can't redefine :baz/quux later?
2017:08:25 19:52:07 alexmiller almost everything defers lookup. this is a known (and unresolved) issue.
2017:08:25 19:53:11 eraserhd If I wrap it with s/spec, will it defer?
2017:08:25 19:54:57 alexmiller maybe? try it
2017:08:25 19:55:46 eraserhd apparently... more testing in half an hour or so
2017:08:26 22:46:24 souenzzo there is some wayt to (s/def :req-un [:foo/bar :as :new-name])?
2017:08:26 23:55:56 alexmiller no
2017:08:27 17:00:24 hmaurer Hi! Is there a way to get only the keys specified in a spec (req and opt) and nothing else after validating a map? And to do so with nested maps too?
2017:08:27 17:00:53 hmaurer I expected conform to do this but it doesn't appear to
2017:08:27 21:16:16 hmaurer the spec-tools package seems to offer a solution to my problem
2017:08:27 18:36:27 ikitommi Hi. The s/conformer seems to take the optional unf function as a second argument for unforming. It defaults to nil currently. Would an identity be a better default?
2017:08:27 18:38:36 ikitommi e.g. this would not fail by default:
2017:08:28 00:35:35 alexmiller yes, it would still fail by giving you the wrong result
2017:08:28 00:36:18 alexmiller given that conformers are used to transform the input, using identity is virtually always as wrong as nil. much better to fail and tell you how to fix imo
2017:08:27 21:45:13 hmaurer Anybody familiar with spec-tools here?
2017:08:28 07:21:19 ikitommi @U5ZAJ15P0 I am, one of developers of the lib.
2017:08:28 19:28:41 hmaurer @U055NJ5CC Hi! I wanted to ask if there is a utility function to walk a (deeply nested) map together with its associated spec
2017:08:28 19:29:03 hmaurer does such a function exist now?
2017:08:29 05:01:46 ikitommi @U5ZAJ15P0 Not yet. Just commented on the issue. If you have ideas, please share them on the issue. Contributions welcome too
2017:08:28 03:21:17 danielcompton Is there a way to spec a nested map that dispatches on type? e.g.
(s/def :app/command (s/keys :req [:command/id
:command/params
:request/id
:idempotency/id]))
(defmulti command-params :command/id)
(s/def :command/params (s/multi-spec command-params :command/id))
(defmethod command-params :test/command [_]
(s/keys :req [:material/id]))
;; This is the kind of map shape I want
{:command/id :test/command
:request/id (random-uuid)
:idempotency/id (random-uuid)
:command/params {:test/id (random-uuid)}}
2017:08:28 03:22:57 danielcompton I thought I could do this with a multi-spec, but I'm not sure how to define a multi-spec where the data is at one level, and the tag is at the parent level
2017:08:28 07:28:38 ikitommi Idea: s/conform could be changed into s/walk that could be parametrised to do either :conform (as now) or :coerce (like conform, but would not return the branch-information, e.g. same results as with conform + unform).
2017:08:28 07:29:17 ikitommi I think this would yield much better perf, which matters at runtime.
2017:08:28 13:38:28 scaturr Is there any sort of consensus on where to put specs? Separate ns? Same ns as things being specced?
2017:08:28 15:41:23 souenzzo +subscribe+
2017:08:28 15:59:43 seancorfield It Depends(tm).
2017:08:28 16:00:32 seancorfield Data specs can usually be in their own ns, possibly with a few predicate functions or closely related helper functions.
2017:08:28 16:01:36 seancorfield Function specs probably should be in the same ns as the functions they are spec'ing -- unless you need to support Clojure 1.8 (or earlier) as well as Clojure 1.9, such as for a library, in which case they must go in a separate, optional ns, that only 1.9 users will load.
2017:08:28 16:38:35 scaturr that is super helpful. thank you!
2017:08:28 13:39:46 scaturr Tooling makes me feel like the same ns might be more convenient - not necessarily better?
2017:08:28 13:40:26 scaturr for instance cider’s doc command doesn’t work unless you explicitly have the spec required
2017:08:28 13:40:39 scaturr ¯\(ツ)/¯
2017:08:28 17:49:47 hmaurer is there a way to “override” an already defined spec?
2017:08:28 17:49:51 hmaurer (with s/def)
2017:08:28 17:50:01 hmaurer it’s a no-op for me if the spec is already defined
2017:08:28 18:37:44 seancorfield Remember that s/def will only update the spec -- not all the other specs that have been defined in terms of that spec.
2017:08:29 00:48:18 alexmiller yeah, this should work (once you re-def the other specs)
2017:08:28 17:58:12 hmaurer Found my answer: hacky (reset! @#'spec/registry-ref {})
2017:08:28 18:00:20 hmaurer actually that seems to break things…
2017:08:28 20:17:24 joshjones i’m not shocked
2017:08:29 00:48:54 alexmiller there is a ticket on the way such that (s/def ::foo nil) will be sufficient to remove a spec
2017:08:29 10:55:19 hmaurer @ikitommi I just started working with spec-tools but I am finding it a bit hard to write functions dispatching on (s/form spec) as most are wrapped in spec-tools.core/spec. Am I doing something wrong?
2017:08:29 11:24:26 ikitommi @U5ZAJ15P0 wrapping into spec records is unfortunate, there is https://dev.clojure.org/jira/browse/CLJ-2116 to enable runtime coercion natively. But, yes the original spec is found in :spec
2017:08:29 12:19:17 hmaurer @ikitommi thank you! I’ll take a look. Overall great work on spec-tools by the way; it appears to solve a lot of the issues I had with clojure.spec 🙂
2017:08:29 12:57:23 hmaurer Out of curiosity, what is the consensus regarding issues described here @ikitommi @alexmiller ? https://dev.clojure.org/jira/browse/CLJ-2116. More specifically @alexmiller , you mention
> While I think some interesting problems are described in the post, I don’t agree with most of the approaches being taken there.
Do you have an alternative approach to suggest? I am a beginner with clojure/clojure.spec but I did find that spec-tools answered some real needs of mine (e.g. strip extra keys when conforming a map)
2017:08:29 13:42:26 alexmiller if you want to transform data, then transform data. why does that need to be in spec?
2017:08:29 13:42:43 alexmiller if the selective conforming only happens at the end, why does it need to be in spec?
2017:08:29 13:46:08 alexmiller the purpose of conforming is not transformation or coercion. it’s to tell you how a value conforms to a spec,
2017:08:29 13:58:01 hmaurer Ah I see. So these sort of tools should be built on top of spec, not as a part of it
2017:08:29 13:59:37 hmaurer How would you approach annotating a spec with additional data that can be used later on by utility functions?
2017:08:29 13:59:49 hmaurer (what spec-tools does by wrapping specs in a Spec record)
2017:08:29 14:53:30 alexmiller I would like to have arbitrary meta on specs and we have some tickets around that
2017:08:29 14:54:03 alexmiller but barring that, specs are named with a fully-qualified name - anyone can make a registry with those keys to store arbitrary stuff
2017:08:29 14:59:30 hmaurer @alexmiller good point; I guess those things will come more naturally to me as I get familiar with specs 🙂
2017:08:29 22:33:03 alexmiller Fizzbuzz is not a use case
2017:08:30 01:21:59 tap Do you mean it’s a fun little problem which we can use whatever we want, right?
It’s good to know for me because I misunderstood the purpose of conformer by this. Already did a csv validating and parsing using conform and additional conformers
2017:08:29 16:40:27 seancorfield A use case we've found very valuable is using spec for conforming REST API parameters and form parameters -- which all start off as strings but we need numbers or Boolean etc. For those specs, we coerce string input to conformed values. Then we have a single step to s/conform the input map to a validated parameters map.
2017:08:29 16:41:26 seancorfield So at the core, we'll have things like (try (Long/parseLong v) (catch Exception _ ::s/invalid)) as a named spec (predicate).
2017:08:29 16:43:37 seancorfield I would agree with @alexmiller that coercion/transformation beyond that level is probably "doing it wrong".
2017:08:29 16:46:14 ikitommi @seancorfield So you allow clients to send numbers as strings over JSON too?
2017:08:29 16:47:19 ikitommi (and over EDN or Transit too)
2017:08:29 16:54:22 seancorfield We accepted form-encoded POST bodies so it isn't always JSON.
2017:08:29 16:54:58 hmaurer @seancorfield well, a simple case which brought me to try spec-tools was having the ability to strip all keys from a map that were not part of the spec
2017:08:29 16:55:25 hmaurer I assume that, were I to do this myself, I would have to write a function to traverse my spec together with the data and strip keys appropriately?
2017:08:29 16:57:25 seancorfield Why do you need to strip other keys? The only case I can think of is if you send the full hash map to a 3rd party that would object to the extra keys -- and it's safer to explicitly build the map you need at those points.
2017:08:29 16:58:02 seancorfield If your code never uses :x in a map, why do you care that you're passed :x in the first place, is my question.
2017:08:29 16:58:07 hmaurer precisely that; I am transacting the data to Datomic
2017:08:29 16:58:50 seancorfield But you'd be building a namespace-qualified map specific to your schema at that point -- I wouldn't imagine the namespace-qualifier would be the same as your JSON input?
2017:08:29 16:59:22 hmaurer no, but I have a step which namespace-qualifies my JSON input
2017:08:29 17:01:00 ikitommi I think it's a good default to strip extra keys when reading external input. Like rest/http parameters from client
2017:08:29 18:17:01 souenzzo Bit off, but I have a experiment that I use for example /user/%5B%3Auser%2Fuuid%20%23uuid%22123%22%5D (encoded form of /user/[:user/uuid #uuid"123"]) to avoid transformation problems (simple edn/read-string transform types).
Not sure if it's a good solution/it's sacale, but it's working well on my tests.
2017:08:31 02:11:25 tjscollins I think I'm slowly losing my mind trying to get my function specs to work right.
This does not work if I try stest/check example-fn. Why? I'm clearly missing something. I've been re-reading the guide for two days, but something's not clicking, and I can't see where why s/fdef's are going wrong.
2017:08:31 02:20:09 seancorfield @tjscollins :args is always a sequence of argument specs so you want (s/cat :s string?) there.
2017:08:31 02:25:42 tjscollins How does s/cat work? Do the keys have to match the variable names, or are they just used as informational labels in the output of s/explain?
2017:08:31 02:29:49 tjscollins Yeah, I've read through that repeatedly. It's just not making sense to me how s/cat and s/alt work.
2017:08:31 02:29:56 seancorfield The keywords in s/cat don't have to match the argument names but it's common convention that they do (otherwise your error messages will be less than helpful).
2017:08:31 02:30:51 tjscollins So what is actually happening when the args are checked against s/cat? Do the args pass if they match any spec that's "s/cat"ed together? Match every spec?
2017:08:31 02:31:18 seancorfield s/cat is a "sequence regex" so it matches the sequence of arguments passed into the function
2017:08:31 02:31:19 tjscollins The guide calls it a regex op, but I don't know what that means in this context
2017:08:31 02:31:43 tjscollins So the first pair passed to s/cat has to match the first arg?
2017:08:31 02:31:51 tjscollins And the second pair matches the second arg?
2017:08:31 02:32:51 seancorfield That matches zero or more simple symbols as an argument sequence and labels that sequence as :names
2017:08:31 02:33:32 seancorfield So in (declare foo bar quux), (s/* simple-symbol?) will match the three symbols and :names will be (foo bar quux)
2017:08:31 02:33:59 seancorfield It's quite literally "regex" for sequences (instead of strings).
2017:08:31 02:34:33 tjscollins So something like (s/cat :s string? :i int?) would match both ("1" 1) and (1 "1")?
2017:08:31 02:34:44 seancorfield No, it's sequential.
2017:08:31 02:34:59 seancorfield It would only match "1" 1
2017:08:31 02:35:27 seancorfield How familiar are you with regex for strings?
2017:08:31 02:35:37 tjscollins Alright, would (s/cat :s (s/* string?) :i int?) match both ("1" 1) and ("1" "1" 1)?
2017:08:31 02:35:48 seancorfield Yes, I believe it should
2017:08:31 02:35:56 tjscollins Yeah, I'm familiar with string regexes
2017:08:31 02:36:05 tjscollins Okay, that makes more sense now
2017:08:31 02:37:04 seancorfield You can try it out directly in the REPL, without worrying about arguments: (s/conform (s/cat :s (s/* string?) :i int?) ["1" "1" 1])
2017:08:31 02:37:49 seancorfield and (s/conform (s/cat :s (s/* string?) :i int?) [1]) => {:i 1} because there were no strings at the beginning
2017:08:31 02:39:27 tjscollins Alright that makes a lot more sense now. Is s/alt basically s/or for a sequence?
2017:08:31 02:43:11 seancorfield Yup.
2017:08:31 02:44:26 tjscollins That helps a lot. Thanks.
2017:08:31 02:44:41 seancorfield Also from that guide:
2017:08:31 02:47:31 tjscollins First element has to be a string, second can be boolean or string, right?
2017:08:31 13:48:21 favila I'm trying to spec a sequence where there is a fixed predicate/spec based on position, but the sequence may omit things at the end; and I'd like it to conform to a flat map, one key per position, nil or no-key for the omitted items
2017:08:31 13:51:50 favila I can't use cat plus nested s/? because that backtracks (last example would be accepted)
2017:08:31 13:53:07 favila the best I can think of is a linked-list type structure of nested s/or that a conformer flattens
2017:08:31 13:53:19 favila any other ideas?
2017:08:31 13:58:46 alexmiller s/or of s/tuple ?
2017:08:31 13:59:04 favila s/tuple won't work because variable-width
2017:08:31 13:59:18 alexmiller I’m saying you could s/or multiple s/tuple together
2017:08:31 13:59:27 alexmiller it’s unclear how much variability there is here
2017:08:31 14:00:02 alexmiller I guess maybe s/alt of s/cat would be better for your conformed value
2017:08:31 14:00:49 alexmiller I think I’d say: don’t fight the conformed value. write the right spec, then transform the conformed value to what you want.
2017:08:31 14:01:34 alexmiller always come back to: how can I state the truth about the data?
2017:08:31 14:01:52 favila that's the part that is difficult
2017:08:31 14:02:34 favila I essentially want s/tuple, with keyed names for each slot (not numbers), and possible truncation on the right side for optional values
2017:08:31 14:03:23 favila for background, this is parsing an edi segment, if you are familiar with those
2017:08:31 14:04:10 alexmiller nah
2017:08:31 14:04:39 alexmiller sounds like you really want s/cat with nested s/?
2017:08:31 14:07:15 favila was the best I could come up with
2017:08:31 14:07:29 alexmiller yeah
2017:08:31 14:07:30 favila and I was thinking maybe an s/& could flatten the keys out
2017:08:31 14:07:33 favila ?
2017:08:31 14:08:28 alexmiller I would separate that from the spec, but yes should be able to regularize both the creation of these (with a macro) and the transformation to something you want
2017:08:31 14:32:01 bbrinck @alexmiller When using a predicate as a spec (which can occur in, say, s/assert), the value of :clojure.spec.alpha/spec is a function. However, if a proper spec is defined, the :pred is a symbol. https://gist.github.com/bhb/de31b73133d6322158a3c0a5d47d67a2
2017:08:31 14:32:24 bbrinck Do you happen to know why there is a difference?
2017:08:31 14:32:58 bbrinck (My goal is to provide a consistent printer that can say “<value> did not match <predicate>” in all cases)
2017:08:31 14:33:37 bbrinck I did not see a JIRA ticket, but would be happy to create one if it is helpful 🙂 I’m not sure if there is a technical limitation here.
2017:08:31 14:35:52 bbrinck I suspect this is related to the fact that s/def is macro whereas explain-data is a function, so it only gets the function value
2017:08:31 14:39:12 bbrinck i.e. the symbol is already resolved by the time explain and explain-data see it. Hm. I wonder if the right solution is to spend time making a pretty-printer for functions that works reliably across CLJ and CLJS
2017:08:31 14:44:14 alexmiller there is a ticket and a patch for that
2017:08:31 14:44:25 alexmiller but your reasoning is correct
2017:08:31 14:48:21 bbrinck My ticket searching skills need work 😉
2017:08:31 14:48:53 alexmiller but maybe it’s not covering what you’re asking
2017:08:31 14:49:31 alexmiller you should not have any expectation about the concrete type of :clojure.spec.alpha/spec, other than that it can be used as a spec
2017:08:31 14:50:42 bbrinck No, you’re correct, this is what I want: I am not concerned about the concrete type, I just want a way to consistently pretty print the problem.
2017:08:31 14:51:03 alexmiller that ticket fixes the :pred in the explain-data
2017:08:31 14:51:10 alexmiller by unmunging the function name
2017:08:31 14:51:19 bbrinck Although in my case, I’d probably print the set differently (instead of a set literal, I’d enumerate options), but that should not be a problem
2017:08:31 14:51:27 alexmiller I’m hoping it will be in the next spec release
2017:08:31 14:52:01 bbrinck Does it apply the same unmunge to :clojure.spec.alpha/spec at the top level of the explain-data?
2017:08:31 14:52:33 bbrinck IOW, would it work for (s/explain odd? 10)?
2017:08:31 14:53:12 alexmiller (running with the patch)
2017:08:31 14:53:51 bbrinck Much appreciated! I will definitely be using this in expound
2017:08:31 14:57:34 alexmiller I have been too busy to look at expound, but hope to soon
2017:08:31 15:01:13 bbrinck Don’t look too closely: the internals are a bit of a mess, but I hope to refactor soonish 🙂 . Most importantly, I want to cleanly separate the pretty printing part (hopefully useful to end users, but not really reusable) from the stuff I did related to adding information to the “in” paths, which could be useful for any library that wants to consume explain-data
2017:08:31 15:06:12 alexmiller seems wise
2017:08:31 16:14:02 didibus Can I trust describe to be consistent on specs of the same kind. Like will all s/keys describe itself in a consistent way?
2017:08:31 16:15:39 didibus I'm going to build reflective logic on spec descriptions, but I want to be sure its reliable.
2017:08:31 16:41:56 hmaurer Hi! Is there a built-in function to conform a spec and throw an error if the spec does not conform?
2017:08:31 16:42:29 hmaurer My use-case is basically this: I want to pipe data through a series of steps, and I would like to throw an error if a spec does not conform
2017:08:31 16:42:35 hmaurer I could easily write a function to do this
2017:08:31 16:42:38 bbrinck @hmaurer Would s/assert work?
2017:08:31 16:42:41 hmaurer just wondering if it’s in clojure.spec
2017:08:31 16:43:04 bbrinck followed by conform?
2017:08:31 16:43:36 hmaurer actually s/assert does exactly what I want; I don’t care too much about conforming
2017:08:31 16:43:39 hmaurer thank you!
2017:08:31 16:45:02 bbrinck @hmaurer Just make sure to call (s/check-asserts true) to turn on assert checking for spec
2017:08:31 16:46:07 hmaurer @bbrinck ok. Is it sensible to use s/assert when validating user input, e.g. an API payload?
2017:08:31 16:50:51 bbrinck @hmaurer I would prefer to just use explain-str in that case personally, since a) assert will raise an exception that you’ll need to catch and b) turning on asserts will enable asserts throughout your program, which you probably don’t want. I’d only use assert for dev-time checks/invariants
2017:08:31 16:51:57 bbrinck I’m assuming the flow is check data against spec if it works, proceed else, show message to user?
2017:08:31 16:52:05 hmaurer pretty much
2017:08:31 16:52:37 hmaurer I would like to have some flow like this:
check data again spec => validate through authorization filters => assign UUID => transact to database
2017:08:31 16:52:39 bbrinck If your user is a dev at dev-time, I think assert is fine. But if you want to show end-user error messages, I’d avoid it
2017:08:31 16:52:51 hmaurer any step can throw an error which should be reported back to the user
2017:08:31 16:52:57 hmaurer end-user, not dev
2017:08:31 16:53:29 bbrinck Right, in that case, I’d use explain-str and check to see if it success.
2017:08:31 16:53:48 bbrinck Then return that string to whatever is going to show it to a user
2017:08:31 16:54:50 hmaurer I am quite new to Clojure; would you in general avoid to throw errors and return error conditions instead? Aka use a “Either”-like type, similar to Haskell/etc?
2017:08:31 16:55:10 hmaurer e.g. return {:ok ...} or {:fail ...}
2017:08:31 16:56:31 bbrinck Good question. I don’t see a lot of use of “either” in Clojure code. Exceptions are fine if it’s actually an exceptional case , but AIUI, in your case, you want to validate user input so you expect things to not conform
2017:08:31 16:56:35 bbrinck (correct me if I’m wrong)
2017:08:31 16:57:16 hmaurer You are right. I would like to be able to chain my pipeline steps with (-> ...), but breaking out of the pipeline on error seems hard to do without exceptions
2017:08:31 16:57:17 bbrinck I don’t have great advice here for general error patterns, but I’d lean towards “use exceptions for exceptional cases, not control flow”
2017:08:31 16:57:26 bbrinck Right
2017:08:31 16:57:43 bbrinck There are libraries for this, but I don’t know how idiomatic they are. I haven’t used them myself
2017:08:31 16:59:19 didibus It's more idiomatic to use exceptions with ex-info. I would wrap explain-str or explain-data in a function which throws an ex-info, where I handle it at the place where my code can return an error msg to the user. There I'd parse out what I need from the exception into the msg I want to show the user and return it.
2017:08:31 16:59:30 hmaurer Thanks! I’ll read this. I like the philosophy of using errors for exceptional cases only though; considered following that approach in a previous javascript projct
2017:08:31 17:00:02 hmaurer @didibus yep, that sounds a bit more idiomatic/practical
2017:08:31 17:01:43 didibus What you do is in the ex-info map, you put a key such as :type :validation-error and so where you catch it, you know this isn't a bug, but a validation error. Normally, the exceptions should not log error or push failure metric, as they are actually expected behaviour
2017:08:31 17:01:54 bbrinck Even if you use exceptions in this case (not necessarily bad IMO), I’d still probably throw an exception manually instead of relying on assert since turning on/off asserts is a global thing. IOW, do you want to enable all asserts in production?
2017:08:31 18:22:03 hmaurer That seems reasonable
2017:08:31 17:02:15 bbrinck That’s a valid question, but you probably don’t want it tied to your user-facing errors messages
2017:08:31 17:02:18 didibus I mean exceptions of type validation-error
2017:08:31 17:02:49 didibus Agree with bbrinck
2017:08:31 17:02:51 bbrinck Good suggestion @didibus
2017:08:31 17:03:54 bbrinck @hmaurer Shameless plug, but if you are considering showing spec error messages to users, take a look at https://github.com/bhb/expound
2017:08:31 18:24:33 hmaurer I stumbled on it yesterday actually! It’s looking very useful
2017:08:31 17:06:23 didibus Big fan of expound, the default spec errors are so full of noise.
2017:08:31 17:08:16 didibus I'm actually disappointed in spec for improving clojure error msgs. They promote the idea, but it really doesn't help because the error is so long. Its only slightly friendlier then a stack trace
2017:08:31 17:09:35 bbrinck IMO, the default error messages are a good default, since they need to be useful “out of the box” in a variety of contexts: shown to a end user, shown to a dev, logged, returned as an API response, etc. Expound makes a different set of tradeoffs that are very much optimized for the “display to dev” case - *much* more verbose, and omits some information, but hopefully helps devs find the root problem faster
2017:08:31 17:10:22 bbrinck Luckily we can swap out the default for custom printers like expound when we want to customize behavior 😉
2017:08:31 17:11:04 bbrinck Expound is probably still not quite optimized for end users, but I am looking to make it more suitable for this in the future.
2017:08:31 17:11:45 didibus I don't know. Id like more beginner friendly defaults in Clojure. Easier to swap out to something else when you're not a beginner anymore. Its a lot to ask to a beginner to learn how to setup expound to figure out why his code doesn't work.
2017:08:31 17:13:37 bbrinck @didibus I agree with you that it’d be nice to get something easier for beginners. I have been wondering if it would be possible to create a lein plugin or library that would set up beginner-friendly REPL / logs by default
2017:08:31 17:14:47 bbrinck Adding a single dependency that loads core specs, turns on instrumentation (with orchestra?) and configures spec to use expound without any further config could be really useful
2017:08:31 17:16:18 bbrinck maybe pretties up/shortens stack traces as well
2017:08:31 17:17:37 didibus That would be really nice. Maybe if it can also add a coloured repl and preload clojure.repl
2017:08:31 17:19:56 didibus But since at my work we use Ant, it doesn't help me promote Clojure to other teams
2017:08:31 17:22:03 bbrinck What editor would you recommend for other teams? Just curious about the constraints.
2017:08:31 17:24:29 didibus Well, I tend to recommend eclipse + counterclockwise or cursive with intelliJ.
2017:08:31 17:25:13 didibus The latter less so, because of its paid nature
2017:08:31 17:27:10 didibus You'd just be surprised how many top class engineers don't bother to learn how to setup any environment properly.
2017:08:31 17:28:24 didibus So when I tell them, if you want to do Clojure and like it, you need to learn at least how to setup a working repl. And already I've lost the interest of half of them
2017:08:31 18:12:56 gdeer81 @didibus I think @sekao is the best person to talk to about this since he has written three IDEs so far and each one tries to lower the burden of getting everything set up
2017:08:31 18:16:56 gdeer81 this is a side topic not really related to spec so I broke it out into this thread
2017:08:31 18:15:04 gdeer81 my favorite is nightmod since I can send someone the download link and they can be in a new project with a repl, starter code, and a live game environment in less than 5 minutes
2017:08:31 18:38:33 favila In clojure.spec.alpha, s/invalid? is defined like (identical? ::invalid ret). I thought we couldn't rely on keyword interning anymore?
2017:08:31 18:40:07 ghadi it still is a thing
2017:08:31 18:40:12 ghadi (not in cljs afaik)
2017:08:31 18:40:47 favila I thought at some point even in clj it was no longer guaranteed
2017:08:31 18:41:05 favila so (identical? (sym|kw) x) became a bad idiom
2017:08:31 18:41:12 ghadi symbols are not interned
2017:08:31 18:41:14 ghadi just keywords
2017:08:31 18:41:27 favila ah maybe that's what I'm thinking of
2017:08:31 18:41:32 favila or I'm just confused with cljs
2017:08:31 18:41:37 favila (I do a lot of cljs)
2017:08:31 18:41:39 ghadi though the codepath goes still through a method called intern
2017:08:31 19:05:03 ikitommi hi. I have a problem creating a recursive spec. the s/spec requires the spec to exist:
2017:08:31 19:06:10 ikitommi Setting something temporary as a spec and double-evaluating the original spec seems to work thou, but don’t think this is the way to do this 😉
2017:08:31 19:06:30 ikitommi
;; declare'ish
(s/def ::route int?)
(s/valid? ::route data)
; false
;; resetting the spec works now (but the childs is wrong)
(s/def ::route
(s/cat
:path string?
:arg (s/? (s/map-of keyword? any?))
:childs (s/* (s/spec ::route))))
(s/valid? ::route data)
; false
;; now it might work?
(s/def ::route
(s/cat
:path string?
:arg (s/? (s/map-of keyword? any?))
:childs (s/* (s/spec ::route))))
;; ....
(s/valid? ::route data)
; true
2017:08:31 21:33:43 hmaurer Hi! Quick question: I have a spec for a map which has required and optional keys (`(s/keys :req [..] :opt [..])`). One of my functions takes “partial” versions of those maps, which essentially means that every key is optional (and every key that was optional in the full spec can be nil, but I can handle that separatly)
2017:08:31 21:34:24 hmaurer it there a neat way to do this, how should I define two versions of the map specs? (e.g. :foo/my-map and :foo.partial/my-map)
2017:08:31 21:42:48 plins hello everyone, im reading the spec docs (`https://clojure.org/guides/spec`)
is it possible to achieve with spec/fdef something similar to this? (catching spec errors at run time):
2017:08:31 22:06:26 wilkerlucio I remember reading that the definitions should be lazy to allow this kind of things, am I missing something here?
2017:09:01 04:12:29 didibus I always known it as not being lazy. Can't you just reorder your definitions? Being lazy would make it risky I feel, what if specs are accross namespaced and lazy, and you forgot to load another namespace.
2017:08:31 22:17:14 wilkerlucio @hmaurer you can define multiple versions, create a simpler one and then you can merge it with other to create a more fully-featured, example:
2017:09:01 08:43:49 lmergen i have a question about project organisation and specs — specifically, i’ve settled on the use of a separate specs.clj file to put all my specs in, that works quite well.
but, i have multiple small subprojects, of which some share the same specs (i.e. the output of one project is the input of another).
what’s the best strategy here ? just copy/paste these common specs into these projects (because they’re different domains), or share the code using a common library ?
2017:09:01 08:52:37 mpenet separate repo for shared specs works well
2017:09:01 08:52:57 mpenet depends what it is
2017:09:01 09:00:06 ikitommi could there be something like s/lazy which would work like s/spec but would not fail if the given qualified keyword doesn’t link to registered spec? could be used with the recursive specs (my example above). (s/lazy ::route) (vs (s/spec (s/and ::route)). The error messages be easier to understand. Happy to do a PR if.
2017:09:01 13:59:41 wilkerlucio I have the same problem when trying to spec some distant relations where I can't guarantee the load order
2017:09:04 13:02:01 wilkerlucio @ikitommi just one thing, you can do (s/and ::route) instead of (s/spec (s/and ::route)) and it will work the same
2017:09:01 09:21:01 lmergen @mpenet in this specific instance it’s the spec of what a job input should look like — it’s produced by one component, and consumed in another.
2017:09:01 09:21:37 lmergen so it strikes me that a common spec library might be a good approach for this, but it also feels like it might create more spaghetti / overhead
2017:09:01 13:04:32 aisamu Hi!
Is there a non-macro alternative to spec’s sequence operators? (`cat`, *, ? …).
To avoid the XY problem: I have to generate medium-sized specs at “runtime”, and my current solution is giving me ERROR java.lang.RuntimeException: Method code too large!. I’ve read that it happens because there is an eval generating classes too big for the JVM. The eval exists because I couldn’t use the spec macros at runtime without wrapping them with make-fn.
Is it unfeasible or just not currently implemented?
Or am I approaching this backwards?
2017:09:01 14:13:47 ghadi @aisamu what specifically are you evalling that is causing the error?
2017:09:01 14:17:39 aisamu The sequence macros, as in (make-fn s/?), which later gets called with apply and a vector of specs
2017:09:01 14:28:25 dealy Hi, I've been trying to write a spec for a function which takes another function as a parameter. However it always fails with an Unable to construct gen error, is there a way to spec a function like this?
2017:09:01 14:33:24 ghadi @aisamu Are you doing something like calling eval on a long collection [(s/?....)......] ? How many before it blows with an error?
2017:09:01 14:57:34 aisamu @ghadi No. It's a small collection (~4), but the specs inside can get rather large. (The last input collection I managed to trace before the blow-up had 50k chars)
2017:09:01 15:05:34 ghadi yeah... spec macros can generate a lot of code
2017:09:01 15:05:55 ghadi the clojure compiler doesn't do method splitting
2017:09:01 15:06:47 ikitommi @aisamu had the same problem. See the source code, most specs have non-documented functional versions beneath. Hopefully will be part of the final public api.
2017:09:01 15:11:45 aisamu @ikitommi I looked into that, but it quickly became clear I'd have to bring along a dozen private functions to get the top level to work =/
2017:09:01 15:23:24 alexmiller the downside of the functional versions is that you lose forms for reporting
2017:09:01 15:23:39 alexmiller as those are captured by the macro
2017:09:01 15:24:16 alexmiller I have talked here in the past about how specs for spec forms (CLJ-2112) can be used to programmatically construct specs via unform
2017:09:01 15:24:29 alexmiller still a work in progress
2017:09:01 15:47:34 ikitommi I don't mind that the forms need to be manually passed to the functions. I'm using a pred->form multimethod to auto-resolve the forms, with most/all core preds mapped.
2017:09:01 15:50:33 ikitommi And looking forward on progress on the CLJ-2112.
2017:09:01 16:33:14 hlship I think clojure.spec could do a bit better job of reporting failures to dynamically load clojure.test.check.generators. My production code doesn't include the org.clojure/test.check dependency, but does instrument a particular function at runtime; that seems to be forcing the code down a path that involves org.clojure/test.check ... possibly because some of the values being specs are functions? Very hard to say.
2017:09:01 16:33:43 hlship Possibly this can be circumvented by providing an explicit generator for those function specs within our giant spec.
2017:09:01 18:04:18 mpenet functions as argument of instrumented fn get tested via gen yep
2017:09:01 18:05:37 mpenet you can overwrite :gen on them with ifn? but it kinda defeats the purpose after all, personally I would prefer if we could make it so that the argument (which is a fn) has its args checked at "invoke" time
2017:09:01 18:05:53 mpenet there's also a ticket on hold somewhere about this if I recall
2017:09:04 12:33:22 alexmiller and you can’t reproduce without direct linking?
2017:09:04 12:33:30 mpenet the bug goes away without dl yes
2017:09:04 12:33:56 alexmiller well would be good to see a jira on that if you can build a reproducible case
2017:09:04 12:34:06 mpenet the code is quite involved I ll try to create a minimal repro
2017:09:04 12:34:24 alexmiller I don’t think multimethod calls are direct linked
2017:09:04 12:34:56 alexmiller the line you have above is part of regex specs, not multispec
2017:09:04 12:35:32 mpenet but it mentions a fn that's used as first arg to a multi-spec
2017:09:04 12:36:07 mpenet (it's a multimethod to be more precise)
2017:09:04 12:36:14 alexmiller sorry, just guessing from too little info
2017:09:04 12:36:17 mpenet np
2017:09:04 19:54:50 gklijs managed to get spec working on clojurescript also after some searching/trying, but did not find any direct help on the internet. I’n now using a cljc file starting with `(ns m-venue.spec
(:require [#?(:clj clojure.spec.alpha
:cljs cljs.spec.alpha
:default clojure.spec.alpha) :as s]))` and on clojure I only need to source path, for clojurescript i need to put it in the source paths and require it. Now I can use the same spec front and back-end 🙂.
2017:09:04 22:45:52 alexmiller cljs automatically rewrites the clojure namespace as the cljs one, so you shouldn’t actually need any of this afaik - just require the clojure ns
2017:09:04 20:42:40 plins hi everyone, im trying to use :pre and :post to produce a more detailed error in run time
2017:09:04 20:44:25 plins why this works for :pre but not for :post?
2017:09:04 22:47:23 alexmiller shouldn’t both of those pre and post be functions? I think they’re both getting evaluated at compilation time, not runtime
2017:09:05 09:32:19 Avichal Hello,
I am trying to spec some higher order functions using fdef and fspec For eg:
2017:09:05 14:39:08 alexmiller or wait, maybe? what are you trying to do?
2017:09:05 14:39:21 alexmiller you might want s/keys*
2017:09:05 14:39:40 alexmiller if you’re matching varargs in a function
2017:09:05 14:39:54 fothaggenda Hmmm, I'll check that out 😃
2017:09:05 14:40:19 alexmiller and even my prior answer is wrong - you can use s/keys there (I just don’t think it’s what you want)
2017:09:05 14:40:45 alexmiller are you taking a map or a series of key/value options in something?
2017:09:05 14:40:55 fothaggenda Odd; I tried that way before and my generative test exploded
2017:09:05 14:40:58 fothaggenda I'm taking a map
2017:09:05 14:41:08 alexmiller oh, then you should be fine with the above
2017:09:05 14:41:57 alexmiller if you want to test it, run (s/exercise (s/cat :first-argument (s/keys :req-un [::a ::b ::c])) 1000) - that’s basically what you’re getting with a generative test on something
2017:09:05 14:42:44 fothaggenda Nice catch, I'll inspect this way
2017:09:05 14:43:01 alexmiller often pushing that out to 1000 samples reveals more :)
2017:09:05 14:43:33 alexmiller if a/b/c are big colls, that can provoke some problematic samples - bounding their size often helps
2017:09:05 14:43:41 alexmiller with :gen-max
2017:09:05 14:44:21 fothaggenda Actually that's not the case, in my scenario ::a, ::b and ::c are just very specific definitions, but I got your point
2017:09:05 18:47:41 arohner are there any projects for associating docstrings with spec definitions?
2017:09:05 18:52:54 alexmiller we will eventually get around to implementing it in spec, just kinda far down the list atm
2017:09:06 08:41:30 gklijs @arohner I have a private project which I use to serialize specced data, where I also store the used spec in either the bytes or the string. I might make it public. It also makes it possible to set defaults, in order to get correctly specced data back, in case you added required keys.
2017:09:06 08:47:32 gklijs It works both with clojure and clojurescript, but it really needs some more work (liks tests for the clojurescript) before I put it public
2017:09:06 11:46:04 ikitommi @gklijs can you re-create the spec instances from serialized format in cljs without (form) eval?
2017:09:06 14:06:03 gklijs @ikitommi Doent’t know what you mean exactly, i use the form function to re-create the instance, but by assuming the spec is a map with either keys, or merge, and by storing the data in a certain way, I only store the values. If there is an additional required key which wasn’t there when the instance was serialized, either nil is filled in, or the default is it is set.
2017:09:07 12:14:36 seantempesta Is there a version of s/or that just conforms the value and doesn’t tell you what value it is? I have a map where certain entries can be either a string or a function that returns a string and I just want to get back either a conformed map or :cljs.spec.alpha/invalid. Am I supposed to traverse the returned map and remove the named vectors?
2017:09:07 13:55:53 seantempesta @misha oh cool. so just call s/nonconforming first?
2017:09:07 13:56:44 misha wrap spec you don't want vectors from in nonconforming
2017:09:07 13:58:28 seantempesta Oh wait, s/nonconforming says that it returns the original value. I want the conformed value though.
2017:09:07 13:59:14 misha what do you think original and conformed values are?
2017:09:07 13:59:33 seantempesta original is the original value, but a conformer function can return a modified value.
2017:09:07 14:00:19 misha if you have custom conformer, I think you can "avoid vector" there
2017:09:07 14:02:23 seantempesta Well I was using all custom conformers. Writing separate functions like str-or-fn-ret-str, but I realized I was duplicating a lot of code. Also, I think that requires me to write custom generators for everything.
2017:09:07 14:03:06 misha can you share str-or-fn-ret-str?
2017:09:07 14:03:30 seantempesta It looks like (s/unform ::key (s/conform ::key val)) is working
2017:09:07 14:04:25 seantempesta Well, it’s actually string-or-react-class. Here’s what it looks like:
(defn string-or-react-class?
"Accepts either a string, react class, or a fn that returns a react class. If it's a fn, props will automatically
convert (js->clj) when the fn is called."
[s-or-c]
(cond
(helpers/is-react-class? s-or-c) s-or-c
(string? s-or-c) s-or-c
(fn? s-or-c) (fn [props & children]
(let [clj-props (js->clj props :keywordize-keys true)
react-c (s-or-c clj-props children)]
react-c))
:else :cljs.spec.alpha/invalid))
2017:09:07 14:04:29 misha if you supply custom unformer - it might work.
But I have a feeling you are doing something redundant
2017:09:07 14:05:23 misha why do you have it as a conformer, rather than spec composed from predicates?
2017:09:07 14:05:47 seantempesta That’s what I was asking. If I do it as s/or predicates I get back vectors
2017:09:08 05:42:30 didibus From my perspective, you conform to get back a vector, or a custom projection of the specced data.
2017:09:08 05:43:02 didibus If you want to know if something validates, you do (when (s/valid? ...) do), or one of the explain fns.
2017:09:07 14:08:35 misha try to wrap s/or in nonconforming. this way you can have more granular conformer for just (fn? s-or-c) case.
2017:09:07 14:08:58 seantempesta Okay. I’ll give that a shot. Thanks.
2017:09:07 14:14:53 misha although I'd like to see examples of coercion in spec done right, myself
2017:09:07 14:17:46 misha @seantempesta it seems like nonconforming or prevents any conformation inside its branches
2017:09:07 14:18:01 misha duh... opieop
2017:09:07 14:26:25 bbrinck @seantempesta Can you talk more about why a call to conform is useful in this case? If you just want the original value, could you just do something like (if (s/valid? ::string-or-int x) x :cljs.spec.alpha/invalid)
2017:09:07 14:27:03 misha I think to wrap fn? in (fn [props & children]
2017:09:07 20:57:11 cgrand How do you prevent a :ret spec to be described by doc? I have a big spec and I’d like to be displayed by name rather than dumping it on the user. Currently I wrap it into spec/and. Is there a more elegant solution?
2017:09:07 21:04:54 alexmiller name it?
2017:09:07 21:05:08 alexmiller then you’ll just see the name in the doc
2017:09:07 21:14:51 cgrand I’ll stick with spec/and then. (btw I also abuse spec/and for late-bound aliases)
2017:09:07 21:25:14 alexmiller I guess I don’t understand why that’s better
2017:09:07 22:07:05 cgrand If I put it in only in the doc then it's not in the spec anymore. If I put it in doc and in spec it doesn't prevent doc from dumping the whole description. So s/and is the hack to have :ret spec'ed without doc being verbose
2017:09:07 22:50:53 alexmiller sorry. I went and tried it and I understand what you’re saying now. So the current behavior is not the intention. there is a ticket for the early aliased spec resolution, don’t know the #.
2017:09:07 21:28:10 ddellacosta what is the best approach for validating a tuple that has optional fields? E.g. [:required :required :optional]?
2017:09:07 21:30:08 misha @alexmiller can you comment on @seantempesta's question, please?
what is a better way to do coercion with spec?
what is expected way to work with conformed compound values (map conformed to some the nested spec)? e.g. when you get map values as an or-conform-vector [:branch conformed-value]. Is "conform-client" code supposed to know the spec structure, and be able to walk the conformed data structure?
2017:09:07 21:35:56 alexmiller don’t do coercion with spec
2017:09:07 21:36:27 alexmiller use conform to tell you how a value is parsed. then use clojure to transform that to what you want.
2017:09:07 21:38:59 alexmiller for the specific case of wanting to avoid conforming an s/or (one of the most common), spec as (s/def :foo (s/nonconforming (s/or :a int? :b string?)))
2017:09:07 21:39:39 alexmiller that is, use s/nonconforming at the leaves, not at the top
2017:09:07 21:40:17 alexmiller for this particular case, I’ve come to believe that nonconforming or is a reasonable use case and either we should keep s/nonconforming or add that as a variant/option on or
2017:09:07 21:41:11 misha so, going back to Sean's example, conform map to spec (will return [:fn? s-or-c] for some key), and then post-process conformed map with a coerce-function, which will replace [:fn? s-or-c] with (fn [props & children] ...?
2017:09:07 21:42:05 alexmiller without having completely read or grokked all of the backchat, sure
2017:09:07 21:42:19 alexmiller Clojure is like, pretty good at transforming data. use it.
2017:09:07 21:48:08 misha I think the issue/question is: "should one wrap coerce-fn in s/conformer and conform to that, or should one just apply coerce-fn after conforming data to simpler spec?"
2017:09:07 22:01:06 alexmiller the latter
2017:09:07 22:01:44 alexmiller if you do the former, you are registering specs that throw away information about the original value
2017:09:08 12:32:25 ikitommi we are running our spec libs tests also with :advanced cljs optimizations. Seems that the cljs.spec.test.alpha/instrument doesn’t work under it. Is there a way to enable it?
2017:09:08 22:05:05 jjttjj let's say the spec decimal-in defined here: https://github.com/SparkFund/useful-specs/blob/master/src/specs/number.clj#L24 is exactly what i need, except that the generator it comes with results in a bigdec, where I need a double. Is there a way to make a new generator that just coerces the output of an existing generator?
2017:09:08 22:06:23 jjttjj basically i just need a spec and a generator for a double with a specified precision and scale
2017:09:08 22:16:22 alexmiller Make a spec with s/decimal-in
2017:09:08 22:16:50 alexmiller Then make a spec that wraps it with s/with-gen
2017:09:08 22:17:37 alexmiller If you want build on the existing gen, use gen/fmap over the s/gen of the first spec
2017:09:09 23:58:00 samedhi I have the following
;; This ::post does not work, recursion limit
;; (spec/def ::post (spec/keys :req [::post-content ::user-id ::posts]))
;; This ::post does work, never seems to recurse though...
(spec/def ::post (spec/keys :req [::post-content ::user-id]
:opt [::posts]))
(spec/def ::posts
(spec/with-gen
(spec/map-of ::post-id ::post)
#(spec/gen (spec/map-of ::post-id ::post :min-count 0 :max-count 1))))
which defines a spec recursively; ::post refers to ::posts .However, if I remove :opt [::posts] and add ::posts to the :req field’s ::post, I get a Maximum call stack size exceeded. Is this the expected behavior?
2017:09:10 00:04:58 samedhi Seems like the two should be equivalent, since ::posts would (probabilistically) stop its recursion when it generates the empty map (`{}`).
2017:09:10 00:09:15 samedhi Pleased to see that the generated data is in fact recursive! I guess I will just let it be, just to close out, my question mostly has to do with why ::posts must be put in the :opt and cannot be included in the :req.
2017:09:10 00:11:09 samedhi Also, it is rather interesting that ::posts can be referenced before being used, which I guess implies that spec/keys does not “dereference” ::posts at its declaration, but waits to do so until the ::post spec is used.
2017:09:10 00:11:15 samedhi Curiouser and curiouser
2017:09:11 18:48:26 bfabry @samedhi well... they're not equivalent, because a optional key is different to a required key which is a (possibly empty) map. it seems like the generator produced by map-of just doesn't produce empty maps often enough to avoid blowing the stack
2017:09:11 18:51:22 bfabry you need to be able to reference something before it's defined in order to support recursive specs, which is an intended feature
2017:09:11 18:52:32 samedhi Hmm, interesting. It appears that I am referring to ::posts from within ::post before ::posts actually exist…
2017:09:11 18:53:11 bfabry yes, that's intended to be supported
2017:09:11 18:59:20 bfabry I don't know enough about how generators work, I'm guessing it's more sophisticated than just random so that's why it's managing to find the pathological case of a very deeply nested tree every time
2017:09:11 20:57:01 tbaldridge Yes, a lot of the complexity in generators comes from the feature of shrinking. When you get a test failure with a generator, test.check will try to shrink the possible data set to give you the smallest failing test. So instead of giving you "Failure when I tried: [123123121 55 3 9 5 23 4]" It may say: "Failure when I tried: [0 2]". And that tells you a lot about went wrong:
2017:09:11 20:57:44 tbaldridge for example you know that the vector has to be at least two items long (it didn't fail when there was only one value in the vector). And that the second number has to be larger than one: (since 0 and 1 didn't cause it to fail).
2017:09:11 20:58:14 tbaldridge That sort of information is very helpful in debugging, but requires a lot of backtracking in the generator, and that's what makes them so hard to write/understand.
2017:09:11 23:15:29 bfabry I would love to know the "right" way to make the generators work for the recursive structure above
2017:09:12 00:04:23 gfredericks you could try writing your own with gen/recursive-gen
2017:09:12 00:05:30 gfredericks or...is this a DAG?
2017:09:12 00:06:51 gfredericks or I guess it's a proper tree?
2017:09:12 00:07:06 bfabry I did try with recursive-gen and got the same error. seems like it should be a proper tree yeah, which I guess is a type of DAG
2017:09:12 00:07:09 gfredericks yeah I think recursive-gen would work
2017:09:12 00:07:24 gfredericks can you share the recursive-gen code?
2017:09:12 00:17:57 bfabry but a post without the ::posts key is not valid, the valid value for the leaf would be {}
2017:09:12 00:20:50 gfredericks Yeah I meant a version where the key always has {}
2017:09:12 05:59:19 levitanong @gfredericks out of curiosity, could you try:
(s/with-gen ::post #(s/gen (s/or :post ::post :empty #{{}})))
2017:09:12 10:50:57 gfredericks no, because it will still try to generate the default generater for a ::post, which can overflow the stack or whatever it was
2017:09:12 07:20:10 lmergen what would be the best way to spec a collection of tuples, e.g.
[[:k1 :v1] [:k2 :v2]]
i'm thinking of just doing an into {} ... before speccing and speccing it as if it were a map, but is there perhaps a more elegant way to do this ?
2017:09:12 07:20:53 lmergen the alternative i came up with was (s/* (s/or (s/cat ...) (s/cat ....))), but that's also ugly
2017:09:12 07:24:00 lmergen (as a bit of background, the reason i'm not using a map is that the ordering of the elements is important)
2017:09:12 07:30:13 tap Maybe this?
(s/coll-of (s/tuple keyword? keyword?))
2017:09:12 07:31:24 lmergen yeah that's similar to the cat use case... the problem is that for a specific key, i need a specific value
so then you would end up with
2017:09:12 17:04:11 seancorfield @jiyinyiyong According to the :args and :in [1], it's not expecting that :require-macros form -- so I'm thinking you tried to run this in Clojure, rather than ClojureScript? (I think :require-macros is ClojureScript-only?)
2017:09:12 23:52:18 Jon it turned out shadow-cljs compiler did not handle it correct 🙂
2017:09:12 17:04:37 seancorfield (but, yes, some of ns spec output is a bit overwhelming)
2017:09:12 17:05:33 seancorfield Extra input generally means a sequence (`s/cat`) ran into a form that was not recognized by the syntax specified in the spec.
2017:09:12 17:07:22 seancorfield :value (respo.app.comp.todolist (:require-macros [... is the value that failed and :in [1] gives you the position within that sequence.
2017:09:12 18:57:05 kevin.lynagh Is validating data with runtime context in the purview of spec? For example, instead of just saying that :foo/id is an integer, can I specify that "it's an integer that corresponds to an entity in the provided database?"
2017:09:12 18:57:38 ghadi not recommended
2017:09:12 18:57:55 kevin.lynagh From what I can tell, even though specs are being checked at runtime, there's no idiomatic way (short of dynamic var based hacks) to pass additional context into the validation logic.
2017:09:12 18:58:43 kevin.lynagh @ghadi Is there any kind of rationale or discussion on this topic somewhere that you know of? I haven't been able to find anything in my research more specific than "don't use spec as a runtime transformation system"
2017:09:12 18:58:51 kevin.lynagh which is sort-of-related, but not quite.
2017:09:12 18:59:14 ghadi if you look up in a DB your predicates / specs might change their contract
2017:09:12 19:00:02 ghadi the meaning of a spec might change based on state, which loses a whole bunch of value (inverting a spec into a generator)
2017:09:12 19:00:44 ghadi I think specs should not depend on any ambient state
2017:09:12 19:01:08 kevin.lynagh besides the implicit spec database
2017:09:12 19:01:16 ghadi e.g. just check that the email address is valid looking, don't try to open up an SMTP connection
2017:09:12 19:02:20 kevin.lynagh Yeah, but that's something that can't be technically enforced by the API of spec
2017:09:12 19:02:27 kevin.lynagh given that it accepts arbitrary predicate fns
2017:09:12 19:02:56 ghadi i get that, it's just a terrible idea to curry in state into your predicate
2017:09:12 19:03:07 ghadi nothing I can do to prevent it ¯\(ツ)/¯
2017:09:12 19:03:12 kevin.lynagh = )
2017:09:12 19:03:28 kevin.lynagh Right, I don't mean to challenge the underlying design decision
2017:09:12 19:03:46 kevin.lynagh as a consumer, it just wasn't clear to me whether what I wanted to do was in the design space at all
2017:09:12 19:04:09 kevin.lynagh I thought it might be, since conceptually it's just further validation
2017:09:12 19:04:21 ghadi i think you'd use spec to check data at the door, then do extrinsic checks afterwards
2017:09:12 19:04:58 ghadi i don't think it's a design goal to be a total system
2017:09:12 19:04:59 kevin.lynagh At that point, it's easier to do the entire validation / generation myself, rather than using spec.
2017:09:12 19:05:10 kevin.lynagh since in my domain pretty much all of the incoming data is just ids
2017:09:12 19:05:16 ghadi there's so many more benefits you'd be dropping
2017:09:12 19:05:21 kevin.lynagh so I need to realize those IDs into entities, then check properties of those entities.
2017:09:12 19:05:53 kevin.lynagh ditto on the generation front, just getting integers won't help. I need to be able to generate integers that correspond to entities of type X
2017:09:12 19:05:54 kevin.lynagh or whatever.
2017:09:12 19:06:33 ghadi don't you need to check the entities?
2017:09:12 19:06:33 kevin.lynagh unless I'm missing something where spec might still be able to help in here?
2017:09:12 19:07:09 ghadi you're right the IDs are trivial, but I doubt that's the extent of the applicability of spec for your use case
2017:09:12 19:07:46 ghadi I'd certainly want something to describe one of the "entities" and get a free generator
2017:09:12 19:08:03 ghadi A non-trivial generator
2017:09:12 19:08:29 kevin.lynagh I can walk you through a concrete example if you have 5 minutes
2017:09:12 19:08:54 ghadi go for it; I might not be able to help, but there are many 👀 in here
2017:09:12 19:09:09 kevin.lynagh cool, thx = )
2017:09:12 19:09:28 kevin.lynagh I'm using datascript as the db, so it's all in-memory.
2017:09:12 19:09:45 kevin.lynagh the application architecture is event sourced
2017:09:12 19:10:09 kevin.lynagh basically, I have (step db event) ;;=> new-db
2017:09:12 19:10:45 kevin.lynagh basically, create a new database from the old database, given an event
2017:09:12 19:11:51 kevin.lynagh The events implement a few protocol methods like (valid? [this, db]) which says whether or not the event is valid for a given database.
2017:09:12 19:13:06 kevin.lynagh So, for example, #events.AddChild{:parent-id 5} is a record that represents the event "add a child to node 5". The valid? function needs to look and see if node 5 exists in the db and, if it does, make sure it's the type of node that is allowed to have children
2017:09:12 19:13:19 kevin.lynagh All this make sense so far?
2017:09:12 19:18:14 kevin.lynagh Anyway, I don't understand how I can implement valid?w/ spec, since everything depends on having the runtime db value. Likewise, I cannot generate these events with spec, because it'd need to only generate integer IDs that correspond to nodes that can have children.
2017:09:12 19:19:24 kevin.lynagh If wrote some specs to define what it means for a node to be allowed to have children, then I could turn the ID into a datascript entity at runtime and validate the entity using spec.
2017:09:12 19:23:48 kevin.lynagh but it doesn't seem like those specs would help me generate events to test against the system
2017:09:12 19:24:00 ghadi your scenario makes sense. I think the fact that you and spec named something the same thing (valid?) but has a different contract is confusing the situation
2017:09:12 19:24:30 kevin.lynagh Ah, I only did that once I gave up on spec =P
2017:09:12 19:30:25 ghadi You might make a spec to check a JWT. (s/def :my.security/jwt ....) Doesn't mean that the token is good enough to get through authentication. Spec will check that it is structurally sound
2017:09:12 19:31:07 ghadi The spec might be pretty involved, but you can't have something that is valid? one minute and not the next
2017:09:12 19:31:42 ghadi spec predicates should not be hooked up to state. a point of the specs is to have enduring, sharable meaning
2017:09:12 19:31:58 kevin.lynagh Right, that makes sense
2017:09:12 19:32:21 ghadi I wish I had a more succinct rationale. Maybe @alexmiller will chime in later
2017:09:12 19:32:33 kevin.lynagh I didn't realize that it was a deliberate decision to prioritize that sharable meaning by disallowing runtime context.
2017:09:12 19:33:06 kevin.lynagh yeah, I hope I don't sound cranky about spec at all --- I think it's a really interesting idea and solves some hard problems.
2017:09:12 19:33:08 ghadi it doesn't disallow it, it's the runtime context (AKA mutable state) that will work against spec's goals
2017:09:12 19:33:20 kevin.lynagh that's why I'm trying to use it in this context
2017:09:12 19:33:51 kevin.lynagh I don't want to rely on implicit runtime context or mutable state
2017:09:12 19:34:00 kevin.lynagh I'm just trying to pass that in
2017:09:12 19:36:10 kevin.lynagh Anyway, thanks for the chat and all your work on this stuff
2017:09:12 19:36:30 kevin.lynagh There are some other places where I've very happily used spec for validation/generation = )
2017:09:12 20:24:37 alexmiller I don’t have a lot to add to Ghadi’s comments which I mostly agree with. Some people are doing runtime generation and registration of specs (by loading values from a db for example). That seems fine to me (it does get a little trickier with s/keys).
2017:09:12 20:26:00 alexmiller in constrast, specs that use stateful stuff (dynvars, atoms, etc) seem bad to me
2017:09:12 20:27:24 alexmiller what is spec buying you in this case vs just writing code to assert whatever predicate you have?
2017:09:12 21:12:49 kevin.lynagh @alexmiller I have the same kinds of problems that spec solves very well: composing predicates and giving back specific errors about why things failed validation. E.g., the AddChild event I mentioned earlier actually needs to take multiple parent-ids, and it would be great to get the detailed validation errors from spec.
2017:09:12 21:13:29 kevin.lynagh "Cannot add child because the third item in the list is of type X, but you can only add children to types U, V, W"
2017:09:12 21:13:48 alexmiller ok
2017:09:12 21:13:54 ghadi you can easily express that in spec if you write a predicate
2017:09:12 21:14:15 ghadi it's the db extrinsic stuff that is a anti-recommendation
2017:09:12 21:16:11 kevin.lynagh @ghadi The predicate can only run against reified entities from the DB. Are you suggesting something like (->> parent-ids #(d/entity db %) #(s/valid? (s/coll-of ::parent))?
2017:09:12 21:17:03 kevin.lynagh Where ::parent is a spec that has the relevant predicates?
2017:09:12 21:18:30 ghadi sure, that's better -- fetch the data before asking spec
2017:09:12 21:19:02 ghadi (I was suggesting just building plain, offline predicates)
2017:09:12 21:21:02 ghadi if there are different types of nodes, some that have parent-ids and some that don't, you should be able to represent that in the data directly, without querying something external
2017:09:12 21:21:32 kevin.lynagh Just to clarify, all of this is happening in an in-memory database, so there is no network fetching or anything like that.
2017:09:12 21:21:50 ghadi Right but if that thing can change, YMMV
2017:09:12 21:21:55 kevin.lynagh it's immutable.
2017:09:12 21:22:12 ghadi ok as long as a payload that is valid? right now, is valid? in the future, that's good.
2017:09:12 21:22:33 ghadi If that can flip, bad kitty
2017:09:12 21:23:42 ghadi i'm assuming you wouldn't call it a database if the data could never change
2017:09:12 21:25:20 ghadi this will probably get me in trouble: specs should only check stuff intrinsic in the data.
Spec can tell you if something conforms to :my.expiration/date... but not if it's expired
2017:09:12 21:28:53 ghadi (aside: are regexes valid specs?)
2017:09:12 21:29:48 ghadi Spec will generate you a bunch of :my.expiration/date's, some might be expired, some not
2017:09:12 21:30:11 kevin.lynagh Another way to phrase that question: Should spec allow you to validate {:time #inst"..." :expr #inst"..."}
2017:09:12 21:30:29 kevin.lynagh part of the spec is that both values should be dates
2017:09:12 21:30:54 kevin.lynagh but should you check in spec whether the :expr date is before or after the :time date.
2017:09:12 21:31:10 ghadi sure
2017:09:12 21:31:56 ghadi it's a bad idea to call (Clock/now) while validating the expiration
2017:09:12 21:32:01 ghadi because that's extrinsic
2017:09:12 21:32:04 kevin.lynagh Yeah, I understand that part
2017:09:12 21:32:13 kevin.lynagh I've been drinking the clojure kool aid for a while, don't worry = )
2017:09:12 21:32:24 ghadi heh
2017:09:12 21:32:51 ghadi Then you should have no problem extrapolating (Clock/now) to a database 😃
2017:09:12 21:33:18 kevin.lynagh calling (d/entity some-db some-id) essentially turns my integer id into an immutable view into the whole database
2017:09:12 21:33:30 kevin.lynagh so reifying the IDs into entities, then calling spec makes sense.
2017:09:12 21:33:34 ghadi it's weird though, when you check {:time #inst"..." :expr #inst"..."} it's a totally different entity that you're checking. Something like a token-request
2017:09:12 21:34:29 ghadi we're in sync
2017:09:12 21:34:48 ghadi i think originally I was thinking the db lookup was happening within the spec definition
2017:09:12 21:35:05 kevin.lynagh well, that was my question, if it'd be possible to do that.
2017:09:12 21:35:11 kevin.lynagh but you can't without dynamic var tomfoolery
2017:09:12 21:35:18 kevin.lynagh which I don't want to do.
2017:09:12 21:35:50 kevin.lynagh It's still awkward, though, since I can't really spec out my events, since I can't spec their integer-id arguments
2017:09:12 21:36:10 kevin.lynagh The best I can do is write my own validation fn that hydrates the ids into immutable entities, then calls spec on those entities.
2017:09:12 21:36:32 kevin.lynagh ditto for generation, I can't use spec to generate anything.
2017:09:12 21:37:04 kevin.lynagh The best I could do is, for a given DB, enumerate all entities, see which ones are valid? against a specific spec, then return their ids
2017:09:12 21:38:49 kevin.lynagh Anyway, I'll give that a shot and see how tidy I can make things. I may come back in a week or two with a follow up report + gist example
2017:09:12 21:38:55 kevin.lynagh Thanks again for the conversation!
2017:09:13 13:14:08 thedavidmeister how would i put together a spec for a map with string keys and values that are vectors of strings?
2017:09:13 13:19:56 thedavidmeister is (spec/map-of string? (spec/+ string?)) ok?
2017:09:13 13:21:19 alexmiller I would recommend coll-of in this case
2017:09:13 13:21:46 alexmiller it gives you more control over size constraints, generation, etc and is a better match for what you described
2017:09:13 13:22:24 thedavidmeister ok cool
2017:09:13 13:22:25 alexmiller s/+ is for describing sequential collections with internal structure along with the other regex ops. while what you have will work, you’re not really using any of those features
2017:09:13 13:22:26 thedavidmeister thanks
2017:09:13 18:21:43 bfabry no doubt this has been asked 1k times, but it's probably evolving. what are people currently doing to turn on instrumentation for their test run? top of the file in test_helpers.clj that's required by every test, fixture that's used by every test?
2017:09:13 18:44:43 alexmiller once fixture seems like a good place for that
2017:09:13 18:54:37 bfabry makes sense
2017:09:13 19:35:33 joost-diepenmaat @bfabry we’re using :once fixtures,
2017:09:13 19:35:50 joost-diepenmaat works ok, unless you forget to use the fixture 😕
2017:09:13 20:53:23 jumar @bfabry we're using lein profiles and injections - something like this:
2017:09:14 20:18:36 akiel I’ve created a lib to support things like form validation using spec. I would be super happy if someone can look at it and give me feedback. Thanks! https://github.com/alexanderkiel/phrase
2017:09:15 06:59:41 mbjarland I have a left field question about spec. There are a number of libraries in clojure for describing binary formats (buffy, octet, etc). Essentially you create a "binary spec" which describes the byte layout of your binary format and then ask the library to parse a byte array, byte buffer etc and back come strings, numbers etc parsed data types. The wish list for the language where you create these binary specs is that they be composable, support references from one spec to another (string length stored in another field), be terse etc. It strikes me that this wish list has a lot of overlap with spec. Would it be sane to try to use spec as the language for describing these kinds of binary formats or is it just a bad fit? You would get a fair amount of power for free and in years to come you would be using a language people are already familiar with...so it seems there are some upsides here.
One example of a good fit is multi-specs which in spec parse maps of different types depending on a 'type' field. There is a very close correspondence in a lot of binary formats with a type int16 or int32 followed by some binary data conforming to the type specified. Collections also match well. Anyway, somebody please call me crazy and tell me why this doesn't work : )
2017:09:15 10:35:11 gklijs It would be great if you would have a higher-level format describing the data, which could be serialized to bytes, and deserialized to language-specific data-structures of different languages. However it is not easy, I had some experience with avro, https://avro.apache.org/ but even for bigintegers it doesn’t work for java out of the box..
2017:09:15 12:40:51 alexmiller I do think there is some overlap in goals and you could build custom spec types for this similar to the regex ops part of spec. Byte level stuff has some special considerations so I don't think you'd want to use what's in spec now for that.
2017:09:15 16:48:33 cgrand @alexmiller when/if you have time could you explain how my hack ruins s/form?
2017:09:15 16:51:46 alexmiller if you have doc, then I guess it’s fine. really I’d just rather actually fix this.
2017:09:16 11:30:13 Olical Afternoon 🙂 I was wondering what sort of performance I could expect from s/fdef? Like, if I don't instrument, is it just a normal defn?
2017:09:16 11:32:50 Olical Also, is it "normal" to automatically enable instrumentation during development within a library? Wondering how much spec goodness I can expose to the consumer of the library automatically without the performance hit in production builds.
2017:09:16 11:35:23 gfredericks @olical the fdef is independent from the defn until instrumentation, so it should have no effect at all
2017:09:16 11:35:52 Olical Awesome, that's what I was hoping for. Apologies if it was mentioned in the documentation and I didn't spot it.
2017:09:16 11:38:30 Olical I was thinking of wrapping (s/instrument) within some sort of (when dev? ...). Not sure if that's possible or a good idea.
2017:09:16 11:39:29 Olical (also I realise I was being dumb, you do defn and fdef next to each other, fdef does not replace defn)
2017:09:18 16:11:32 bbrinck @olical Keep in mind you can place the fdef above the defn (which I prefer for readability). Sometimes poeple don’t realize this
2017:09:18 19:43:59 christianromney @alexmiller love these useful type predicates in clojure 1.9! I'm putting (ex-info?) on my wish list for Clojure Claus... 😄
2017:09:18 19:46:06 alexmiller Ticket and patch would be useful
2017:09:19 12:46:39 alexmiller no, exceptions should be … exceptional
2017:09:19 12:47:27 danielneal Congrats on the 1.9 beta release! I'm just wondering how much of clojure is planned to be specced? I'm guessing it's just the macros - or are there plans to spec clojure core functions as well?
2017:09:19 12:48:03 alexmiller tbd both from a short-term and long-term perspective
2017:09:19 12:49:20 alexmiller there are several categories of fns that can’t currently be instrumented (protocols, multimethods, primitive fns, inline fns) so that limits some things
2017:09:19 12:49:57 alexmiller I’ve worked on others and I think it is valuable to have them. it is also pretty tricky to spec some of them, esp the higher order stuff
2017:09:19 12:50:13 danielneal yeah that makes sense
2017:09:19 12:50:58 danielneal so "everything" - probably out of scope - but there are plans over the long term to spec what would be useful, including the functions
2017:09:19 12:51:39 alexmiller I would hesitantly say yes as there are also performance things to be aware of when you load a large number of specs
2017:09:19 12:52:05 alexmiller right now, the core.specs.alpha has only macro specs in it and that’s loaded implicitly on first macro expansion
2017:09:19 12:52:30 alexmiller I think for fns, we may want to put those in a separate ns that you explicitly load if you want to instrument core
2017:09:19 12:52:44 danielneal ah I see that makes sense
2017:09:19 12:53:26 alexmiller since core.specs.alpha is a separate lib, this is something that can evolve and be released in between clojure releases too. so if it doesn’t get done for 1.9 ga, can still make progress before the next major release
2017:09:19 12:53:53 danielneal Thanks alex
2017:09:19 15:46:32 seancorfield @danieleneal I have almost everything spec'd in clojure.java.jdbc and run the test suite with it instrumentd -- it's quite an overhead. When I get to my desk I can get numbers.
2017:09:19 16:59:16 seancorfield So I have rough numbers now: clojure.java.jdbc has about 600 assertions. Against 1.8 it takes about 30 seconds (about 14 seconds "real" time). Against 1.9 with instrumentation it takes about 100 seconds (about 70 seconds "real" time). @danieleneal /cc @alexmiller
2017:09:19 17:14:31 alexmiller what does it take on 1.9 without instrumentation? :)
2017:09:19 17:14:50 alexmiller comparing apples to orange juice there
2017:09:19 17:33:41 seancorfield Let me just comment out the instrument call to check that...
2017:09:19 17:36:22 seancorfield Pretty much identical. 1.8 looks a hair faster but I'd have to run the test suite a lot more times to be certain.
2017:09:19 17:36:58 seancorfield And the 30/100 seconds above is "user" (cpu) time. The "real" time is elapsed.
2017:09:19 19:53:34 jeaye Yeah, jdbc was heavy for me, when instrumenting as well. Much, much heavier than all of our intrumenting combined.
2017:09:19 19:53:51 jeaye Had to disable it during development.
2017:09:19 19:55:48 seancorfield @jeaye It has specs for all the public API and JDBC stuff tends to get called a lot in DB-centric apps 🙂 If you think it's doing something suspect in its specs that causes undue performance overhead, let me know.
2017:09:19 20:47:37 richiardiandrea hello folks! is there somewhere a wrapper for passing spec keys directly to generate? I mean without the extra nested call to s/gen, ideally (somewhere/generate :my-ns/key)?
2017:09:19 22:40:59 jeaye @seancorfield Will do. I did end up looking, when I saw how long it was taking, and the specs seemed sane to me. It might be worth digging deeper, to see why these specific specs are taking so long. If I had to guess, it's the extensive usage of s/or and s/cat with s/*, wheras just about every spec we have is a s/keys or a clojure.core predicate.
2017:09:20 03:43:24 seancorfield Good point. I might look at refactoring the specs to make them more efficient...
2017:09:20 07:04:04 odinodin Is the JVM opt to disable spec asserts still -Dclojure.spec.compile-asserts=false or did this change with clojure.spec.alpha?
2017:09:20 07:49:37 mpenet it's a compile time thing
2017:09:20 07:49:49 mpenet so doing that at runtime with compiled asserts will do nothing I believe
2017:09:20 07:50:58 mpenet otherwise you might be looking for clojure.spec.check-assertsif asserts were compiled
2017:09:20 12:44:33 alexmiller I believe in both cases we did not put alpha in the property name, but you can check the source
2017:09:20 12:47:26 mpenet no alpha yes
2017:09:20 14:12:43 wagjo Hi, let's say I have a function that takes a nested map and adds a new key somewhere deep.
(defn add-bar
[m]
(assoc-in m [:model/foo :model/bar] 42))
Given the spec for the input argument (and its :model/foo key), what's the best practice to spec out the the return value with :model/bar as required key for :model/foo?
2017:09:20 19:40:07 dbushenko Hi all!
2017:09:21 01:12:40 richiardiandrea Hello! I have probably asked this already at some point in the past, but is there a way to create a Generator that basically uses a function of mine? I am trying with (tcgen/generate (tcgen/->Generator #(faker/name.findName))) but no luck, it always returns nil
2017:09:21 01:14:42 gfredericks @richiardiandrea what sort of thing are you trying to generate?
2017:09:21 01:15:51 richiardiandrea fake data generated from a function...it looks like this works:
2017:09:21 01:22:36 gfredericks is this because you want to use the generators for friendly examples?
2017:09:21 01:22:42 richiardiandrea yep exactly
2017:09:21 01:22:50 gfredericks I almost feel like that should be a different piece of functionality all together
2017:09:21 01:26:35 gfredericks maybe not; I guess it's just the strings that were bothering you?
2017:09:21 01:28:10 richiardiandrea yes I completely agree with you, it could be a separate piece functionality, but generators by definition can be a good fit for that as well
2017:09:21 01:29:20 richiardiandrea if I have a restricted set of things I can use tcgen/elements and that's nice, but for names...something like the above works well
2017:09:21 11:01:28 gfredericks Now I'm thinking this belongs in spec, if anywhere. A function called s/example which will use custom code if registered and fall back on the generator otherwise
2017:09:21 18:35:21 bbrinck If often find myself redefining fdefs a few times to get them right and AFAICT, you need to call instrument after each redefinition. Has anyone else run into this? I suppose I could define a new macro named fdef! that did both steps and then just remove the ! before I commited the code?
2017:09:21 18:36:10 bbrinck some sort of auto-instrumentation mode that worked as I redefined things would be handy. Has anyone written something like this?
2017:09:21 20:40:47 danielcompton @bbrinck I have instrument called in my user namespace that is reloaded after all of the other dependencies are loaded
2017:09:21 20:41:22 danielcompton So whenever I reload the code, (with clojure.tools.namespace) there is an instrument call at the end
2017:09:21 20:41:39 danielcompton I still haven't figured out what's the best way to do this for testing though
2017:09:21 20:43:53 bbrinck Ah, you’re right clojure.tools.namespace would be useful here. Thanks!
2017:09:21 21:15:54 danielcompton I've got a lazy way of doing it, which is just to put the instrument call directly in the user ns, but a fancier way to do it would be to attach it to the c.t.n.r/refresh :after handler
2017:09:22 00:29:53 jeaye Given a fn like (defn foobar ([a] (inc a)) ([a b] (str a b))) where arity 1 expects a number and, let's say, arity 2 expects two strings, can I spec this so that arity 1 doesn't accept a single string?
2017:09:22 00:30:22 jeaye Everywhere I see people spec'ing multiple arities, they'll just use s/or. That leaves all sorts of open holes.
2017:09:22 00:41:25 jeaye Damnit, that worked perfectly. Cheers, @seancorfield. I think I had tried s/alt there, with two s/cats, but it failed.
2017:09:22 00:41:58 seancorfield s/alt will alternate multiple sequence regexes <-- this is poorly worded but I'm not quite sure how to describe its interaction with s/cat
2017:09:22 00:42:39 jeaye This would be a good thing to have in the spec guide, I think; multiple arities aren't covered at all.
2017:09:22 00:44:04 seancorfield It probably doesn't help that the docstrings for s/alt and s/or look so similar. Seems to be a common point of confusion.
2017:09:22 00:44:13 jeaye Yep, agreed.
2017:09:22 00:49:28 seancorfield I just tried it with s/alt in place of s/or and it worked -- but the spec failure you get for (foobar "a") is different...
boot.user=> (foobar "a")
clojure.lang.ExceptionInfo: Call to #'boot.user/foobar did not conform to spec:
In: [0] val: "a" fails at: [:args :one :a] predicate: number?
val: () fails at: [:args :two :b] predicate: string?, Insufficient input
:clojure.spec.alpha/spec #object[clojure.spec.alpha$or_spec_impl$reify__810 0x55365ffb "
compared to
boot.user=> (foobar "a")
clojure.lang.ExceptionInfo: Call to #'boot.user/foobar did not conform to spec:
val: () fails at: [:args] predicate: (alt :two (cat :a string? :b string?)), Insufficient input
:clojure.spec.alpha/spec #object[clojure.spec.alpha$regex_spec_impl$reify__1200 0x373ea576 "
2017:09:22 00:50:38 seancorfield In the first case, it fails to match :args :one :a because "a" is not a number? and it also fails to match :args :two :b because only one string was provided.
2017:09:22 00:51:40 seancorfield In the second case, it only reports the failure to match on alt :two (again, because only one string was provided).
2017:09:22 00:51:58 seancorfield The spec I used was (s/fdef foobar :args (s/alt :one (s/cat :a number?) :two (s/cat :a string? :b string?)))
2017:09:22 00:53:02 seancorfield ^ @jeaye
2017:09:22 01:15:32 jeaye Ah, I was getting the second one, but I was also likely testing with invalid inputs and ... bah, I don't remember. Awesome to see it works well; thanks again.
2017:09:22 01:16:18 jeaye Not having to calculate the various alternations between arities for use with s/or is going to make my macro writing significantly simpler as well.
2017:09:22 05:11:19 jeaye Is there a good spec floating around for specs themselves? They can be ifn? but also things like s/coll-of return something else. I've also tried s/spec?.
2017:09:22 11:50:58 alexmiller For spec forms that is
2017:09:22 11:51:39 alexmiller Currently spec instances should just be considered to be opaque and respond to true for s/spec?
2017:09:22 17:03:11 jeaye Awesome, thanks. Good to see there's work being done here.
2017:09:22 05:36:36 seancorfield That would be a good question for Alex when he's around tomorrow @jeaye
2017:09:22 07:17:13 jeaye Alright, I'll bug him in the morning.
2017:09:22 08:12:55 dbushenko will Alex be today?
2017:09:22 09:23:02 vikeri I can’t seem to get my mutimethods to become intstrumented. Anyone else also having this issue?
2017:09:22 09:39:17 rovanion Multimethods are weird, one of those things I sometimes restart my repl for and it just works.
2017:09:22 09:48:56 andrewmcveigh @vikeri see above (hope the link works)
2017:09:22 09:51:33 vikeri @andrewmcveigh Ok thanks!
2017:09:22 18:43:14 richiardiandrea @gfredericks is it possible to write a generator that continues to generate until the sum of an operation on the generated stuff meets a condition?
2017:09:22 18:43:37 richiardiandrea for instance generate integers until their sum is 5
2017:09:22 18:46:24 tbaldridge @richiardiandrea that doesn't make a lot of sense, since there's a almost unlimited set of numbers that would sum up to 5
2017:09:22 18:46:41 gfredericks you want a collection of integers in particular?
2017:09:22 18:46:45 richiardiandrea right, let's say I restrict the domain
2017:09:22 18:47:05 gfredericks and you want the sum to be =5, or <=5?
2017:09:22 23:25:50 richiardiandrea I see now the reason of this question, I am basically now generating a list of things such that their sum <= total randomly, if < then the last element is manually crafted
2017:09:23 01:11:06 gfredericks yeah, that's definitely a legit way to do it
2017:09:22 18:47:30 richiardiandrea basically I would want to have a such-that, but where I accumulate results first, then I apply pred on the result
2017:09:22 18:47:55 gfredericks one idea is to generate larger sequences and fmap it to the largest prefix that meets your criteria
2017:09:22 18:47:57 gfredericks would that be sufficient?
2017:09:22 18:48:22 richiardiandrea uhm, let me try that, I will be back with the result 😄
2017:09:23 19:43:55 jstew Where is the canonical place to put specs? I read that clojure.core has them in a clojure.core.spec namespace . I'm thinking of making a separate namespace of .spec for each namespace in my app. But I will have some common specs (boilerplate stuff), but don't know where to put those. I'm not sure it matters since all of the defined specs are global.
2017:09:23 20:03:39 gfredericks clojure is canon-poor
2017:09:23 20:57:39 bbrinck I don’t know if there is a standard, but I’ve been putting them in the same namespace as the functions that manipulate the data. That way, other namespaces can require a single ns and get functions+specs
2017:09:23 22:04:46 jstew @bbrinck What about specs that are shared between namespaces? Say you have a spec definition for a core.async channel or a map structure that's shared all over your app, do you duplicate that everywhere or keep those sorts of things in a single namespace
2017:09:23 22:07:26 bbrinck In that case, I’d probably extract to a common “specs” namespace like you’ve suggested.
2017:09:23 22:08:55 bbrinck It’s kind of like generic util functions. I’ll put them in a generic ns until some natural grouping emerges, then extract a more well-defined ns. YMMV
2017:09:25 13:12:26 acron I notice that a value which returns :clojure.spec/invalid under conform does not necessarily return false from a valid? call - is this reasonable? Am I misunderstanding something about valid??
2017:09:26 13:24:23 gfredericks modulo getting all the namespaces right
2017:09:26 13:24:53 mpenet I guess we could have something better in test.check for this
2017:09:26 13:25:33 mpenet in the same vein as gen/vector
2017:09:26 13:25:38 gfredericks Something specialized for strings?
2017:09:26 13:25:42 mpenet yeah
2017:09:26 13:26:24 gfredericks The whole string thing needs an overhaul if anybody can think of a reasonable approach to unicode
2017:09:26 13:26:34 mpenet I know we can abuse string-for-regex in test.chuck for some of it
2017:09:26 13:31:10 gfredericks That is a good example of the issues. Try generating matches for #".*" and see how much you like it
2017:09:26 16:36:01 uwo is it likely or unlikely that s/def will accept docstrings in the future?
2017:09:26 17:55:14 alexmiller Likely
2017:09:26 17:13:30 bfabry I wish there was a way to mark a function as impure in an fdef/fspec that had the consequence of turning on :ret validation for instrumentation and turning off validation of things against that fspec
2017:09:26 17:16:05 fedreg Hi all, is there a way in spec to define a map with something like this: (s/def ::my-map :req-un [::name ::id (one-of ::role ::title)]?? ...So the data could be {:name "tom" :id "1" :role "mgr"} or {:name "tom" :id "1" :title "ceo"} for example. Thx!
2017:09:26 17:18:30 bfabry so xor?
2017:09:26 17:19:51 bfabry it's been a while but what little is left of my Maths for CS class in my brain is saying you can't create an xor with just or and and. so I'd say you'd have to just use or + a predicate
2017:09:26 17:21:01 fedreg I need an s/or but for the keys instead of just the vals
2017:09:26 17:23:33 bfabry well s/keys supports or, so (s/def ::my-map :req-un [::name ::id (or ::role ::title)]) is perfectly valid
2017:09:26 17:24:13 bfabry my point was just that {:name "tom" :id "1" :role "mgr" :title "ceo"} would be valid with that spec (as would your two examples)
2017:09:26 17:25:34 fedreg ahh, missed that. Thanks for clarifying. Didn't realize it was that simple. Thx!
2017:09:26 17:27:08 bfabry np
2017:09:26 20:00:51 mrchance Hi, is there an automated way to get a spec for the output of conform, called with another given spec?
2017:09:26 20:03:41 mrchance I mean:
> (def sp (s/cat :x double? :y double?))
> (s/conform sp [1.0 2.5])
{:x 1.0, :y 2.5} <- I'd like a spec describing this map
2017:09:26 20:15:57 alexmiller no, although that’s been asked a few times.
2017:09:26 20:18:14 mrchance Thanks! So if I wrote one myself, how would I check that they fit together? generative tests?
2017:09:26 20:20:12 alexmiller sure!
2017:09:26 20:20:46 mrchance Ok, will play with it more, thanks 🙂
2017:09:27 01:28:25 waffletower Is there a better way to parameterize generators than to use macros? In this case, I’d like to reuse a generator in different specs with different parameters:
2017:09:27 10:56:42 gfredericks @waffletower at a glance that looks like a macro that could trivially be a function
you don't generally need to write any macros to build generators
2017:09:27 16:45:42 gfredericks @waffletower btw, if you want to retain the gradual-growth property of the builtin generator, you could replace (gen/resize (dec lim) ...) with (gen/scale #(min % (dec lim)) ...)
2017:09:27 16:46:46 waffletower thanks, I had noticed that side effect
2017:09:28 02:28:38 shark8me Hi! is there a (short) way to reference a spec defined in a different namespace?
(ns a.b.c
(:require [cljs.spec.alpha :as s]))
(s/def ::foo int?)
;;; in a different ns
(ns a.d
(:require [cljs.spec.alpha :as s])
(s/valid :a.b.c/foo 10)
Is there a way to avoid typing the fully qualified :a.b.c/foo by using :require :as ?
2017:09:28 02:41:02 shark8me I couldn't find this documented, I assumed it would be :ns/foo instead of ::ns/foo
2017:09:28 06:54:54 jumar No, it's not, the proper syntax really is ::specs-ns/my-spec
2017:09:28 06:20:55 witek Hello. I have a function: (defn f [m]). I expect m to be a map containing a value unter the key :name. The value has to be a keyword. How do I spec this function using (s/fdef f :args #!%&)? What do I have to put in :args?
2017:09:28 07:01:08 jumar @witek If I understand your problem, then s/keys should work just fine:
2017:09:29 00:29:39 seancorfield @johanatan Generators always validate against the spec. Your generator probably generates values that do not satisfy your predicate.
2017:09:29 00:30:34 seancorfield What does your predicate return for "-a" or "---"?
2017:09:29 00:37:44 johanatan @seancorfield ah, good call. looks like first digit needs to be non-dash
2017:09:29 00:39:13 johanatan @seancorfield interesting that I wasn't getting a more obvious exception though (which I did get with a more egregious disparity between the validator and generator)
2017:09:29 00:47:41 johanatan FYI... this version works:
2017:09:29 01:01:15 seancorfield You probably ought to look at Gary Fredericks' test.chuck library -- it contains a generator for regex strings which would make this a lot simpler!
2017:09:29 01:03:14 seancorfield Aside from anything else, you could then define ::kebab-str as a regex predicate and use test.chuck's regex generator directly on the same regex.
2017:09:29 02:20:18 amar Hello! Any ideas on what I’m doing wrong here. (s/valid? ::bar-path ["foo" 1]) is ok, but not so in an s/fdef
2017:09:29 05:34:55 seancorfield @amar The s/cat calls combine to make one sequence regex. I think you can say (s/spec ::bar-path) as the :path argument spec and it will work.
2017:09:29 05:35:45 seancorfield You might also want to look at s/tuple for your ::bar-path spec instead of s/cat.
2017:09:29 11:57:36 amar seancorfield: Thanks. The following worked.
2017:09:30 17:44:59 eggsyntax I feel as though I remember reading that it's OK to have advance spec definitions (ie to define specs whose s/def refers to a spec defined later in the same ns. Can anyone confirm or deny that?
2017:09:30 19:16:51 alexmiller yes (although there are some known cases where that does not yet work right)
2017:10:02 09:21:00 borkdude Is it me or is clojure.future.spec not checking macro specs at compile time?
2017:10:02 11:04:21 gfredericks it couldn't without patching some monkeys, right?
I have no idea if it attempts to do that or not.
2017:10:02 21:27:40 ajs Is it typical to use spec during runtime in production apps, or is more of a debugging tool in development to ensure data is flowing as expected in your app, but turned on off in production?
2017:10:02 21:34:01 bfabry turned off in production
2017:10:02 21:34:18 bfabry except possibly in very specific spots (system boundaries etc)
2017:10:02 21:34:31 bfabry we use it in development, and in our master and staging environments
2017:10:02 21:39:31 guy How big a performance inpact is it to have it running?
2017:10:02 21:48:39 ajs it would seem to me that spec/conform is designed with runtime use in mind for coercing data into a desired format, is this not accurate?
2017:10:02 21:49:57 bfabry @ajs no it absolutely is designed for that, those are the "specific spots" I was talking about
2017:10:02 21:50:37 bfabry ie it'd be odd to be validating/conforming data somewhere other than a system boundary, unless you're looking for bugs in your own code
2017:10:02 21:50:54 bfabry which is what instrument/test.check are for, but they're expensive so just use in lower envs
2017:10:02 21:50:58 ajs is something like goog.debug used for turning it off in production, or is there a spec-specific way to do so
2017:10:02 21:51:09 bfabry you still need to validate/conform at system boundaries though, because you can't trust other people's code
2017:10:02 21:52:48 bfabry there's a few different levers to tweak. instrument is used to add input validation to every function with an fdef. so just not calling that in production. spec/assert is a more explicit validation check, which can be disabled using compile-asserts or clojure.spec.check-asserts
2017:10:03 11:06:18 dbushenko hi all!
2017:10:03 11:07:48 dbushenko I'm specifying an anonymous function, and its return value does not match the spec. Instead of normal error report or something it says like this: clojure.lang.ExceptionInfo: Can't convert path {:form (#function[.../fn--20377/fn--20384]), :val {}, :in [0], :in' []}
2017:10:03 11:07:56 dbushenko is it normal or I'm doing something wrong?
2017:10:03 13:01:56 alexmiller hmm, not sure. repro?
2017:10:03 15:00:46 dbushenko sorry? do you want a gist?
2017:10:03 15:04:45 alexmiller yeah, want to know how to reproduce that error
2017:10:03 15:05:05 alexmiller and then if you can follow it with a (pst *e)
2017:10:03 15:05:41 dbushenko what is this (pst *e) ?
2017:10:03 15:07:55 guy previous stack trace ?
2017:10:03 15:08:04 guy my guess
2017:10:03 15:08:09 alexmiller print stack trace
2017:10:03 15:08:13 guy dang
2017:10:03 15:08:18 alexmiller it is a function in clojure.repl
2017:10:03 15:08:30 dbushenko know what, looks like spec went ok, it was test who produced this strange error report...
2017:10:03 15:08:31 dbushenko 😞
2017:10:03 15:09:36 alexmiller good, because I didn’t know where the “convert path” error was coming from :)
2017:10:03 15:14:35 dbushenko @alexmiller are there any plans to support specing the protocol implementations in records?
2017:10:03 15:17:38 alexmiller not near term. If you could spec them, I’m not sure you have the opportunity to actually do useful stuff with those specs.
2017:10:03 15:25:03 dbushenko @alexmiller maybe you remember I tried to invite you to our conf in December and you couldn't make it 🙂 Can you probably recommend someone from clojure core developers who might be able to visit us with a talk?
2017:10:03 15:28:07 alexmiller lots of fine people - take a look at any of the recent clojure confs (EuroClojure, Clojure/west, Clojure/conj, ClojureX, clojureD, clojuTRE, etc)
2017:10:03 15:28:16 dbushenko ok, thanks
2017:10:03 15:28:44 alexmiller maybe post something here in #events, on the mailing list, on the reddit, etc
2017:10:03 18:37:26 jonas Is there a function in spec similar to spec/assert but that conforms the value? (spec/assert ::a-spec x) returns x (or throws) and not the conformed value of x.
2017:10:03 18:39:21 jonas Perhaps the reason is that you can set *compile-asserts* to false which results in all assertions to compile to x?
2017:10:03 18:49:46 souenzzo instrument?
2017:10:03 18:41:02 bfabry @jonas that would seem like a good reason why that wouldn't work
2017:10:05 07:39:33 dbushenko hi all!
2017:10:05 07:39:47 dbushenko do we have any means of specifying records?
2017:10:05 07:41:20 mpenet s/keys should just work on them
2017:10:05 08:17:39 dbushenko thanks!
2017:10:05 15:19:32 guy Whats the best way to test a spec'd function that has side effects?
2017:10:05 16:04:36 jaymartin @guy I’m not sure about the best way, but mocking can help speed things along if that’s what you’re after.
https://github.com/benrady/specific
2017:10:05 16:26:37 guy Thanks @jaymartin have you tried it with the latest version of clojure.spec.alpha (17) ?
2017:10:05 16:26:47 guy it looks like specific is using alpha13
2017:10:05 18:05:24 jaymartin No, I’ve just messed about with it playfully. But the ideas should be rather generic.
2017:10:05 18:08:07 guy kk thanks 👍
2017:10:05 22:20:20 aengelberg Is there an equivalent of declare for specs?
2017:10:05 22:20:50 aengelberg alternatively, is there a way to disable the check to see if a spec name exists when used in other specs?
2017:10:05 22:24:11 aengelberg basically I would like to structure my code in the following order:
(s/def ::x ::y)
(s/def ::y ...)
2017:10:05 22:24:28 aengelberg replacing the first ::y with (s/and ::y) seems to work.
2017:10:05 23:08:59 bfabry feels kinda odd that it doesn't just work, seeing as most of things in spec are lazy evaluated
2017:10:05 23:08:59 bfabry feels kinda odd that it doesn't just work, seeing as most of things in spec are lazy evaluated
2017:10:06 02:07:15 alexmiller There is a ticket about this. The intent is that it should work
2017:10:06 15:35:59 bfabry good to know, thanks alex
2017:10:06 19:59:55 aengelberg Oh cool, I'm glad that that is the intent
2017:10:05 23:36:51 guy @aengelberg what do you mean
2017:10:05 23:39:13 guy Sorry im probably being slow
2017:10:05 23:39:24 guy he wants to use a different spec and call it something new?
2017:10:05 23:43:41 bfabry he wants to say that the spec for ::foo is the same as the spec for ::bar, and it be ok that the spec ::bar has not been defined yet
2017:10:05 23:44:49 bfabry @aengelberg when I've wanted that I've just used (s/def ::bar any?)
2017:10:06 20:02:17 aengelberg that works too. thanks
2017:10:06 07:39:36 guy kk thanks @bfabry
2017:10:06 09:16:57 kennethkalmer I need some advice, I’ve hit a strange situation while trying to writes specs for data coming in from an external source… The source returns a map that roughly translates to this: {:productStatus {:productStatus "active" ...} ...}
2017:10:06 09:17:50 kennethkalmer how on earth would I spec :productStatus the map, and the string value? I have no control over the output generated by the service
2017:10:06 09:18:58 mpenet either you overspecify by creating separate nses for each, or use something more generic like map-of, if it's only one key its fine, if that s a lot of keys that s so so
2017:10:06 09:23:16 kennethkalmer think separate ns’s are maybe the way to go here, will test that out, thanks!
2017:10:06 09:28:25 kennethkalmer thanks @mpenet! simple, it works, don’t know why I got myself stuck in another loop
2017:10:06 17:58:25 jaymartin This may be highly subjective, but will Spec replace :pre and :post conditions as the idiomatic way to validate inputs and outputs? For example, something like this:
(ns rna-transcription)
(def m
{\G \C
\C \G
\T \A
\A \U})
(defn to-rna [xs]
{:pre [(every? #(contains? m %) xs)]}
(apply str (map m xs)))
2017:10:06 18:00:02 jaymartin Or will :pre and :post still have their place and time?
2017:10:06 18:04:41 bfabry I could still see myself using :pre and :post to check non-local invariants, but that'd be pretty uncommon
2017:10:06 18:23:15 seancorfield @jaymartin You could define specs for your DNA/RNA stuff and use s/valid? in :pre...
(s/def ::dna m)
(s/def ::dna-sequence (s/coll-of ::dna))
(defn to-rna [xs]
{:pre [(s/valid? ::dna-sequence (seq xs))]}
(apply str (map m xs)))
2017:10:06 18:24:01 guy that looks nice
2017:10:06 18:24:05 seancorfield or go further and define ::dna-string as (s/and string? #(s/coll-of ::dna (seq %))) and then {:pre [(s/valid? ::dna-string xs)]}
2017:10:06 18:27:03 seancorfield (I hardly ever use :pre or :post, I must admit -- but I'm an advocate of leaving assertions on in production if you're going to use them at all)
2017:10:06 18:27:24 guy what do you think about instrument in production? 👀
2017:10:06 18:28:03 seancorfield instrument is a different beast, not least because it can trigger generative testing on higher-order function.
2017:10:06 18:29:08 seancorfield In addition, instrument can introduce a big performance overhead depending on how much is spec'd and how they're written. The specs for java.jdbc are a case in point: they introduce a huge overhead.
2017:10:06 18:29:31 jaymartin Super helpful!
2017:10:06 18:29:36 seancorfield (Hmm, that reminds me that I forgot to add :keywordize? to the optional specs in release 0.7.3!)
2017:10:06 18:30:28 guy thanks!
2017:10:06 18:47:11 jeaye @guy We're using it in production.
2017:10:06 18:47:32 jeaye Given our specs, I've measured that we spend about 2% of our time with instrumentation.
2017:10:06 18:48:14 jeaye As Sean said, that can very wildly, depending on how the specs are written. Primarily whether or not there are a lot of alternate paths which need to be considered.
2017:10:06 18:48:52 jeaye For us, it's almost entirely clojure.core predicates and s/keys though, so it ends up being very fast and entirely practical for our production builds.
2017:10:06 19:40:22 guy nice thanks @jeaye!
2017:10:07 07:18:48 ajs Since instrument does not check the return value, it would seem to me that a :post condition is still quite useful.
2017:10:07 08:10:31 mpenet I tend to use s/assert instead nowadays
2017:10:07 09:08:33 ajs with :post at least, you don’t have to let your fn’s return value so you can assert it before you return it
2017:10:07 09:25:55 ajs does s/valid? work on all components of fspec?
2017:10:07 09:29:51 ajs i know that instrument only looks at args, but i haven’t tried out how valid? works on it. will try that when i get home.
2017:10:07 09:29:55 ajs would be cool if that worked
2017:10:07 09:32:15 guy well instrument works on the return value too ?
2017:10:07 09:32:26 guy thats why u have :args, :ret :fn
2017:10:07 11:59:24 ajs @guy no it doesn't. Are you sure? Check the docs. Only args are supported for instrument, though you can specify ret and fn for other purposes.
2017:10:07 12:01:29 ajs "Note that the :ret and :fn specs are not checked with instrumentation..."
2017:10:07 12:08:37 jaymartin @ajs Where exactly are you seeing that note, as I can’t find it.
2017:10:07 12:09:23 ajs The spec guide itself, in instrument section. Why do you think libraries like Orchestra exist?
2017:10:07 12:10:12 jaymartin Silly me, I was looking in the doc string for instrument.
2017:10:07 12:11:46 jaymartin So then, is fdef orthogonal in some way. Its doc string says: “Once registered, function specs are included in doc, checked by
instrument, tested by the runner clojure.spec.test/run-tests, and (if a macro) used to explain errors during macroexpansion.”
2017:10:07 12:12:45 jaymartin I think I need to start over at the spec guide itself and re-read.
2017:10:07 12:12:48 ajs The are tested by instrument for :args only I think
2017:10:07 13:39:43 alexmiller correct
2017:10:07 14:00:55 ajs Other than tests, I gather there's really no way to perform validation on the entirety of a function's signature, including its return value, and also validating that the entire fn received in a higher order function, is what is expected, is that right?
2017:10:07 15:50:33 alexmiller stest/check is the only thing in spec that validates :ret and :fn specs
2017:10:07 15:51:18 alexmiller I’m not sure I fully understand the question re hof
2017:10:07 16:25:28 ajs That answers it!
2017:10:07 13:47:02 taylor I’m trying to write a spec for a collection where I need the first item to conform to another spec. s/cat works great, but when I fdef and check an unary function of that collection it looks like test.check is applying the collection to the function, and so I get arity exceptions. Should I change the function to take varargs?
2017:10:07 13:48:46 gfredericks are you using s/cat to express the whole arglist or just the first arg?
2017:10:07 13:49:04 gfredericks I think to use a regex for a single arg you'll need to wrap in s/spec, maybe that's what you're missing
2017:10:07 13:49:16 gfredericks (s/cat :first-arg (s/spec (s/cat ...))) something like that
2017:10:07 13:49:42 taylor yeah, my :args is (s/cat :clause (s/spec ::group))
2017:10:07 13:50:11 gfredericks I don't know then. You certainly should be able to spec that.
2017:10:07 13:56:17 taylor actually it does work when I wrap with s/spec, but I think I need to limit the test cases because it’s running forever
2017:10:07 13:57:09 taylor the spec is recursive and I guess is getting really complex samples
2017:10:07 13:57:23 gfredericks yeah 😞 the Big Problem of spec/test.check
2017:10:07 13:59:21 taylor well the good news is this is a toy problem 🙂
2017:10:07 14:13:46 taylor it runs pretty quickly like that
2017:10:07 14:18:12 gfredericks I think so. what's it default to?
2017:10:07 14:20:16 tbaldridge The depth of spec checking is somewhat limited by the JVM stack size, that keeps the generators from blowing the stack. Without it, you’d get a stack overflow from almost any recursive structure.
2017:10:07 14:25:33 gfredericks @tbaldridge you're saying *recursion-limit* refers to checking rather than generation?
2017:10:07 14:28:05 tbaldridge It only refers to generation. Checking is assumed to be bounded by the depth of the input
2017:10:07 14:29:05 gfredericks okay.
I think there could be a more automatic approach to handling the recursion, but I'm not sure.
2017:10:07 14:29:47 gfredericks I have to come up with something for the vanilla test.check collection generators, so whatever works there might apply to how spec assembles generators
2017:10:07 14:30:13 tbaldridge There probably could be. That dynamic var got added in one of the pre-alpha builds when I tried to build a recursive spec and it would stack overflow on the first structure generated.
2017:10:07 14:31:01 tbaldridge Checking is fine since that's GIGO, so it's on the user not to check data that's millions of levels deep.
2017:10:07 14:31:42 tbaldridge @gfredericks any insight in the past year about shrinking recursive generators?
2017:10:07 14:32:49 tbaldridge They currently shrink by width (and a b) -> (and a), but not by depth: (and (and (and a))) -> (and a). I know we discussed this once, not sure if anything has changed in the past year
2017:10:07 14:33:09 gfredericks I don't think I have any thoughts that I didn't put on the ticket
2017:10:07 14:59:59 taylor Ha. This is exactly the type of structure I’m generating too
2017:10:07 15:04:10 gfredericks @U3DAE8HMG important to note here that spec doesn't use recursive-gen
2017:10:07 15:12:15 gfredericks But spec's approach has the same problem, but even worse
2017:10:07 14:35:43 gfredericks the unfortunate part is that after generating a collection, test.check can't (in general) tell what parts the children are
2017:10:07 14:36:47 gfredericks and you'd need their shrink-trees in any case, not just the data
2017:10:07 14:37:49 gfredericks oh ummmm
2017:10:07 14:39:54 gfredericks I was thinking that there's an unsound approach where you wrap the child generator in an observer that keeps track of how it was called and use that to add shrinking info
I thought it was unsound because you can't tell just because the generator was called, that the result actually appears in the final structure
but I just realized that that probably doesn't matter. anything generated by the child generator should be a valid value from the final generator, so even if it doesn't appear in the actual data, it's not an invalid thing to shrink to in the strict sense
2017:10:07 14:41:36 gfredericks so now the worst thing about this approach is that it's a bit messy
2017:10:07 14:42:04 gfredericks but I think everything still ends up being deterministic
2017:10:07 14:42:56 gfredericks I'll go comment on the ticket. I don't have time to work on it until at least thursday
2017:10:07 15:14:20 gfredericks Well maybe not worse....
2017:10:07 15:30:03 tbaldridge ah right, I forgot about that. Spec's stack overflow happened during the creation of the generator
2017:10:07 15:48:39 gfredericks ah yeah
2017:10:07 15:49:05 gfredericks the thing about spec's approach is that it's committing to a particular depth in the generator rather than describing the recursion the way the spec itself does
2017:10:07 15:49:22 gfredericks it might be making other commitments in the process, I'm not sure
2017:10:07 15:49:42 gfredericks I'm not sure this is a significant problem, it just complicates things
2017:10:09 10:05:03 jwkoelewijn Hi all, does someone know if there is a way to limit generators? I have created specs with generators, and generating a top-level entity now takes multiple seconds, which in turn seems to limit me from using generative testing
2017:10:09 11:06:55 gfredericks @jwkoelewijn I'm not familiar with all the entry points, but test.check has a :max-size option that spec will pass through, that you could set to 50 or 15 depending on what you're doing exactly when the multiple-second thing happens
2017:10:09 11:13:17 jwkoelewijn cool, will look into that options, thanks!
2017:10:10 01:50:24 erichmond Is there a good doc/faq/something on using spec's ability to generate mock data? Generating a string like "001212314" in particular? (meaning, n length, chars consisting of just 0-9?
2017:10:10 02:47:38 alexmiller spec itself does not really have something like that built in
2017:10:10 02:47:59 alexmiller you can use fmap to build a set of characters, then apply str to them
2017:10:10 02:48:10 alexmiller or use the regex gen support in the test.chuck library
2017:10:10 03:09:48 erichmond I had been using test.chuck, but was curious if there was a more spec-native solution. thanks, I'll keep going with what I have!
2017:10:10 03:31:35 alexmiller test.chuck is rad
2017:10:10 13:53:54 jwkoelewijn question, when i run (clojure.spec.test/check 'merge-users {:clojure.spec.test.check/opts {:num-tests 15 :max-size 2}}) it seems the num-tests option seems to be ignored. Am I passing the options for quick-check wrong? or is this to be expected and should I address my own expectations? 🙂
2017:10:10 13:55:18 jwkoelewijn where I use the backtick instead of the quote (to help in markdown slack formatting)
2017:10:10 15:13:26 borkdude In Clojure spec, how could I generate symbols or strings of max length 2 that contain only lowercased alphabetic characters?
This works, but maybe it can be simplified?
2017:10:10 17:05:00 jonas With spec/keys (or spec in general), what’s the best way to express: One and only one of the keys ::foo or ::bar is required?
2017:10:10 17:06:25 jonas Or alternatively “Either ::foo or ::bar or both is required”
2017:10:10 17:06:26 bfabry (s/keys :req [(or ::foo ::bar)]) and then and'd with a spec to enforce only one if you want that
2017:10:10 17:07:09 jonas Thanks, let me try that
2017:10:10 17:07:12 bfabry
The :req key vector supports 'and' and 'or' for key groups:
(s/keys :req [::x ::y (or ::secret (and ::user ::pwd))] :opt [::z])
2017:10:10 17:07:47 jonas I think this is exactly what I was looking for :+1:
2017:10:10 20:48:44 nwjsmith Crazy, I was just reading your blog post. Great stuff!
2017:10:10 23:59:53 nwjsmith How are you folks debugging Couldn't satisfy such-that predicate errors?
2017:10:11 00:00:41 nwjsmith I’m having trouble figuring out which generator is causing the error
2017:10:11 00:06:00 bfabry @nwjsmith one option would be to walk the registry using s/registry and try and generate an example for each one until you fail
2017:10:11 00:06:19 bfabry I believe there's a jira open to figure out a way to add the offending spec to the error message you're seeing
2017:10:11 00:07:10 gfredericks s/and is a common culprite
2017:10:11 00:08:24 gfredericks the way to add the offending spec to the error message is implemented in the latest alpha of test.check, but I doubt has been incorporated into spec. I assume there's an open ticket for that.
2017:10:11 00:16:41 nwjsmith my utilities.string/ascii? function is broken
2017:10:11 00:18:25 gfredericks phew
2017:10:11 07:27:27 msolli I’m specing a function that calls another function, stubbing out that second function with instrument. When I’m doing stest/check on the function, it’s really slow. Takes ~30 s with :num-tests 1000. With num-tests 100, it’s sub-second. What gives?
2017:10:11 10:58:17 gfredericks @msolli how about {:num-tests 1000 :max-size 100}?
2017:10:11 11:07:15 msolli Yeah, that took the time down to ~4 s.
2017:10:11 11:09:02 msolli So I understand :max-size somehow controls the way generated values “grow” as the number of tests increase. How does this explain why larger sizes takes so much more time?
2017:10:11 11:17:42 msolli I can confirm that the values generated for any? really do become huge.
2017:10:11 11:20:53 msolli So I guess it’s the GC-ing of those large objects that is the bottleneck.
2017:10:11 11:22:02 alexmiller There are some tickets and work to do in this area still
2017:10:11 11:25:20 alexmiller @gfredericks the most critical fix was the bug in s/coll-of that went into the spec release last week actually. That was the cause of one set of problems (not the one here though).
2017:10:11 11:27:38 gfredericks I can't see anything that looks particularly relevant in the last 6 months of commits
2017:10:11 11:28:18 gfredericks oh was it generating a bunch of stuff and throwing it away?
2017:10:11 11:28:53 msolli The conj, well, enjoy yourselves! And thanks for the explanation and all the great work you’re doing.
2017:10:11 11:32:52 gfredericks @alexmiller after reading the ticket, I assume CLJ-2103 is what you were referring to
Any thoughts (ha!) about any?? The underlying gen/any-printable in test.check could have its size scaled down. I feel like that would make it more useful for almost any conceivable use case.
2017:10:11 11:38:57 alexmiller Yeah that would be good
2017:10:11 11:39:43 alexmiller And yes 2103
2017:10:11 11:41:04 gfredericks although now that I think about it, making sizing better in collections and gen/recursive-gen might take care of the gen/any-printable problem
2017:10:11 13:30:43 mmer Is there anyway to get the context of a element being validated by a spec. I need to get hold of the overall datastructure that is being validate by a set of specs so that I can do some cross validation in my own predicate?
2017:10:11 14:13:06 taylor I commented on your Stack Overflow question too. I think maybe the solution to your problem is to enforce that invariant from your outer spec instead of your inner spec?
2017:10:11 14:20:47 taylor posted example on SO
2017:10:11 14:32:52 taylor
(s/def ::tag string?)
(s/def ::inner (s/keys :req-un [::tag]))
(s/def ::outer
(s/and
(s/keys :req-un [::inner ::tag])
#(= (:tag %) ;; this tag must equal inner tag
(:tag (:inner %)))))
(s/conform ::outer {:tag "y" ;; inner doesn't match outer
:inner {:tag "x"}})
;=> :clojure.spec.alpha/invalid
(s/conform ::outer {:tag "x"
:inner {:tag "x"}})
;=> {:tag "x", :inner {:tag "x"}}
2017:10:11 15:46:57 mmer Thank you - I kind of understand. I need to work through some details.
2017:10:11 22:05:42 mmer What seems strange is that with the whole conformed model that it is not easier to walk the model. In other words it woul dbe good to be able to apply the equivalent of an xpath to the specs to be able to find associated data and perform link and checks between them
2017:10:11 21:45:30 dealy trying to spec a function that takes 2 params, a keyword and a set of string (which can be nil), but my spec always fails when a set is passed to the function, how should it be spec'd
2017:10:11 21:46:12 dealy (s/fdef cs!
:args (s/cat :mis keyword? :table-names (or nil? (s/coll-of string?))))
is the spec I was using
2017:10:11 21:50:16 bfabry you want (s/or (s/coll-of string? :kind set?) nil?) but really you want (s/nilable (s/coll-of string? :kind set?)
2017:10:12 04:24:32 uwo Is there a way to ask of a spec, “what keys can you have?”
2017:10:12 04:25:44 uwo I know we can call (s/describe :my/spec) and get back a form. I would need to parse that s expression to get that answer - which is fine of course
2017:10:12 14:48:39 guy Hi i've got a newbie spec question.
Should you reuse specs from one ns to another.
Say i have some data that has :name :email and wanted to validate it with spec.
Would i create a ::name ::email in each namespace that checks it, or would i create a spec namespace and import it from there.
Or should i do something else?
2017:10:12 15:29:13 taylor Yes, it’s normal and sometimes necessary to use specs across namespaces. If you have common/shared specs then you should definitely consider defining them in one place and referencing them where they’re needed.
2017:10:12 15:31:53 guy ok thanks!
2017:10:12 18:51:25 the2bears Hi, I'm starting out with spec and adding it to a small project we have. However, the project uses Incanter and a 'lein compile' now fails.
Caused by: clojure.lang.ExceptionInfo: Call to clojure.core/defn did not conform to spec:
{:clojure.spec.alpha/args (matrix "\n Returns a matrix or vector, in a valid core.matrix format. You can use the slices function to\n access the rows.\n\n Equivalent to R's matrix function.\n\n Examples:\n (def A (matrix [[1 2 3] [4 5 6] [7 8 9]])) ; produces a 3x3 matrix\n (def A2 (matrix [1 2 3 4 5 6 7 8 9] 3)) ; produces the same 3x3 matrix\n (def B (matrix [1 2 3 4 5 6 7 8 9])) ; produces a vector with 9 elements\n\n ; since (plus row1 row2) adds the two rows element-by-element\n (reduce plus A) ; produces the sums of the columns\n\n ; and since (sum row1) sums the elements of the row\n (map sum A) ; produces the sums of the rows\n\n " ([data] (m/matrix data)) ([data ncol &] (m/matrix (partition ncol (vectorize data)))) ([init-val rows cols] (m/compute-matrix [rows cols] (constantly init-val))))}
2017:10:12 18:51:51 the2bears I imagine Incanter has not been updated, but I also don't read the spec errors very well yet.
2017:10:12 18:54:07 bfabry what version of incanter?
2017:10:12 18:54:54 bfabry and do you have more error/stacktrace?
2017:10:12 18:54:58 the2bears 1.9.0
2017:10:12 18:55:07 the2bears I do have more trace, hold on
2017:10:12 19:00:12 the2bears Shoot, that cut off some
2017:10:12 19:02:54 guy u cud try putting it in a gist
2017:10:12 19:03:25 the2bears yeah, would be better I think.
2017:10:12 19:06:00 bfabry ah, yes, bug in incanter
2017:10:12 19:06:10 bfabry this is not a valid args list
2017:10:12 19:09:01 bfabry oh actually it's fixed in master
2017:10:12 19:09:15 bfabry try the latest snapshot maybe
2017:10:12 19:09:37 the2bears okay, I can do that for testing this out. Thanks again for the help.
2017:10:12 19:09:48 bfabry oh no I'm on the wrong branch
2017:10:12 19:09:49 bfabry 1s
2017:10:12 19:10:42 bfabry yeah fixed on 'develop' branch
2017:10:12 19:12:49 the2bears okay, I'll pull it down and test a local.
2017:10:13 09:49:49 misha great talk, @gfredericks, thank you!
now we need to persuade you into writing a book on test.check and gen testing
2017:10:13 12:03:32 alexmiller that is a fantastic idea
2017:10:15 01:24:42 codonnell am I missing something about conformers? I'm confused as to why (s/valid? (s/conformer (fn [_] :clojure.spec/invalid)) nil) is true. If the conformer function returns :clojure.spec/invalid, shouldn't the value be invalid?
2017:10:15 01:27:50 codonnell ah, it should be clojure.spec.alpha/invalid or just ::s/invalid:duck:
2017:10:15 07:26:35 jumar what exactly is the relationship/difference between clojure.spec.gen.alpha and clojure.test.check.generators?
So far, I've found two differences:
1. missing nat generator in clojure.spec.gen.alpha
2. cannot reference generators directly, but has to call them as functions:
e.g. with clojure.test.check.generators I can do this:
(gens/hash-map
:boolean gens/boolean
;; note: `nat` generator is not in spec
:small-integers (gens/vector gens/nat)
:large-integer gens/large-integer
:double gens/double
:color (gens/elements [:red :green :blue])
:uuid gens/uuid
:string-or-keyword (gens/tuple gens/string gens/keyword))
But with clojure.spec.gen.alpha I have to do this:
(gen/hash-map
:boolean (gen/boolean)
;; note: `nat` generator is not in spec
:small-integers (gen/vector gens/nat)
:large-integer (gen/large-integer)
:double (gen/double)
:color (gen/elements [:red :green :blue])
:uuid (gen/uuid)
:string-or-keyword (gen/tuple (gen/string) (gen/keyword)))
2017:10:15 07:34:43 jumar clojure.test.check.generators/let is also missing in clojure.spec.gen.alphanamespace
2017:10:15 07:37:48 jumar Related to the 2.) When I actually try to use (gens/double) it throws an exception so the usage pattern is a bit inconsistent
2017:10:15 13:28:03 gfredericks what kind of exception?
2017:10:15 18:13:33 jumar
1. Unhandled java.lang.ClassCastException
clojure.test.check.generators.Generator cannot be cast to clojure.lang.IFn
2017:10:15 18:14:22 jumar I don't have anything against either approach, I just thought that they behave in the same way O:-)
2017:10:15 18:37:55 gfredericks oh, you mean between the two namespaces it's inconsistent
yes absolutely, and that's unfortunate
2017:10:15 07:51:28 jumar btw. @gfredericks thanks a lot for the great Conj talk. I missed it at the conference but watched it now. Really useful!
2017:10:15 07:58:56 jumar just a small thing. imho, gen/large-integer doesn't accept min/max options. I had to use gen/large-integer* (slide 43).
Funny thing is that if you use clojure.spec.gen.alpha/large-integer it doens't throw an exception but generates wrong data
2017:10:15 13:29:36 gfredericks the other thing is definitely a slide typo 😞
2017:10:15 13:30:15 gfredericks my org-mode⇒beamer workflow is not yet sophisticated enough to automatically verify that the code works
2017:10:15 13:48:19 stathissideris what do people normally do for spec’ing var-args
2017:10:15 13:48:43 stathissideris do they put the whole thing in a s/?
2017:10:15 13:49:07 stathissideris it feels slightly wrong because that means that it’s all or nothing
2017:10:15 13:51:53 gfredericks @stathissideris does s/* not work?
2017:10:15 13:54:15 stathissideris ok some context: I’m trying to extend spec-provider to support inference of function specs, and when it comes to varargs, if I see an example of a function call where the varargs happen to be 1 "foo" I’m not sure if I should infer a spec of (s/cat ... :rest (s/? (s/cat :integer integer? :string string?)))
2017:10:15 13:54:43 stathissideris which says that there are up to 2 varargs but they’re both optional
2017:10:15 13:55:11 stathissideris feels like it’s more restrictive than the function signature
2017:10:15 13:55:32 gfredericks if all you know is one example call, I don't think you can know very much at all for sure
2017:10:15 13:56:07 stathissideris agreed…
2017:10:15 13:56:34 stathissideris but even if I get more calls, I’m not sure what’s the “best” spec to infer
2017:10:15 13:56:57 stathissideris for example let’s say I get one more example where the varargs is just 20
2017:10:15 13:57:21 stathissideris does it then become (s/cat ... :rest (s/? (s/cat :integer integer? :string (s/? string?))))
2017:10:15 13:58:09 stathissideris or maybe I always wrap the individual var-args with s/? because they’re optional
2017:10:15 14:01:24 gfredericks there's no right answer, it's all heuristics
2017:10:15 14:01:36 stathissideris I know 🙂
2017:10:15 14:03:39 gfredericks which I guess means I have no further advice 🙂 those kinds of problems make me just throw up my hands and give up
2017:10:15 14:04:58 stathissideris they are hard, but they make you think hard about idioms and coder expectations
2017:10:15 14:05:14 stathissideris I had to do a lot of that for spec-provider
2017:10:15 14:05:31 stathissideris @gfredericks great talk btw
2017:10:15 14:05:50 gfredericks thanks
2017:10:15 14:54:05 stathissideris what’s the difference between s/alt and s/or?
2017:10:15 14:54:19 stathissideris I saw Ambrose used alt for multiple arities
2017:10:15 14:54:29 stathissideris I’ve always used or
2017:10:15 14:55:02 taylor
;; unlike `or`, `alt` works on sequence elements, not on values themselves!
2017:10:15 15:00:06 stathissideris ok, so it’s like with alt you have an implicit s/cat around each option
2017:10:15 15:00:06 stathissideris ok, so it’s like with alt you have an implicit s/cat around each option
2017:10:15 15:08:26 taylor I think it's more like all the "regex" spec types (`cat`, alt, *, +, ?) are specifically meant to specify elements of a sequence. I wouldn't say there's an implicit cat around alt or any of the regex specs, they all work on sequences whether you use them alone or in combination
2017:10:15 15:11:10 taylor I'm probably missing some context for your use case though? Writing specs for multi-arity functions?
2017:10:15 15:18:26 taylor The fdef docstring has a nice example:
2017:10:15 15:22:55 stathissideris yeah, it’s longer, but are there any other disadvantages?
2017:10:15 15:27:50 taylor Not sure what other disadvantages there might be. The :args spec isn't just longer with or, it's more complex. They also produce slightly different output e.g. when incorrectly calling a instrumented function
2017:10:15 15:28:46 taylor The or version has additional, unnecessary tags in its explanation
2017:10:15 15:30:42 taylor from the :clojure.spec.alpha/problems: :path [:args :sym] vs :path [:args :sym :sym]
2017:10:15 15:36:22 alexmiller Regex specs compose together to describe a single sequential context and are generally preferred for describing args
2017:10:15 15:37:28 alexmiller Generally it's best to use a top level cat as you will then get a map conformed to component names that you can use in the :fn spec
2017:10:15 15:38:19 stathissideris @U064X3EF3 but are different arities a “single sequential context”?
2017:10:15 15:38:35 alexmiller For any given case, yes
2017:10:15 15:38:49 alexmiller I would actually write that example now as
2017:10:15 15:39:58 alexmiller Sorry, I can't type it all, on the phone
2017:10:15 15:43:20 stathissideris no problem, would be grateful to have your example when you get some time
2017:10:15 15:44:38 alexmiller Use alt for alternatives, ? for optional args, * for varargs etc. you can describe any set of arities I've encountered with a regex spec
2017:10:15 15:45:44 stathissideris on a bit of tangent: what about keyword args?
2017:10:15 19:34:42 alexmiller Use keys* - that's what it's for
2017:10:16 08:21:10 stathissideris ok, many thanks!
2017:10:16 13:58:47 bbrinck I’m unclear on how conformers are intended to be used.
2017:10:16 13:59:26 bbrinck I’ve seen cases like:
- converting a string into a seq so spec regex operators can validate a string
- converting a string into an int
- converting a string into a UUID
2017:10:16 14:02:18 bbrinck Are the above cases unintended uses of conformers? What would be an intended use case? I ask because I’m trying to figure out how to include support for conformers in Expound
2017:10:16 15:31:04 alexmiller the intended use case was for writing new composite spec types (like s/keys*)
2017:10:16 15:31:23 alexmiller so basically not what most people try to use them for
2017:10:19 07:41:17 jrychter FWIW, I also use them for things like converting strings to keywords in JSON.
2017:10:19 07:42:45 jrychter I think that while this might not have been the intended usage, it shows that there is a need here. I am having hopes that Rich will take this on as something to think about in the future.
2017:10:23 22:13:05 bbrinck For what it’s worth, using conformers in this way also makes it harder to give good error messages. I’m not sure what the best solution is, but perhaps two phases where first the data is validated, then transformed
2017:10:16 14:03:54 mpenet to give you information about what you are validating for reuse "later", this makes more sense in the context of validating/destructuring data passed to a macro such as defn
2017:10:16 14:05:23 bbrinck Do you happen to have an example? My main source of examples of macros specs is https://github.com/clojure/core.specs.alpha, but I don’t see any conformers there
2017:10:16 14:07:27 mpenet my bad I read conforming and I got carried away
2017:10:16 14:10:16 mpenet I am guilty of using it for transformation, not sure my example is clean but here you go: we have a dsl a bit like sql for a rule engine, we validate the syntax with spec and use a conformer to spill the ast (parsed form)
2017:10:16 14:14:19 bbrinck @mpenet Hm, that is interesting. Is the DSL originally a string then? Do you use spec to parse the string?
2017:10:16 14:14:22 mpenet also guilty of using it do to json transforms, "old" stuff
2017:10:16 14:14:25 mpenet yes
2017:10:16 14:14:47 mpenet the spec is basically a conformer that wraps the parse fn
2017:10:16 14:14:58 bbrinck Right
2017:10:16 14:15:13 mpenet in reality it s a lot hairier because we have custom explain and whatnot but that s the idea
2017:10:16 14:15:49 bbrinck I see, thanks for the example. One reason it’s hard to add support for conformers is that a) I’m unclear on the intended use and b) in practice, it seems like they are often uses for transformation. I’m not sure if it’s practical to not support a common use case.
2017:10:16 14:15:57 bbrinck Right
2017:10:16 14:16:30 mpenet it s made for transformations, I guess their validity depends on the context
2017:10:16 14:21:20 bbrinck This is the part I don’t fully understand: conformers do transform values, but what are examples of recommended transformations? @alexmiller said: “I don’t think we are interested in turning spec into a transformation engine via conformers”, but I’m not sure what that includes/excludes
2017:10:16 14:22:34 mpenet it's a good question indeed
2017:10:16 14:24:10 mpenet neither the guide or rationale pages mention it
2017:10:16 14:24:30 bbrinck @mpenet In any case, I appreciate you explaining how you use them in practice. This is very helpful for my research, since I may end up wanting to support different use cases
2017:10:16 14:25:30 mpenet you're welcome. love expound. thanks for that
2017:10:16 15:32:17 alexmiller the guide page intentionally does not mention conformers as we consider them to be primarily useful for writing new custom composite spec types (not for general data transformation)
2017:10:16 15:44:07 bbrinck By “composite spec types” do you mean new spec types that work similar to or or regex specs? i.e. where I would want to name the parts?
2017:10:16 19:11:03 bbrinck Whoops, sorry I missed your comment addressing exactly this question in the thread above. Thanks!
2017:10:20 10:31:30 carocad @alexmiller would you be so kind to put an example of the “expected use” in ? 🙂
Currently only the “unintented use” is documented so I guess that this leads people in the wrong direction (and might continue to)
http://clojuredocs.org/clojure.spec/conformer
2017:10:16 15:33:04 alexmiller I think having a tool for transforming data informed by specs would be awesome. I don’t think conformers are that tool.
2017:10:16 17:24:41 ikitommi @alexmiller any ideas how could we find such a tool? It looks like we could do that with conformers, with just small changes to how it works. Or is there something new coming from Rich?
2017:10:16 15:50:46 danielneal I'm trying to use spec to document a map that contains a :key-fn that returns a collection
i.e. on this data {:aliases #{:bob :robert}}, {:key-fn :aliases} would be suitable.
However this happens
2017:10:16 15:50:46 danielneal I'm trying to use spec to document a map that contains a :key-fn that returns a collection
i.e. on this data {:aliases #{:bob :robert}}, {:key-fn :aliases} would be suitable.
However this happens
2017:10:16 15:56:28 taylor I think the problem is that a keyword (`:aliases`) won’t conform to your fspec
2017:10:16 15:57:16 taylor You could add an s/or case for keywords in your ::key-fn spec
2017:10:16 15:57:50 danielneal
2017:10:16 15:57:54 taylor (s/or :kw keyword? :fn (s/fspec ...))
2017:10:16 15:58:23 danielneal the thing is it's the fact that the function returns a collection that is the important thing I'm trying to capture
2017:10:16 15:58:54 danielneal I wouldn't mind if I have to help it along with a generator or something
2017:10:16 15:59:09 danielneal but I don't know where or how or if I'm approaching the whole thing in a stupid way
2017:10:16 16:00:15 danielneal oh no wait
2017:10:16 16:00:20 danielneal the inlined function with constants worked
2017:10:16 16:01:25 danielneal I would like to specify that if you're gonna give a keyword it's got to look up data that is a collection
2017:10:16 16:05:42 taylor hmm maybe need another approach if you need that guarantee… a keyword could return anything that happens to be in the map
2017:10:16 16:08:50 danielneal mm yes
2017:10:16 16:09:16 danielneal as far as I can make out specifying the return types of functions <as> parameters is a bit of an edge case
2017:10:16 16:09:55 danielneal but would love to be able to provide it to help readers understand the contract
2017:10:16 16:10:26 taylor so I guess you’d need more context if you were going to validate that the supplied keyword would return a collection; you’d need to know 1) what the keyword is and 2) the value for that keyword in the other map it’d be applied to?
2017:10:16 16:14:20 danielneal yeah you're right
2017:10:16 16:14:46 danielneal I suppose I could just put it in the doc string
2017:10:16 16:15:15 taylor one weird idea: if you had all this info (including the “subject” map) in one data structure, you could write a spec for the whole structure to ensure any ::key-fn refers to a collection value in the “subject” map
2017:10:16 16:15:34 danielneal ooh interesting
2017:10:16 16:32:23 taylor Anyway, the return value spec for your function isn’t going to matter for valid?. It’d be useful if you were using test.check to check the function though
2017:10:16 20:38:47 akhudek Is there a way to write a spec that checks properties of pairs of elements in a sequence? E.g. in [{:a 1 :b 2} {:a 2 :b 3} {:a 3 :b 4}] I’d like to check that :b is equal to :a for each pair of items from the sequence.
2017:10:16 20:38:56 akhudek I can do this as a function on the sequence.
2017:10:16 20:39:13 akhudek But then on failure explain returns the entire sequence as the value causing the problem.
2017:10:16 20:39:32 akhudek erm, let me fix that
2017:10:16 20:40:00 ghadi you can do it but not as a function on the sequence, but each element of it
2017:10:16 20:40:39 ghadi unless I understand you wrong
2017:10:16 20:41:08 ghadi Do you mean there is some relationship between adjacent maps? or within individual maps?
2017:10:16 20:42:42 akhudek I mean that for {:a 1 :b 2} and {:a 2 :b 3}:b of the first and :a of the second must match, and then for {:a 2 :b 3} and {:a 3 :b 4}, we have the same check and so on
2017:10:16 20:42:46 ghadi gotcha
2017:10:16 20:42:50 ghadi you have a tradeoff here
2017:10:16 20:43:37 ghadi You can individually check pairs by running (map #(s/valid? ....) (partition 2 1 collection))
2017:10:16 20:43:52 ghadi iff you want more targeted error messages
2017:10:16 20:44:09 ghadi otherwise you can continue to check the whole collection and get errors at the collection level
2017:10:16 20:44:22 ghadi depends on what you want
2017:10:16 20:44:32 akhudek ok, thanks, that’s what I thought but wanted to make sure there wasn’t some other way 🙂
2017:10:16 20:44:53 ghadi I don't believe that (at this time) there is a have-your-cake-and-eat-it-too...
2017:10:17 05:51:55 talgiat Are there ways to validate a form with spec, with user friendly error messages (not developer friendly). I’ve found this: https://github.com/alexanderkiel/phrase but I don’t want to do field by field validation but validate a whole spec that is a map with field dependent error messages.
2017:10:17 13:54:29 bbrinck Can you talk more about the map of fields? Do you have an example?
2017:10:18 17:39:28 seancorfield @joost-diepenmaat bear in mind that s/nonconforming is undocumented and may disappear. I believe the Clojure/core folks have said the most likely "solution" to this is that they may provide a specific non-conforming version of s/or, rather than exposing general non-conforming machinery... /cc @andrewmcveigh
2017:10:18 17:47:54 ikitommi calling s/unform on the conformed value will also remove the branching info.
2017:10:18 17:49:09 ikitommi s/nonconforming also stops all conforming, so the conformers on the or branches are not run.
2017:10:18 17:50:29 joost-diepenmaat I could possibly use a multi spec. For now I could make it work with a single spec
2017:10:19 07:46:43 jrychter I am having problems with circular dependencies. Spec strongly encourages splitting your data entities into separate namespaces. I have projects (`project`) which contain entries (`entry`). But entries also refer back to :project/id via a :entry/project-id attribute. If I split entries into their own namespace, I have to refer to it when defining what a project is, but the entry namespace also needs to know what a :project/id is.
2017:10:19 14:01:22 alexmiller the qualifiers used in spec names do not have to correlate to an actual namespace nor do they have to be defined in that namespace. The word “namespace” is dangerously overloaded in this area.
2017:10:19 14:09:31 alexmiller I think it’s most useful to declare stuff like this in some totally different “domain data” namespace
2017:10:19 15:36:30 jrychter Interesting. But if they don't correlate to namespaces, I lose the (rather convenient) :: syntax. I guess the namespaced map reader syntax could make up for that.
2017:10:19 15:45:40 bbrinck schec lets you create aliases to non-existing namespaces, and then you could use the alias to shorten writing specs
2017:10:19 17:32:19 alexmiller support for handling keyword aliasing in better ways (without doing that ^^) is something Rich is working on for future release btw
2017:10:19 07:53:20 jrychter Also, I'm not sure about the new reader syntax for namespaced maps, specifically how it should work with destructuring:
(let [{a :some.ns/a} {:some.ns/a 1}] a) => 1
(let [#:some.ns{:keys [a]} {:some.ns/a 1}] a) => 1
(let [#:some.ns{a :a} {:some.ns/a 1}] a) => error
Is map destructuring with key renaming not supported?
2017:10:19 11:15:02 gfredericks a problem with that last one is that the binding form is read in as {some.ns/a :a} which is sort of backwards from what you want, because the symbol gets namespaced but not the key
2017:10:19 12:29:52 jrychter @gfredericks Right. But what now, do we give up key renaming? Is that expected/intended?
2017:10:19 12:32:25 gfredericks you can do it the long way, at worst
2017:10:19 12:34:17 gfredericks I have no idea whether there was any intention to make something like this work
2017:10:19 12:36:32 jrychter Hmm. In that case, the push to use namespaced keys everywhere seems slightly premature.
2017:10:19 13:57:53 alexmiller that’s not valid syntax
2017:10:19 13:58:29 alexmiller the “left” part of destructuring is always the unqualified name of the local being bound
2017:10:19 13:59:14 alexmiller so you want: (let [{a :some.ns/a} {:some.ns/a 1}] a)
2017:10:19 13:59:33 alexmiller (this has worked since Clojure 1.6 I believe)
2017:10:19 14:11:35 jrychter @alexmiller I am moving to ns-qualified keys everywhere and have many places where I do {:keys [a b c d e f]}. I was hoping I could do #:some.ns{:keys [a b c d e f]} and this works well, but what if I additionally want to extract {renamed-g :some.ns/g}?
2017:10:19 14:11:59 alexmiller you can do both
2017:10:19 14:12:25 alexmiller {:some.ns/keys [a b c d e f], renamed-g :some.ns/g}
2017:10:19 14:13:09 alexmiller I think using the #:some.ns prefix is kind of confusing in this context - I usually put it inside the map
2017:10:19 15:31:04 jrychter @alexmiller Oh, so that's the syntax! Thank you, I've never seen it until now.
2017:10:19 15:49:47 alexmiller Means the same thing - just alternate syntax
2017:10:19 15:50:31 alexmiller You can do multiple keys destructuring from different namespaces too
2017:10:19 16:44:37 jrychter Thank you for your time and help!
2017:10:19 18:58:58 rads can anyone explain to me why spec doesn't check :ret specs during instrumentation by default? I know there was a discussion about it a while back, but I'm curious if anyone has a succinct explanation for the decision
2017:10:19 19:09:11 bbrinck IIRC, the rationale is that instrumentation is for checking that you called function X correctly, not for checking that the function X implemented correctly
2017:10:19 19:09:40 bbrinck The recommendation is to use generative testing to confirm that function X works correctly (based on the :ret and :fn specs)
2017:10:19 19:10:48 bbrinck I personally disagree with that distinction, since I have functions that can’t be realistically verified with generative testing due to a) side effects or b) disproportional work to get the specs perfectly tuned
2017:10:19 19:12:21 bbrinck In my limited experience, instead of using check to test function X, we ended up just not speccing :ret or :fn because they were unverified and I preferred no documentation rather than documentation that was potentially incorrect or out of date
2017:10:19 19:16:49 bbrinck In particular, using orchestra to instrument functions during non-generative testing gives me confidence that my :ret and :fn functions are correct, which makes them more valuable as documentation
2017:10:19 19:19:04 rads thanks, that's very helpful
2017:10:19 21:50:31 eriktjacobsen @alexmiller Is there any plan to allow multiple, or nested namespaces in spec? Example: A spec ::bar/type declared in the foo.core namspace that could be referenced as :foo.core/bar/type? We are currently struggling with a seeming limitation of spec, and while the underlying problems deserves a longer form post, before heading down that path, was just curious if this was something being thought about.
2017:10:19 22:03:39 seancorfield A spec of ::bar/type would need a namespace alias of bar available in order to be legal. That could alias any namespace @eriktjacobsen but could be foo.core.bar if you wanted it to be. Then ::bar/type would be :foo.core.bar/type.
2017:10:19 22:04:31 seancorfield Since namespaces are (supposed to be) globally unique and can be arbitrarily "nested" already, I'm not sure what you're asking for...
2017:10:19 22:16:02 eriktjacobsen Thank you Sean. I realize what the current system provides, and how to work around it. I was mostly curious if that specific solution has been talked about... will write up an explanation in a bit.
2017:10:19 22:21:06 alexmiller There is no plan (or need) for any kind of nesting
2017:10:19 22:21:51 eriktjacobsen Thanks for answer. put another way, has there been any thought to allowing a component of a spec identifier that isn't tied to a namespace or part of the unqualified key
2017:10:19 22:22:34 alexmiller No, why?
2017:10:19 22:23:48 alexmiller That is, please help me understand the problem you're trying to solve
2017:10:19 22:23:51 eriktjacobsen writing the why up now, mostly due to tradeoffs in workflow my team is seeing. Perhaps it just needs a better framework for how we're structuring things, but it could also be solved by what I mentioned.
2017:10:19 22:25:12 alexmiller I think sometimes it's also unclear that keyword qualifiers can refer to Clojure namespaces but don't have to have any relation to them
2017:10:19 22:25:40 alexmiller There is an unfortunate overload and common use for the word "namespace"
2017:10:19 22:27:27 alexmiller Currently namespace aliases also have the unfortunate downside of being tied to the particular case where a qualifier is actually a namespace (I do not expect this to be as constrained in the future)
2017:10:20 00:03:47 eriktjacobsen @alexmiller Thanks for the quick answers. Explained: https://redd.it/77if23 If the answer is "you need a separate namespace per data type with colliding keywords", so be it.... just wanted to have the discussion.
2017:10:20 00:42:36 seancorfield @eriktjacobsen Yes, the answer is to provide unique namespace-qualifiers for each of the colliding keywords -- remember that those namespaces do not have to exist, but they should have names that represent the different concepts being modeled.
2017:10:20 00:47:22 seancorfield The whole point of namespaces is to provide unique ways to name things that are distinct (but otherwise have the same unqualified name).
2017:10:20 01:01:53 eriktjacobsen Right, it's just unfortunate to lose the relationship of between using the spec and knowing where it's defined, losing ability to use the :: and aliases (until patched), and other limitations mentioned. I understand that might be the answer, its just my team is unhappy if that's the final answer.
2017:10:20 01:07:35 gfredericks :foo.bar/baz.hullabaloo is a valid keyword I think
2017:10:20 01:07:56 gfredericks I don't suppose it's likely to make anybody happy though
2017:10:20 01:09:42 eriktjacobsen Correct, but when used in a (spec.keys :req-un) it will require the keyword to be :baz.hullabaloo, correct? Rather than :hullabaloo. That is basically the same solution as my first proposal, just with . instead of /
2017:10:20 01:11:15 gfredericks yep. it's just sadness all over
2017:10:20 01:16:08 eriktjacobsen it does seem like differentiation between specspace and namespace seem slightly confused, or dare I say complected.
2017:10:20 01:26:02 gfredericks OTOH it would be weird and confusing to have many parallel namespace systems
e.g. those lisps that keep functions separate from the other kind of thing
2017:10:20 01:10:24 potetm Why use :req-un for a namespaced key?
2017:10:20 01:13:53 eriktjacobsen Because the incoming data I'm trying to spec is generally using unqualified keys. Such as from parsing json.... I suppose I could add a step before doing spec validation that transforms all the unqualified keys into qualified keys using some mapping scheme, validates with spec, then unqualifies them again for usage downstream, but that's basically the functionality i'm looking for from spec and seems like a lot of overhead
2017:10:20 01:33:13 potetm Ah, okay. It sounded like you had control over the keyword. That makes sense though.
2017:10:20 07:03:28 jrychter @eriktjacobsen FWIW, I am struggling with similar issues. I tried (briefly) using spec keywords in "fictional" namespaces (e.g. not corresponding to clojure ns), but then I end up with an explosion of long namespaced keywords. Lots of typing, lots of things to read and process when looking at code. The :: really helps in reducing that.
2017:10:20 07:05:57 jrychter Somewhat related: I'm also thinking about dropping :req-un and moving to fully-qualified keys everywhere. But there is a price to be paid in storage (JSON database), transmission (websocket trafic isn't compressed, Transit helps but I'm not sure to what extent), code complexity (need to include keyword namespaces in all destructuring code), and ClojureScript code building keywords from strings. I'm not sure what that price is.
2017:10:20 08:05:48 mpenet if you throw in some utils it's not so bad
2017:10:20 08:07:02 mpenet i use a macro that creates "relative" aliased ns and tend to spec maps using it following a pattern like ::map ::map/foo ::map/bar ::map/baz etc
2017:10:20 08:10:15 mpenet you can just do (rel-ns 'foo.bar.baz.bad) if deeply nested and then use it as ::foo.bar.baz.bad/xxx, it's quite readable imo
2017:10:20 18:06:18 eriktjacobsen Thank you! We have thought of similar workarounds, mostly wanted to bring the discussion up for a solution from the core.specs team, though we could incorporate this if we have to workaround it.
2017:10:20 16:45:39 metametadata Hi! Given ::allowed-val "enum spec", is there a way to print all the allowed values in the error message? Currently it prints (into #{} allowed-vals).
And let's consider it's the requirement that I need allowed-vals extracted as a var because I need to iterate over it in other places.
2017:10:20 17:49:59 metametadata yeah, it's a bit backwards but should work, thanks
2017:10:20 20:04:21 ajs There is no forwards or backwards with a set, right? Unordered.
2017:10:21 00:03:35 metametadata right 🙂 I meant that it's backwards in a sense that I'd like the variable to be the source of allowed values instead of putting them into spec
2017:10:20 18:13:17 metametadata and what if the enum values cannot be hardcoded like that in spec? e.g. when I need to read them from file
2017:10:20 18:46:10 ghadi right now calling eval to build specs is probably the best choice, but work is apparently underway on making spec more "programmable" -- make specs from external stuff
2017:10:20 20:04:35 bsima how would I spec a function with variadic keyword args like (myfn :a 1 :b 2)? I’m trying to use (spec/* (spec/keys :req-un [::a ::b])) but that’s not right
2017:10:21 00:11:55 falak So how do I test the dispatch-value of the dispatch-fn in the defmulti definition?
2017:10:21 02:55:22 James Vickers I'm trying to make a spec for a function that takes in a map with un-qualified keys (I know how to do this with qualified keywords, just use s/keys). The function arguments look like this: [{:keys [interest term balance] :as m}], and for that I make a function spec with (s/cat :m (s/keys :req-un [::interest ::term ::balance])) - I have specs named ::interest,`::term`, ::balance. Is there a more direct way to make a s/fdef spec for :args? I don't use the whole map (`:m`) in the function body and only added that part to use in the spec. I tried the example in the clojure.spec documentation for s/keys with unqualified keywords but couldn't get it to work. Alternatively, am I just way off by using un-qualified keywords as keys in the map the function takes?
2017:10:21 02:55:22 James Vickers I'm trying to make a spec for a function that takes in a map with un-qualified keys (I know how to do this with qualified keywords, just use s/keys). The function arguments look like this: [{:keys [interest term balance] :as m}], and for that I make a function spec with (s/cat :m (s/keys :req-un [::interest ::term ::balance])) - I have specs named ::interest,`::term`, ::balance. Is there a more direct way to make a s/fdef spec for :args? I don't use the whole map (`:m`) in the function body and only added that part to use in the spec. I tried the example in the clojure.spec documentation for s/keys with unqualified keywords but couldn't get it to work. Alternatively, am I just way off by using un-qualified keywords as keys in the map the function takes?
2017:10:21 02:57:40 taylor the :as m in the function argslist shouldn’t matter to spec, you should be able to remove it if you’re not using it
2017:10:21 03:01:17 James Vickers Thanks for answering so fast. So s/cat needs key/pred argument pairs, so if I remove :m from the definition of the function spec, what would I put instead?
2017:10:21 03:07:08 taylor the :m in your function spec doesn’t have any relation to the :as m in your arglist, it could be named anything really
2017:10:21 03:07:27 taylor s/cat requires each “element” be tagged with some keyword though, so you can’t omit it. The actual keyword name doesn’t matter so much unless you’re interested in the conformed version of it i.e. it could be (s/cat :tgif ::map-spec) and it’d still work
2017:10:21 03:07:57 taylor it looks like you’re doing it right
2017:10:21 03:10:54 taylor if it doesn’t work when you remove :as m from the fn arglist, something else must be wrong
2017:10:21 03:11:17 James Vickers You are totally correct - took out :as m in the function args list, still works. Can change :m to any keyword (e.g. :foo) in the :args spec and that works too. But that's so weird! In this usage, I guess spec doesn't use the keyword arg for this spec?
2017:10:21 03:13:00 James Vickers That's weird. I wonder if there'll be other functions coming in spec like s/cat that don't have that setup, it seems weird that for this case (which seems common?), it doesn't matter what keyword you put there - just a placeholder. Thanks!
2017:10:21 03:14:46 taylor s/cat does use the keyword to tag conformed outputs, it just doesn’t matter for your use case
example from spec guide, notice the output has :opt and :val “tags”
2017:10:21 03:23:09 James Vickers I guess the reason I didn't understand is I don't quite get s/cat and the other regex ops in spec.
2017:10:21 04:57:25 seancorfield @jamesvickers19515 So your function has one arg, a hash map? (s/cat :m ::account) perhaps, with (s/def ::account (s/keys :req-un [::interest ::term ::balance]))
2017:10:21 05:10:05 James Vickers Thanks @seancorfield. @taylor showed me that when using s/cat in this instance, the first argument actually didn't matter - could be :foo for all it mattered.
2017:10:21 05:10:50 seancorfield Right, s/cat takes a sequence of (whatever) argument names and specs.
2017:10:21 05:11:17 seancorfield It's convention to use the same (keyword) name for each argument as the function but there's no reason to.
2017:10:21 05:11:43 seancorfield In your case, you have a destructuring as the first (only) argument so its name is somewhat arbitrary anyway.
2017:10:21 05:12:58 James Vickers I guess what was surprising was that there wasn't a spec function for this case that doesn't require the unused keyword as the first arg - I was under the impression that functions that take a single map were common in Clojure.
2017:10:21 05:13:38 seancorfield Yes... not sure what you're asking...
2017:10:21 05:13:55 James Vickers Sorry, wasn't a question 🙂 just a comment
2017:10:21 05:14:06 seancorfield (s/cat ...) is how you specify an argument list.
2017:10:21 05:18:11 seancorfield Specs name things, as part of their conformance. The names don't have to correspond to anything in the source code (`s/or` is a good example, s/cat is similar).
2017:10:21 05:23:01 James Vickers I think I sort of see now. I did something like this at the REPL:
2017:10:21 05:27:20 seancorfield Yeah, argument lists are sequences. s/cat matches a sequence of (named) specs. So, in this case :m is the name and the s/keys is the spec for it.
2017:10:21 05:27:27 James Vickers And this call to valid? with a vector that looks like the function signature:
2017:10:23 19:46:07 alexmiller Well first, nothing actually wrong with this example. But I think I would prefer (fn [] id-gen) now.
2017:10:23 00:11:57 drewverlee ugh, i shouldn't ping ppl at 8pm on sunday. got excited.
2017:10:23 02:51:34 seancorfield @drewverlee It's OK to be excited at weekends 🙂 Just needs to be tempered with a little patience until most folks get to their desks on Monday morning 🙂
2017:10:23 15:44:20 mmer Issues with : Call to clojure.core/let did not conform to spec:
In: [0] val: () fails spec: :clojure.core.specs.alpha/bindings at: [:args :bindings :init-expr] predicate: any?, Insufficient input - Should I be using clojure.spec or clojure.spec.alpha in my namespace?
2017:10:23 15:45:33 taylor if you’re using the latest version, the namespace is clojure.spec.alpha
2017:10:23 15:46:34 mmer Thanks - but ...why am I seeing this error?
2017:10:23 15:47:35 taylor can you post a code snippet of your let?
2017:10:23 15:51:53 mmer Thanks Taylor - I had been getting errors all the time just when I added the clojure.alpha module to my project that I assumed the spec error was in spec not my code! But of course it was my code!
2017:10:23 17:53:44 hiredman I am using spec as a parser, and have just noticed that when I parse a seq with 40 entries (all already realized in memory) I get out of memory errors, any ideas how to avoid this?
2017:10:23 17:55:27 bfabry seems weird, lots of alternates?
2017:10:23 17:57:32 hiredman 2 alternates with 2 and 4 cases
2017:10:23 17:58:28 hiredman I haven't played with changing how the spec is structured yet to see if that improves things
2017:10:23 17:59:11 hiredman I do use s/& a few times, which I can imagine being expensive for backtracking
2017:10:23 18:48:09 ghadi @hiredman any way you can file an issue?
2017:10:23 18:49:50 hiredman maybe
2017:10:23 19:43:52 alexmiller don’t know that I can help without a repro case or more info
2017:10:23 22:12:53 drewverlee Any idea how you can get the keys from a spec?
2017:10:23 22:14:07 bfabry @drewverlee there's a few different options. s/form will give you the spec definition as data that you could walk
2017:10:23 22:32:08 alexmiller Spec form specs will eventually make this easier to answer (see clj-2112)
2017:10:23 22:53:14 hiredman while I was fiddling with coming up with a minimal test case, I realized I could change the format slightly to have an explicit delimiter instead of an (s/& ...) check, which seems to have solved my performance issues
2017:10:24 08:52:51 msolli Where is a good place to put (test/instrument) so that all my spec’ed functions are instrumented in dev? I have a fairly standard Luminus-based webapp.
2017:10:24 11:55:39 ikitommi spec parsing is the new black 😉
2017:10:24 11:58:22 stathissideris @ikitommi do you think we could use spec to parse spec forms or would reality collapse under the weight of self-reference? 😄
2017:10:24 12:01:50 stathissideris way ahead of us 🙂
2017:10:24 12:07:24 ikitommi but even when the spec-of-specs ships, we still need to utilities to do something with the parsed data, e.g. collect keys out of (s/merge ::a-map ::another-map), spec-tools is one bin collecting this kind of stuff. And there are many others.
2017:10:24 14:29:14 taylor I’m curious about this too
2017:10:24 14:43:06 andre.stylianos so am I
2017:10:24 15:17:58 jeaye I'm just using re-frame, not Luminus, but it's in my-app.core behind a macro.
2017:10:24 15:24:22 jeaye That'll do the trick, allowing you to specify bits in your leiningen profiles and then conditionally bring in code by reading the profile using environ in a macro.
2017:10:24 15:24:46 taylor thanks, the only caveat is that any namespaces you want to instrument must have already been loaded before that’s called, right?
2017:10:24 15:24:55 jeaye That's right.
2017:10:24 15:25:09 jeaye I don't think you can get around that though.
2017:10:24 15:26:04 jeaye Our my-app.core requires every model ns we have. Our my-app.core-views requires every view ns we have, as well as requiring my-app.core (so view code isn't needed for unit testing).
2017:10:24 15:26:07 taylor I wouldn’t imagine so, was just curious what other people’s approaches to that might be. I was thinking another approach would be to just put the (conditional) instrument calls at the bottom of each spec’d namespace
2017:10:24 15:26:49 jeaye Hm, that would be every single ns for us, which seems less manageable. Sounds like it would work though, if that's your preference.
2017:10:24 18:00:40 msolli Thanks, @U4986ECDQ, seems like a reasonable solution. Those are some handy macros! 👍
2017:10:24 15:55:11 tony.kay I have a project where I use tools-ns to reload code. If I don’t carefully set refresh dirs then suddenly defn will start causing spec failures…as in defn- doesn’t conform to spec. Is this a known issue? Narrowing refresh dirs seems to fix it.
2017:10:25 12:53:52 alexmiller I haven’t heard of that before
2017:10:25 13:22:12 robert-stuttaford is the right way to spec a map that must have at least one of several keys (s/or (s/keys :req []) (s/keys :req []) (s/keys :req [])) ?
2017:10:25 13:22:37 taylor you can use a single keys spec with or inside of the :req vector
2017:10:25 13:23:33 mpenet sounds ok, but if they are sharing the same context and have something like a :type key it might make sense to make a multi-spec out of it
2017:10:25 13:24:14 robert-stuttaford it truly is a situation where you can provide 1, 2 or 3 of the keys, but providing 0 of them makes the whole map redundant
2017:10:25 13:25:55 robert-stuttaford modelling a rule system where there are 3 possible outcomes - no-value-yet, yes, or no. you have to model at least one outcome, or there’s no point to testing for the rule
2017:10:25 13:26:32 taylor those three states should be mutually exclusive?
2017:10:25 13:27:02 robert-stuttaford the rule can be reused for multiple cases in a doc
2017:10:25 13:27:10 robert-stuttaford some may 0, some may 1, some may 2
2017:10:25 13:33:00 robert-stuttaford @taylor you were right!
2017:10:25 13:33:31 robert-stuttaford (s/keys :req [(or :a :b :c)]), plus the fact that all ks are validated gives me what i need
2017:10:25 14:17:21 ajs looking at this example from @cemerick do i read correctly that specs default to passing valid? even if the namespaced keyword was never def'd? https://twitter.com/cemerick/status/875748591310168065
2017:10:25 14:19:06 taylor I’ve only seen this behavior w/`key` specs, but yeah it doesn’t require that the keyword have a registered spec
2017:10:25 14:19:24 mgrbyte nope. (require '[clojure.spec.alpha :as s]) (s/valid? ::some-undefined-spec [}) will throw an exception (unable to resolve some-undefined-spec)
2017:10:25 14:20:42 mgrbyte even when ::some-spec has not been defined yet
2017:10:25 14:21:14 ajs hmm, i wonder what the reasoning is for allowing that keys
2017:10:25 14:21:29 ajs that's doesn't make any sense to me
2017:10:25 14:21:46 taylor maybe so that you can easily spec required map keys without writing specs for the keys’ values, or having to define the key specs before the keys spec
2017:10:25 14:22:21 taylor otherwise you’d have to do (s/def ::foo any?) for every key before using it in a keys spec?
2017:10:25 14:22:36 ajs lot of room for user error though, as @cemerick has noted. as i am very prone to typos (perhaps we all are), i can see this biting me
2017:10:25 14:23:16 taylor it’s true, and I’ve actually seen some code in the last week that checks map spec keys for “typos” but I can’t remember where 🙂
2017:10:25 14:23:35 ajs i'm currently evaluating a variety of validation libraries like funcool/struct and truss and others. the one that most ably catches my typos is the one i will use.
;; The example program below lets you identify "missing" keys specs at
;; the time and place of your choosing, and then handle them as you
;; deem appropriate, without imposing those decisions on other
;; users of spec.
2017:10:25 14:25:30 ajs i find it a little bizarre that you have to jump through those hoops and write that code just to have spec do what you are expecting; perhaps that leaves room for a little library on top of spec to help mitigate disaster
2017:10:25 14:28:46 ajs i don't understand why specifying the presence of a key uses the same syntax/naming as specifying the presence of an actual spec/predicate -- those are two different things, but they are expressed the same in spec, hence the room for easy errors
2017:10:25 15:01:31 bbrinck It seem like most of my typos are around keywords. It seems like a tool that would macroexpand code and look for keywords that are unique would catch a large number of my typos.
Sets (maps) are about membership, that’s it
As per above, maps defining the details of the values at their keys is a fundamental complecting of concerns that will not be supported. Map specs detail required/optional keys (i.e. set membership things) and keyword/attr/value semantics are independent. Map checking is two-phase, required key presence then key/value conformance. The latter can be done even when the (namespace-qualified) keys present at runtime are not in the map spec. This is vital for composition and dynamicity.
2017:10:25 15:14:44 andre.stylianos From what I understand there are two different things.
-- Membership:
"This map has (required or optional) such and such keys"
(s/def ::map (s/keys :req [::foo]))
-- Value:
"Anything with this key should conform to this value"
(s/def ::foo string?)
2017:10:25 15:16:09 andre.stylianos That's the way I understood this at least
2017:10:25 16:55:30 ajs Perhaps one could consider spec a lower level tool on top of which you could build an API to better express concrete structural requirements that eliminate some of that ambiguity, and use that instead of spec directly.
2017:10:25 16:56:26 ajs Otherwise there seems ample room for user error, which is typically what you're trying to avoid in the first place by using a validation library
2017:10:25 16:38:05 mmer Apart from expound is there a way to associate an error message with spec definition so you can force an error message that makes sense and also allows for globalisation of error messages?
2017:10:25 16:47:11 bbrinck Is this for devs or end users?
2017:10:25 16:47:44 bbrinck @mmer If the messages are for end users, you probably don’t want Expound. https://github.com/alexanderkiel/phrase is likely a better fit
2017:10:25 16:54:35 mmer Would it not be reasonable to include some method of defining the error message directly alongside the definition?
2017:10:25 17:16:53 bbrinck How would you want to use those errors messages? In explain?
2017:10:25 17:21:29 bbrinck or would you want to show the error messages to end users?
2017:10:25 17:25:51 bbrinck My understanding is that specs are getting metadata, so if you just need a place to add some info and you want to later pull that out, you could potentially use metadata? https://dev.clojure.org/jira/browse/CLJ-2194
2017:10:25 18:46:51 mmer I guess I need to get a usable error message for end users. My usecase revolves around validating yaml files edited by people. As these are nested structures the explanation from explain? get complex. I was hoping for a simple way to add a message to def that explain in human terms what is required. In my case the errors from spec are about a format the user does not see. Why use spec? I have found no other suitable tool that allows me to define in effect a schema for Yaml.
2017:10:25 18:51:09 bbrinck OK, that makes sense.
2017:10:25 18:52:50 bbrinck So, you could assign an error message to each spec. You could then call explain-data and, for each problem, convert the failing spec into the error message.
2017:10:25 18:53:38 bbrinck That may be sufficient for your use case. However, remember that problems are flat - if you have a “or” spec, you’ll get two different problems for the same value
2017:10:25 18:54:39 bbrinck Additionally, it’s not trivial in my experience to show where the non-conforming is located within the larger data structure
2017:10:25 18:55:32 bbrinck Spec also will print errors in terms of the Clojure data structure, which may not be useful if your users are expecting to see errors in terms of Yaml
2017:10:25 19:03:09 bbrinck It really depends on your spec, but probably the simplest approach is to assign errors to each spec (you could just build a static map for now, since each spec name is a unique keyword), then call explain-data, then take the first problem for each in, then print out the in path + your custom string error (you can find the relevant spec in via). That might be sufficient, although possibly misleading if you have a lot of “or” specs.
2017:10:25 19:39:41 stathissideris \|||||||||
2017:10:25 22:31:20 Brendan van der Es Anyone know a better way to conform an or spec without keywords? This is the best I got:
2017:10:25 22:33:58 mmer @bbrinck Thank you sir for a long and consider answer - I like the map of spec errors approach as even with an or you can explain the options. . Much appreciated.
2017:10:25 22:41:35 mmer A follow up from my previous question - is it posible to get the set of specs that are currently in registered?
2017:10:25 22:42:47 hiredman there is a registry function that returns the registry (a map of names to specs)
2017:10:25 22:51:33 Brendan van der Es @hiredmanThanks. I am trying to s/or arbitrary specs. Essentially I'm trying to confom different argument lists, e.g.[& args], to the same format of input.
2017:10:25 22:53:56 Brendan van der Es Specifically for datomic, [db entity-hash-map] conforms to EntityMap & EntityMap conforms to EntityMap. Maybe this approach is flawed anyways 😛, I'll keep playing around with it. Thanks
2017:10:25 22:54:20 hiredman I would use spec to recognize (tag) different formats so you know which formats to deal with, not to try and smoosh all the formats in to one
2017:10:25 23:06:35 Brendan van der Es Makes sense, you do than have to handle the different inputs in your function, which arguably is the better place for this logic anyways. I'm hoping this logic might be more reusable if I can embed the fact that the different arg-list formats essentially carry the same information in the spec.
2017:10:26 00:39:52 bbrinck @b.j.vanderes FWIW, my understanding is that conformers are not intended to be used to change the shape of the data, so you may run into problems here. Perhaps it’d be easiest to just conform as normal, then write a transformation function that uses the conformed values and converts them
2017:10:26 01:59:58 Brendan van der Es @bbrinck @hiredman Yea, this is all probably too counter to intention to be usable (performance might also be an issue). But just for fun this is what I came up with as far as defining functions that process the conformed arguments:
2017:10:26 02:04:55 Brendan van der Es Can combine with above to mitigate an outer layer of branching in the function.
2017:10:26 08:23:00 tap I get OutOfMemoryError when calling instrumented function with wrong number of arguments where one of the argument is a huge collection. I know it’s considered user mistake but it’s a kind of mistake that is quite hard to figure out the cause. Should clojure prevent that, or break circuit or something?
2017:10:26 12:49:33 alexmiller Would help to know more about the spec and understand why its generating huge collections
2017:10:26 12:49:49 alexmiller There are some known issues in this area
2017:10:27 02:16:25 tap Sure. I’ll try to come up with a sample project replicating the issue I found this weekend
2017:10:26 11:01:07 danielneal What is the best way of getting the specs of arguments to an fdef'd function?
2017:10:26 11:01:11 danielneal I've got an approach here
2017:10:26 11:01:37 danielneal but it uses eval and is probably all wrong 🙃
2017:10:26 12:44:18 alexmiller Yeah, don’t do that :)
2017:10:26 12:45:55 alexmiller Just (-> sym s/get-spec :args)
2017:10:26 12:46:47 alexmiller Or do you want the spec of each argument separately?
2017:10:26 12:48:01 alexmiller In the most general sense (considering multi arity and variadic), I think that is difficult
2017:10:26 12:52:15 danielneal yeah - the spec of each argument separately
2017:10:26 12:53:11 danielneal thanks for the get-spec tip 🙂
2017:10:26 12:55:45 danielneal just doing a very rough and ready thing to see if I can go from a value to the (specced) functions that you can invoke that take that value as an arg
2017:10:26 12:55:51 danielneal like intellisense, but stupid
2017:10:26 12:55:57 danielneal stupidisense, if you will
2017:10:26 14:04:30 alexmiller this is kind of what’s happening inside of spec regex validation checking, but it’s really happening left to right and every possibility is being evaluated and narrowed based on the prior matches (of which there may be multiple)
2017:10:26 14:05:11 alexmiller so a priori before starting the parse there is not necessarily a single spec but really a set of specs that may apply at each point
2017:10:26 14:05:58 alexmiller that said, there is the very common case that the args spec is a single non-variadic arity and each arg matches up 1-for-1 with an s/cat args spec
2017:10:26 14:06:22 alexmiller in which case, you can just pull the s/cat spec apart to match it up
2017:10:26 14:07:05 danielneal mm yes I follow you. Technically the args spec for fns could be any regex (or anything) but in practice there will be a common case like you described
2017:10:26 14:10:42 danielneal this is what I've got so far - https://gist.github.com/danielneal/d82c142c9eab9f8caec0fa93b87ff7f3 - I'd like to see if I can get it completing in emacs to see if it 'feels' useful. Dumb but proof of concept, like when IDEO demonstrated the talking kitchen with people hidden behind the fridge saying stuff and responding to voice commands.
2017:10:26 19:17:43 zclj Sometimes I get errors such as:
`java.lang.IllegalStateException: Attempting to call unbound fn: #'clojure.test.check.generators/choose, compiling:(generators.cljc:499:3)
Exception in thread "main" java.lang.IllegalStateException: Attempting to call unbound fn: #'clojure.test.check.generators/choose, compiling:(generators.cljc:499:3)`
It happens with different test.check unbound fns, it is not the same every time and a rebuild usually resolves the error. The file that is beeing compiled is a quite basic cljc file with some specs in it.
Does anyone have any advice to point me in the right direction?
2017:10:26 21:49:55 alexmiller Any chance this is happening during check?
2017:10:26 21:51:20 alexmiller There is a dynamic load race that can happen with loading the test check generator namespace that we have a ticket on. I don’t remember /cdn-cgi/l/email-protection presents, but it was weird looking
2017:10:30 12:03:35 zclj @U064X3EF3 Sorry for the late reply. The stack trace only show line 1 for my files i.e. in the ns loading. The file do include a stest/check test but from the stack trace I do not seem to hit that. It blow up on the first line. I'll try and find the ticket you mentioned and see if that holds any clues
2017:10:26 19:26:08 gfredericks O_O
2017:10:26 19:35:57 bill in an s/fdef :fn spec for a tree function of mine, it seems like I need to get access to the un-conformed argument (a tree) to the function so I can pass it to some other tree functions to build up my validation.
is there a way to gain access to the un-conformed args or perhaps to un-conform the conformed args?
2017:10:26 19:44:31 bill is the right word for “un-conform” “unform”?
2017:10:27 08:53:50 mmer Is there a way to associate a string of text with a spec definition. I would like to annotate my definitions
2017:10:27 09:17:22 gklijs You can do something like (def label (s/and (s/spec string?) #(> (count %) 3) #(< (count %) 40))) and then use hat spec somewhere else like (s/def ::nl-label label)
2017:10:27 09:45:42 danielneal I think doc-strings for specs are coming later, too
2017:10:28 19:19:46 mike706574 Is it intentional that the functions in clojure.spec.gen that alias functions in clojure.test.check.generators aren't considered generators?
2017:10:28 19:19:53 mike706574 Or am I doing something wrong?
2017:10:28 19:20:32 mike706574 (clojure.test.check.generators/generator? clojure.spec.gen.alpha/int) seems to be returning false
2017:10:28 19:22:10 mike706574 In my case, I'm trying to write a spec for an 5-10 character alphabetic string with a
custom generator that uses clojure.spec.gen.alpha/char-alpha
2017:10:28 19:23:00 mike706574 Sampling is failing with an AssertionError with a message of Assert failed: First arg to vector must be a generator (generator? generator)
2017:10:28 19:24:23 mike706574 It works fine when I use clojure.test.check.generators/char-alpha - just seems counterintuitive that the aliased function doesn't work
2017:10:28 19:40:58 hiredman the spec aliases are actually single argument functions that return generators
2017:10:28 19:41:20 mike706574 oh
2017:10:28 19:41:26 mike706574 that makes more sense
2017:10:28 19:41:40 hiredman this is to support lazy loading of clojure.test.check
2017:10:28 19:41:57 mike706574 so i shouldn't be using them?
2017:10:28 19:42:31 hiredman you can use them, you just have to be aware of that fact when mixing with plain test.check generators
2017:10:28 19:43:21 hiredman and if I recall correctly, most places in clojure.spec where you can supply a custom generator, you supply a no arg function that returns a test.check generator
2017:10:28 19:43:42 mike706574 yup
2017:10:28 19:44:35 mike706574 i just put parens around it - (gen/vector gen/char-alpha 5 10) to (gen/vector (gen/char-alpha) 5 10) - and it worked
2017:10:28 19:44:59 hiredman that's lisp for you
2017:10:28 19:45:11 mike706574 cool, thanks
2017:10:29 10:17:22 olivergeorge Quick sanity check. I think this code should stub a function (in second case). It does on CLJ but doesn't on CLJS.
2017:10:29 10:18:00 olivergeorge Am I doing something obviously wrong?
2017:10:29 15:24:29 eoliphant Is it possible to do ‘cross’ specification/validation with spec?
If I have a map
{:a "even"
:b 2}
Where say :b should be odd? if :a is "odd" and vice versa?
2017:10:29 15:25:54 mingp @eoliphant Perhaps treat them as separate cases and combine with spec/or.
2017:10:29 15:26:40 mingp It would depend more on what the more immediate problem you're trying to solve is.
2017:10:29 15:26:51 eoliphant hmm ah I see, when you’re doing the ‘map’ spec cover both cases with the or
2017:10:29 15:26:58 eoliphant i’ll give that a try
2017:10:29 16:08:03 mrchance Hi, is there a way to generate self contained specs? I want to turn a swagger.json into a spec and run it on some data at runtime, and it looks like that requires eval with the way the spec registry currently works?
2017:10:30 16:26:06 alexmiller This is a known issue in the doc build process
2017:10:30 16:41:36 mrchance Ok, just wanted to let you know. Any insights into the other problem? I feel pretty dirty if I have to fiddle with global mutable state just to get some spec validation working...
2017:10:29 22:17:17 drewverlee If i want to use spec in order to pass the user of my application a error msg how would i do that? Is there a place to override the error msg with a custom one?
2017:10:29 22:18:13 drewverlee Does that even fit with the rational? Like if i have a command line app and i want to validate the input and send back an error msg, would spec fit into?
2017:10:29 23:03:25 seancorfield The idea is to use the explain-data result and map it to whatever error messages you want in your app @drewverlee
2017:10:29 23:16:39 bbrinck But yes, if your specs are fairly simple, it might be easies to just map explain-data to error messages. It gets a little trickier if your specs are more deeply nested
2017:10:29 23:24:58 drewverlee @bbrinck phrase is in the spirit of what i want. Its odd i couldn't search it down.
2017:10:30 13:26:58 drewverlee I poked around with the idea of phrase and i’m not sure it make sense for what i’m doing. As @seancorfield suggested i can pull the information with explain-data.
And if i want to reference something inside the spec it might make sense to pull it into a var to use across multiple domains rather then pulling it out via regex matching (which is what phrase seems to do).
I’m would be interested to hear how something like phrase worked in a larger project, as it seems prescriptive.
2017:10:30 14:36:30 mpenet did anyone create a web based spec browser thing?
2017:10:30 14:39:44 bbrinck @mpenet What’s your use case? Do you want to browse specs at dev time, or, say, publish specs online similar to API docs?
2017:10:30 14:41:02 mpenet I guess this could be adapted, but just asking in case somebody did that work already
2017:10:30 14:42:47 bbrinck Hm, not that I know of. On a related note, I wonder if it’s be useful to start a wiki that collects these spec helper libraries as a resource
2017:10:30 14:50:24 jebberjeb @drewverlee I’ve been working on something like this which tries to map a human readable message by a problem’s spec keyword, and falls back to a symbolic predicate match (like Phrase) if it can’t. Trying to embloginate it now.
2017:10:30 15:00:03 drewverlee I was thinking this over last night. I don’t think there is any generic relationship between a human readable message and the code.
The only way i could see to do this that was better then plain old clojure would be some sort of semantic analysis of the function words. I feel like a neural net might be able to make something out of the function names and values.
I was curious about phrase, but felt it wasn’t doing much for me that i couldn’t get from core spec and a mapping to human readable message.
I’m curious about your work.
2017:10:30 15:06:41 jebberjeb I suspect its there, but I’m not even close to finding it. My work is mostly a straight dispatch based on the problem’s spec keyword, falling back to a non-macro based Phrase implementation if that fails. And only the latter out of necessity. s/keys specs for example, generate problems that really require you to look at the predicate to know exactly what went wrong.
2017:10:30 15:07:58 jebberjeb So the predicate dispatch is mostly to catch cases where :via just doesn’t tell you enough (in the s/keys case, I mean the exact field) about what went wrong.
2017:10:30 15:10:09 jebberjeb I’m hoping that approach casts a wide enough net to handle most of the problems generated by our specs. Chopping the specs up, and def’ing each piece seems to help.
2017:10:30 15:29:08 drewverlee I think dispatching on the problem spec’s keyword and maybe some context importer to the end user might make sense. The advantage to just handing them as data and passing them around is that you could organize your code around them.
2017:10:30 15:29:41 drewverlee Is that in line with your thinking? I should get back to you tomorrow when im feeling better.
2017:10:30 15:30:22 jebberjeb I think it is 🙂 Yeah, would love to talk more.
2017:10:30 14:51:09 jebberjeb It’s in use now in a fairly large project, but was introduced only about a month ago. So still waiting to see how it pans out.
2017:10:30 15:06:38 amann Not sure if this is the correct place to ask this question (if not, feel free to boot me to another channel!), but I recently stumbled across some performance related things with respect to spec generative testing specifically regarding every and the :kind and :into parameters.
So the spec I had is loosely similar to this:
As you can see, :into is orders of magnitude faster than :kind for generative testing.
My question is this: when should I be using :kind and when should I be using :into?
2017:10:30 16:20:22 alexmiller Are you using latest spec? This problem has been fixed.
2017:10:30 17:08:17 amann @alexmiller looks like there was a very fun dependency error with pedantic being enabled in leinengen. I was pulling alpha17 when my deps required beta2. Rerunning things I got the following (which looks acceptable to me):
2017:10:30 15:27:46 mpenet the originating code was a bit weird anyway (just calling s/explain-data in a when-let to validate a piece of content and grad the error in one go), but maybe printing here is not something desirable
2017:10:30 16:22:50 hiredman @mpenet that is what explain does, if you look at the code up above it as full of prs, the default for explain is to print stuff out
2017:10:30 16:27:13 mpenet No prob with explain printing. Just found a bit odd that -data would
2017:10:30 16:23:17 alexmiller You’ll get that if you ask for explain on something valid
2017:10:30 16:23:38 alexmiller Which leads me to why you would do that... :)
2017:10:30 16:30:13 mpenet The code in question is gone. But since there an if branch in that code for the printing in case of success maybe it's expected people would do that
2017:10:30 16:23:41 mpenet yes that was the case, not very nice
2017:10:30 16:23:45 mpenet Indeed
2017:10:30 16:25:29 mpenet I was not expecting it to trigger a print regardless
2017:10:30 16:25:51 mpenet ex-data doesnt for instance
2017:10:30 16:29:55 hiredman are you sure you aren't just calling explain, not explain-data?
2017:10:30 16:31:28 mpenet Not in front of my computer anymore. I think it was -data, but not sure
2017:10:30 16:39:02 mpenet Ah no it was explain, sorry about the noise 🙏 All good then
2017:10:30 17:13:34 amann 🐬
2017:10:30 18:38:08 stex I'm was wondering if the following is possible with clojure.spec, since I'm not sure if it's one of the use-cases it tries to solve. Let's say we have a user which can have two roles: (s/def ::role #{:admin :normal}) and a admin specific field which is only required when the user is an admin, not a normal user. I currently have: (s/def ::user (s/keys :req [::role] :opt [::admin-specific-field]), but that's only partially true, since the second key is required depending on the value of ::role. Anyone have an idea how to spec this? Thanks 🙂🙂
2017:10:30 18:39:44 taylor I think there’s a multi-spec example in the spec guide very similar to that
2017:10:30 18:44:06 stex @taylor Thanks! Didn't catch that for some reason 😄
2017:10:30 21:55:39 abp Hey, considering https://dev.clojure.org/jira/browse/CLJ-2116 is the top two voted ticket for Clojure right now, I thought about this general issue for quite a while last year and as a result wrote a spec that, given a root spec, recursively parses specs into reversed dependency order, to do a spec rewrite to a new root, while replacing certain parts of the spec-tree. The latter algorithm isn't implemented yet, I put it aside for quite a while. It shouldn't be too much work but maybe I'd need to get tools.analyzer into the mix for rewrites of multispecs. And then there was Rich announcing better programmability of specs in his conj keynote. So the question is, is it still worth pursuing this? Any ideas about how deep the rabbit hole might get? I'm aware of some libs doing spec parsing into records etc. but for some reason I'd like to avoid that (you're welcome to convince me). Also I haven't looked into term rewriting at all, but maybe that's what I'm trying to do.
2017:11:01 12:05:16 rickmoynihan Does clojure/spec have a predicate that matches g/simple-type?
2017:11:01 12:05:44 rickmoynihan but no more
2017:11:01 12:05:54 rickmoynihan i.e. doesn’t match complex (collection) types
2017:11:01 14:43:27 alexmiller no, but (def simple? (comp not coll?)) is a good first approximation
2017:11:01 14:56:22 mpenet Or (complement coll?)
2017:11:01 14:56:32 alexmiller yep, gooder
2017:11:01 14:58:21 alexmiller well, mine was shorter :)
2017:11:01 14:58:42 alexmiller “complement” is a long word :)
2017:11:01 15:30:59 mpenet 🙂
2017:11:01 20:34:16 uwo am I right to understand that you can’t document keys with (s/keys :opt [::some-key]) without also providing the spec for ::some-key?
2017:11:01 20:47:59 seancorfield It won't generate without a spec for ::some-key but it will validate just fine -- it will treat missing specs as any? I believe.
2017:11:01 20:50:26 seancorfield ^ @uwo is that the scenario you are asking about?
2017:11:01 21:06:18 uwo @seancorfield yes! that’s the way I thought/hoped it worked, but I was calling s/assert in some clojurescript code and it was complaining about some keys that I had declared in s/keys, but that did not have their own spec.
2017:11:01 21:55:07 ajs @alexmiller those were some meaningful tweets
2017:11:02 03:51:46 thedavidmeister i have a really basic question...
2017:11:02 03:51:48 thedavidmeister how do i get
2017:11:02 03:52:12 thedavidmeister (def foo [:foo/bar]) (spec/def (spec/keys :req foo)) to not throw errors?
2017:11:02 03:52:36 thedavidmeister Don't know how to create ISeq from: clojure.lang.Symbol
2017:11:02 04:18:09 seancorfield @thedavidmeister That's because spec/keys is a macro so it doesn't eval its arguments.
2017:11:02 04:18:27 thedavidmeister hmm, is there a way to get around that?
2017:11:02 04:18:37 thedavidmeister it would be nice to reuse foo elsewhere
2017:11:02 04:18:39 seancorfield What problem are you trying to solve?
2017:11:02 04:18:47 thedavidmeister just generally keeping things DRY
2017:11:02 04:19:03 thedavidmeister i have a list of keys that i'd like to put in a spec, but also use elsewhere
2017:11:02 04:19:25 thedavidmeister e.g. putting in a list of options for a dropdown menu
2017:11:02 04:22:35 thedavidmeister mmm, because (::foo (s/form ::foo)) didn't work
2017:11:02 04:23:13 seancorfield It's just data shrug
2017:11:02 04:23:24 seancorfield The spec should be the system of record tho'...
2017:11:02 04:23:43 thedavidmeister maybe
2017:11:02 04:23:59 thedavidmeister i don't necessarily see why it should be
2017:11:02 04:24:12 thedavidmeister and it seems like trying to use it that way is clunky at best
2017:11:02 04:24:33 seancorfield There are JIRA issues around providing a more programmatic API at some point.
2017:11:02 12:47:38 alexmiller Rich is working on a significant spec update that will include more support in this area
2017:11:02 15:27:29 uwo super awesome! around the idea of clj-2112?
2017:11:02 15:37:41 alexmiller no, this is in the internals
2017:11:02 04:25:02 thedavidmeister if it hasn't been prioritised to date, isn't that evidence against the idea of "should be used in this way"?
2017:11:02 04:26:30 seancorfield Spec is designed for human writing and comprehension right now, not for programmatic writing. But at least it's data so you can programmatically read and comprehend them.
2017:11:02 04:26:52 thedavidmeister it's true, it's better than nothing atm
2017:11:02 04:27:08 thedavidmeister i can manually pick apart the form of the spec and alias it to something more convenient
2017:11:02 04:27:46 thedavidmeister but if the intention is truly that spec is a "one stop shop" for this type of thing, then this API needs some polishing >.<
2017:11:02 04:28:53 thedavidmeister (::foo (:keys (s/xxx ::foo))) seems to be what i'm reaching for here
2017:11:02 04:29:12 thedavidmeister where s/xxx is not s/form but something along those lines
2017:11:02 04:30:12 seancorfield I just tried this in the REPL. I think it's clearer:
2017:11:02 04:30:29 seancorfield That will destructure all four possible keys options.
2017:11:02 04:30:45 thedavidmeister oh nice
2017:11:02 04:30:49 thedavidmeister yeah something like that 🙂
2017:11:02 04:31:20 thedavidmeister thanks for the help
2017:11:02 04:31:29 seancorfield It won't deal with s/merge constructs tho'... but what we do is have a bunch of basic specs with s/keys and those are what we destructure to get the actual keys, and then we s/merge them for compound specs.
2017:11:02 04:31:50 seancorfield (funnily enough I was writing some code to do exactly this at work today)
2017:11:02 04:32:13 seancorfield And we have other code that uses specs as the basis of CRUD-style data functions too...
2017:11:02 04:32:21 thedavidmeister i think atm it's pretty good getting things in to spec but not so much getting them back out
2017:11:02 04:32:46 thedavidmeister actually it would be pretty sweet if i could plug spec straight into my UI
2017:11:02 04:33:02 thedavidmeister but it seems to not quite be there yet
2017:11:02 04:34:01 seancorfield I think it's mostly a matter of tooling -- a lot of which will come from the community.
2017:11:02 04:34:23 thedavidmeister totally, i've seen a lot of cool tooling already just lurking in this chat
2017:11:02 04:34:35 seancorfield Given that 1.9 is still in pre-release, not everyone is using it yet, which is holding back tooling.
2017:11:02 04:35:06 seancorfield We have 1.9 Beta 3 in production and we're heavy users of spec but it's all still a bit of an adventure...
2017:11:02 04:35:35 thedavidmeister yeah i know that hoplon had to make some changes to keep pace with 1.9, and also has started playing around with spec but it's WIP
2017:11:02 04:35:40 thedavidmeister i imagine it is the same for everyone
2017:11:02 04:36:24 seancorfield Beta 4 broke our code due to removing bigdec? which was added earlier in 1.9. But it was a small change. We expect to have Beta 4 in production on Monday.
2017:11:02 04:36:33 thedavidmeister nice 🙂
2017:11:02 04:36:40 thedavidmeister well i g2g run some errands
2017:11:02 04:36:42 thedavidmeister thanks for the help
2017:11:02 16:11:54 uwo @seancorfield I was able to create minimal reproduction. This only occurs in cljs. Should I make a ticket for this?
2017:11:02 16:14:17 uwo guessing this is related to fspec invoking a generator, but dunno
2017:11:02 21:28:22 alexmiller I believe it is. fspecs will be checked by using the :args generator to verify the function can accept those args. In this case it’s trying to generate ::test instances that sometimes include ::unspeced-keyword instances, but it doesn’t know how to do that.
2017:11:02 21:28:40 alexmiller or at least that’s my guess, maybe something else going on
2017:11:03 13:43:19 uwo Should I create a jira cljs ticket for this?
2017:11:03 14:06:20 alexmiller sure
2017:11:02 16:54:35 seancorfield If that works in Clojure but fails in cljs, raise a JIRA issue. If it failed in both I'd say it was related to generators. If it's only failing in cljs, I'm not so sure.
2017:11:02 16:55:57 seancorfield Confirmed it works in Clojure, so that looks like a cljs bug.
2017:11:02 16:58:24 uwo thanks. will do
2017:11:02 19:07:44 bmabey How do I use a sequence spec as a function argument? This is what I've tried so far without any luck:
2017:11:02 19:11:17 taylor two changes: your ::int-then-strings spec wrapped in s/spec to prevent it from getting “flattened” into one big sequence spec, and your test call was invalid according to your spec: it was an integer and not a map
2017:11:02 19:11:19 bmabey Thanks! Why is the extra s/spec needed around the s/cat?
2017:11:02 21:30:12 alexmiller because all regex specs combine to describe a single level of sequential collection. The s/spec forces a boundary such that the outer regex spec includes a collection which has an inner regex spec.
2017:11:02 20:58:15 xiongtx Been getting some problems when using clojure.spec.alpha, clojure.test.check.generators, and boot-clj’s cljs compilation.
See issue: https://github.com/clojure-emacs/cider/issues/2104 and minimal repo: https://github.com/xiongtx/reload-error-boot
@alexmiller Is it not recommended to use clojure.test.check.generators directly when using clojure.spec? It seems to me there’s weird interaction b/t the lazy combinators, CLJS compilation, and namespace reloading. I suspect we can avoid this problem by using only clojure.spec.gen.alpha.
2017:11:02 21:33:16 alexmiller I don’t understand the problem - too much tooling and other stuff for me to get it. Shouldn’t be any issue with using clojure.test.check.generators directly - that’s the same thing clojure.spec.gen.alpha is doing, just with a delayed load step.
2017:11:02 21:40:50 alexmiller generators in test.check are records iirc - maybe you’re running into a case where the record is being redefined and the old generators are no longer instances of the new (reloaded) record class?
2017:11:02 23:19:43 xiongtx Good insight! Yes, that seems to be precisely what’s happening. After a refresh:
The problem is that clojure.test.check.generators was reloaded (and the defrecord re-evaluated), but the same was not done for the lazy combinators in clojure.spec.gen.alpha. This is probably b/c clojure.spec.gen.alpha does not :requireclojure.test.check.generators, so the dependency tracking in clojure.tools.classpath... isn’t working properly.
2017:11:02 23:55:41 xiongtx I believe there was some talk of giving clj a way to lazily load :requireed namespaces. That, if implemented, would seems like the ideal solution here.
2017:11:03 01:00:04 James Vickers What is the shape of the data that gets passed to the :fn argument of s/fdef? Does someone have an example of that (or a way to print it)?
2017:11:03 01:00:57 taylor it’s a map with :ret and :args keys
2017:11:03 01:01:39 taylor the :args value is the conformed value of the args I think
2017:11:03 01:02:36 taylor
:fn (fn [{:keys [args ret]}]
...
2017:11:03 01:03:45 James Vickers Thanks. Is that in the docs somewhere?
2017:11:03 01:04:52 alexmiller That is all correct - the values of the map are the confirmed values of the args and ret specs
2017:11:03 01:05:25 alexmiller I’m not sure where it would be doc’ed in the docstrings
2017:11:03 01:05:39 James Vickers Thanks
2017:11:03 01:05:46 taylor
:fn A spec of the relationship between args and ret - the
value passed is {:args conformed-args :ret conformed-ret} and is
expected to contain predicates that relate those values
2017:11:03 01:05:52 taylor fdef docstring
2017:11:03 01:06:46 James Vickers Ah, thanks.
2017:11:03 01:07:41 James Vickers So, if the function returns a single value (like a number), then (% :ret) in the :fn spec should yield the return value?
2017:11:03 01:08:41 alexmiller Depends on the ret spec
2017:11:03 01:08:51 taylor it might be tagged/conformed though I guess?
2017:11:03 01:08:52 alexmiller If it’s a simple pred then yes
2017:11:03 01:09:10 alexmiller Right
2017:11:03 01:10:16 James Vickers Sweet, thanks.
2017:11:03 01:10:48 James Vickers I was surprised that putting a do expression with a println in the :fn spec didn't print anything when the function was called (instrumentation on)
2017:11:03 01:11:12 taylor the :fn spec doesn’t come into play re: instrument calls
2017:11:03 01:11:45 taylor it does get used if you check the function though
2017:11:03 01:11:50 James Vickers oh
2017:11:03 01:12:30 taylor > Instrumentation validates that the :args spec is being invoked on instrumented functions …
2017:11:03 01:12:59 taylor > check will generate arguments based on the :args spec for a function, invoke the function, and check that the :ret and :fn specs were satisfied.
2017:11:03 01:13:55 James Vickers Cool. I ran a check and it printed out what was passed to :fn.
2017:11:03 13:17:06 zclj If I have a production spec of say an email address, it will require a custom generator if I want to generate test data. Since a generator typically will not be used in production would it be a good practice to redefine the production spec in i test ns that will include the generator, keeping any generators out of production code? Can this lead to problems if I have nesting and to be able to generate the top level spec I need to redefine lower levels with the generator in my tests?
2017:11:03 14:04:18 alexmiller Another thing to consider is using generator overrides at the point of testing. That way you don’t have to redefine a spec, you just supply a set of alternate generators.
2017:11:03 14:19:33 zclj But in the case of a nested spec such as a person having an email, and I am testing the person don't I have to redefine the email spec refered by the person spec?
Is generator overriding done by using with-gen or are there other ways I have missed?
2017:11:03 14:29:45 alexmiller no, you can supply custom generators in exercise, instrument, check, etc
2017:11:03 14:31:03 alexmiller in stest/check for example, you can supply a :gen map in the options: “map from spec names to generator overrides”
2017:11:03 14:31:27 alexmiller so that way you can supply overrides just in the context of a test
2017:11:03 14:53:48 zclj ah I see, will try that out, thanks for your help!
2017:11:03 21:30:01 seancorfield Good to be reminded of that. So far, we've tended to write wrappers for generators so we can lazy load the testing library (and therefore keep the actual "testing generator" in the production code without needing the testing libraries in production).
2017:11:03 13:25:47 wilkerlucio hello, has anyone here though about the idea of specing nested structures? currently s/keys only supports flat structures, if I need a nested I have to define the nested structure ahead of time on that key, this prevents different nesting structures depending on the context
2017:11:03 13:26:35 wilkerlucio maybe we could support via a syntax like the datomic pull syntax, eg: (s/keys :req [:user/name {:user/address [:address/line1 :address/city]}])
2017:11:03 14:07:01 ikitommi also would like to see support for nested keys-specs in spec itself.
2017:11:03 14:07:51 alexmiller that’s not in line with the “set of attribute” thinking and no plans for that
2017:11:03 14:55:34 wilkerlucio the issue I'm facing is that for the "container" specs I might want different subsets of keys, depending on the context
2017:11:03 14:55:43 wilkerlucio so by not having a fixed for the children, it gets tricky
2017:11:03 15:06:57 alexmiller you don’t need a fixed spec for the children
2017:11:03 15:07:10 alexmiller an empty (s/keys) is sufficient to cover an open set of attributes
2017:11:03 15:07:22 alexmiller or use s/multi-spec to select the spec based on the contents
2017:11:03 15:07:30 alexmiller or s/or multiple choices, etc
2017:11:03 15:08:58 wilkerlucio thanks for the tips Alex, I like the open one, but at same time, if I want to require different sets of keys depending on the context, that doesn't work, the multi-spec can work, but its a lot more involved
2017:11:03 15:11:20 wilkerlucio I love the idea of living by just attributes, but if I need to give a name to a context of sub-attributes, I feel like backing again to the "box" (class, entity, whatever...) constraints again
2017:11:03 15:17:38 wilkerlucio @U055NJ5CC thanks for pointing that out, I'll look it up
2017:11:03 15:18:17 alexmiller what you’re saying is that they key at the top level does not have a stable semantic meaning so I would think about what that means and whether it’s a good idea
2017:11:03 15:19:58 wilkerlucio yeah, in general terms, any sub-set might be unstable, if what you care is just about the leaf attributes validation
2017:11:03 15:21:09 wilkerlucio I've been writing a considerable amount of code regarding to data fetching apis (om.next style), and many times I see that the sub-set of keys of a child element can vary wildly, in my case I'm embracing this and it's working pretty good, but I can't get the specs around to match it in the way it is
2017:11:03 15:21:31 alexmiller (s/keys) ! :)
2017:11:03 15:21:58 wilkerlucio the problem is the children, for example, working in micro-service architecture
2017:11:03 15:22:12 wilkerlucio I have many endpoints across services that can return different sub-sets of the data
2017:11:03 15:22:32 wilkerlucio and altough they share some root keys, what comes in the children is variable, some endpoints give more, some give less information
2017:11:03 15:22:57 wilkerlucio and in current spec way, I can't define what is required for each return (or input) on a case-to-case bases
2017:11:03 15:23:57 wilkerlucio in the same way we can't have a good definition of what are the required fields for an user (a login might be user/password, a signup would require much more)
2017:11:03 15:24:18 wilkerlucio I feel the same problem when trying to specify requirements for a nested item
2017:11:03 15:24:30 alexmiller I think that’s all ok, and you should not try to define every key set aggregation
2017:11:03 15:24:38 alexmiller lean on the attributes
2017:11:03 15:27:26 wilkerlucio but that's the problem, if I have one key like :user/address, I expect this key to be the same always, but what I expected to be inside of it can change from case to case
2017:11:03 15:27:38 wilkerlucio on the top level, just create a new set and we are all good
2017:11:03 15:27:46 wilkerlucio for the nested, there is no way to override the requirements
2017:11:03 15:28:00 wilkerlucio to say that the sub-set is different a separated case, makes sense?
2017:11:03 15:29:19 alexmiller why not just (s/def :user/address (s/keys)) ?
2017:11:03 15:29:35 alexmiller then rely on your address attribute specs to do the work
2017:11:03 15:34:16 wilkerlucio the only issue there is that we can't make some attributes required for a context that way
2017:11:03 15:34:28 wilkerlucio so the validation gets too loose
2017:11:03 15:36:08 alexmiller if you need that then it sounds like s/multi-spec to me, or state all the possibilities with s/or, or add outer constraints s/and’ed at the top level
2017:11:03 15:36:34 wilkerlucio humm, the top level constraint sounds like a good path for the cases I'm thinking
2017:11:03 15:36:52 wilkerlucio thanks, I'll try that and see how it goes
2017:11:03 15:36:56 alexmiller always come back to “state the truth about what it’s in your data” - what can occur in your actual data? say that in the spec.
2017:11:03 13:27:19 wilkerlucio had anyone felt the desire to specify the kinds on that way?
2017:11:05 20:26:17 drewverlee I just spenT a day thinking about how to use specs for human error msgs. i think phrase is interesting but it seems to lack a way to do some rather useful things and makes others somewhat more difficult then then should be.
The one problem i can't think around is naturally combining your "for user erorrs" together when you concat specs together.
2017:11:06 13:27:49 dominicm @drewverlee I think the human error messages also need to include a location which is separate from their spec position. Sometimes you need to do validation across keys for humans, but place the error message on a particular key (password & password_confirm must match, but if they don't, the error is in password_confirm)
2017:11:06 14:24:07 gfredericks
2017:11:06 18:13:40 seako i find them noisy and like that test.chuck turns them off when using the checking macro
2017:11:06 16:51:02 drewverlee Is there a rational for the :reason key in the problem map produced by spec/explain-data?. I cant find any discussion on it
2017:11:06 20:34:42 drewverlee is there a function that you can call on a spec to get the code for the spec?
2017:11:07 16:38:20 drewverlee gotcha. I was actual experiencing another mis understanding the same time concerning lists.
2017:11:06 21:12:40 taylor s/fdef is using s/def “behind the scenes” to register the spec to the function symbol, so you can use the function symbol to resolve the spec (instead of a namespaced keyword)
2017:11:07 09:26:20 mattiasw map-like? I have an application and I use maps a lot. Added spec, most of them using s/keys. In some cases I want to apply operations to the maps like filter. But this breaks the specs, since it isn't a map any more, but a sequence of pairs. The simple solution is of course to convert the result of filter back to a map, but is there a better way? Here is a small sample
2017:11:07 13:17:22 alexmiller You can spec them as s/coll-of an s/tuple of key value pairs. That’s not great if you are still relying on attribute keys. I guess you could also use s/keys* on the kv tuple.
2017:11:07 14:19:39 mattiasw If I understand it correctly, s/keys* wants the structure [:a 1 :b 2], but I have [[:a 1][:b 2]]
2017:11:07 12:08:51 rickmoynihan What is the best way to spec a map which has a single required key (s/keys :req-un [::id])(s/def ::id int?) where every other (optional) map key is a string? with a string? value? Such that (s/valid ::spec {:id 123}) ;; => true, (s/valid ::spec {"foo" "bar" :id 123}) ;; => true, (s/valid ::spec {"foo" "bar"}) ;; => false
2017:11:07 13:08:17 taylor here’s a really naive way to do it:
2017:11:07 13:37:49 rickmoynihan taylor: yes I had something similar to your first, but the use of and & or in a single monolothic predicate bothered me, as it kills error message granularity further down the tree.
2017:11:07 13:38:22 rickmoynihan I think as you’ve discovered the use of every and or looks to be how to do it 🙂
2017:11:07 13:38:33 rickmoynihan Thanks @U064X3EF3 for the pro tips
2017:11:07 12:31:44 sushilkumar How to write spec for “string of integer”? I want to create a generator of "string of integer" which should only generate valid string of integer (value within Integer/MIN_VALUE and Integer/MAX_VALUE).
2017:11:07 12:31:44 sushilkumar How to write spec for “string of integer”? I want to create a generator of "string of integer" which should only generate valid string of integer (value within Integer/MIN_VALUE and Integer/MAX_VALUE).
2017:11:07 12:45:35 taylor you could write a predicate function int-str? that returns true/false if the given string can be parsed as an integer, then it’s trivial to use that predicate as a spec
2017:11:07 12:45:59 gfredericks The generator would be (gen/fmap str an-appropriate-int-generator)
2017:11:07 14:36:58 sushilkumar Thanks @U3DAE8HMG and @U0GN0S72R for sharing your ideas. Now I am able to do it as follows and it will also help in writing fdefs.
2017:11:07 18:54:55 seancorfield Quick Q about double-in -- if I specify :min 0.0 and :max 1.0, does that automatically exclude NaN and infinity? Or do I also need to specify :NaN? false :infinity? false?
2017:11:07 19:24:23 seancorfield Thanks @U3DAE8HMG That's sort of what I intuitively expected to happen but I wasn't sure how "special" NaN was... I guess that begs the question of what do you do if you want 0.0 .. 1.0 or NaN? I suppose you have to :or two specs together... but what would that second spec look like, i.e., how would you allow only NaN or a range?
2017:11:07 19:27:14 taylor
2017:11:07 19:27:47 taylor a more qualified person could very well have a better answer!
2017:11:07 19:59:50 seancorfield That allows any double, it seems.
2017:11:07 20:33:10 taylor ha! yeah it does… disregard
2017:11:07 20:34:47 taylor looking at double-in impl. :NaN? true is a no-op
2017:11:08 17:12:55 akhudek What is the rational for s/cat returning a map when conformed?
2017:11:08 17:13:19 akhudek I suppose there is no way to conform a sequence and get a sequence rather than a map?
2017:11:08 17:13:47 akhudek I know you can use coll-of but that doesn’t work if you want a sequence rather than a collection
2017:11:08 17:16:50 bfabry you can make it non conforming
2017:11:08 17:17:15 bfabry the rationale is that s/cat starts a regex spec, and regexes have alternates, so you need to give everything names to know which part of the regex it matched
2017:11:08 17:30:13 rickmoynihan are there any docs/guides etc on conforming / conformers and tweaking output etc? I understand roughly how they work, but am curious about how to apply them in practice. What it’s ok to use them for, what it’s not a good idea to use them for etc…
2017:11:08 18:13:00 alexmiller conformers exist primarily to build custom composite spec types (s/keys* for example)
2017:11:08 18:27:47 rickmoynihan interesting hadn’t seen s/keys* before
2017:11:08 19:04:46 alexmiller s/nilable was originally written this way too although I ended up rewriting a custom impl for better performance
2017:11:08 18:13:29 alexmiller generally I would say you should not use them for data coercion or tweaking output
2017:11:08 18:13:34 zclj I have a case where I try to spec a HOF but I can not get check to satisfy. Here's a toy example of the problem:
2017:11:08 18:37:12 zclj Good to know, then I can stop scratching my head. Yes, that example yields the same result
2017:11:08 18:50:57 alexmiller yeah, I see the bug in the code. can’t say I know how to fix it though.
2017:11:08 18:52:04 alexmiller when the fspec is conformed by exercise (or by check), it “checks” the generated function spec, but that conform does not have access to the gen overrides so it gens without them.
2017:11:08 18:54:10 zclj I see, that explains the observed behavior
2017:11:08 18:59:19 zclj should I file a ticket? Have not done so before but I am willing to try if that would be helpful
2017:11:08 19:05:53 zclj thanks, will follow the activity, and thanks for your help and explanation of the problem
2017:11:08 19:07:33 alexmiller it’s kind of a deep problem - we’ve got other tickets that are related but I never took the time to really get it.
2017:11:08 19:26:29 zclj I would be happy to help, but it sounds like its above my knowledge of the code base if it is a deep problem, not a novice ticket
2017:11:08 18:15:29 seancorfield @rickmoynihan Where I've found conformers useful is dealing with input data that is all strings, but is expected to conform to numeric, boolean, date, etc types.
2017:11:08 18:16:04 rickmoynihan @seancorfield: yeah I understand the input coercion at boundaries case
2017:11:08 18:16:49 seancorfield That's pretty much the only place I'd "recommend" them, having worked with spec in production since it first appeared.
2017:11:08 18:19:07 rickmoynihan ok thanks, that’s useful. I just remember rich saying that conformed values are basically the parsed/labelled output you want; and it’s very close to a shape I want for a specific case… thinking it’s better to conform then post-process, rather than use the conformers to do it.
2017:11:08 18:20:51 rickmoynihan but it occurred to me that if I used conformers then I’d really need to write unformers too, and my spec output would no longer be spec output… so what you’re saying seems to agree my gut feeling
2017:11:08 18:39:51 rickmoynihan so one thing I’d quite like (for myself) is to write a variant of s/keys and s/def that uses URIs as keys instead of clojure keywords.
I work with RDF, so mapping to keywords just to spec something feels somewhat redundant, when the URI straight out of the database is basically the same thing so I’d like to basically save a redundant mapping and write:
(def rdfs:label (URI. "http://,,,/label"))(rdf/def rdfs:label string?)(rdf/keys :req [rdfs:label])
And have it do the same thing as keys. Is there an easy way to do this? Thinking I need to extend s/Spec/`s/Specize` to URI and then rewrite s/keys? s/keys looks pretty hairy, is there any easy way?
2017:11:08 19:03:58 alexmiller I think this is roughly how you would do this, yes, and it would not be super easy. Also, I expect some of that plumbing to possibly change soon.
2017:11:08 19:11:45 rickmoynihan I’m assuming there are no plans/proposals in the pipeline to let you plugin and you use arbitrary values as keys/keywords?
2017:11:08 18:40:16 rickmoynihan or am I thinking the thoughts of a madman?
2017:11:08 18:43:14 rickmoynihan I think what rich says about the design influence of RDF on clojure and spec is very clear; in many ways they’re almost identical. I’d basically like to reduce the friction of working in clojure with RDF… Also compare SHACL/SHEX to spec, they’re almost just different syntaxes for the same abstract model.
2017:11:08 18:58:08 akhudek ok, thanks for the comments bfabry and alexmiller, I’ll avoid using spec for transforms.
2017:11:09 00:02:10 rickmoynihan I think I’ve found a bug in clojure 1.9 / clojure.core.specs.alpha:
(if-let [foo 1] foo :clojure.spec.alpha/invalid)
CompilerException clojure.lang.ExceptionInfo: Call to clojure.core/if-let did not conform to spec:
In: [2] val: :clojure.spec.alpha/invalid fails at: [:args :else] predicate: any?
#:clojure.spec.alpha{:problems ({:path [:args :else], :pred clojure.core/any?, :val :clojure.spec.alpha/invalid, :via [], :in [2]}), :spec #object[clojure.spec.alpha$regex_spec_impl$reify__1188 0xeabf22e "
2017:11:09 00:05:12 rickmoynihan issue appears to be that you can’t return the value :clojure.spec.alpha/invalid from macros. It’s kinda like a macro hygiene issue, but for s/invalid
2017:11:09 00:06:49 bronsa it's known
2017:11:09 00:07:46 rickmoynihan cool. Figured it would be
2017:11:09 12:26:46 sushilkumar Hello everyone, I am learning Spec'ing higher order functions.
Here are the examples which I am experimenting with (actual HOFs I am working on are quite complex) :
(Works fine with exercise-fn.)
(defn hof1 [x f] (f x))
(s/fdef hof1
:args (s/cat :x number?
:f (s/fspec :args (s/cat :y number?)
:ret number?))
:ret number?
:fn #(= ((-> % :args :f) (-> % :args :x)) (:ret %)))
(exercie-fn doesn't work :(. Here I am expecting input f to be either + or - or * fn.)
(defn hof2 [f x] (f x 1))
(s/fdef hof2
:args (s/cat :x number?
:f (s/fspec
:args (s/cat :y number?)
:ret number?))
:ret number?
:fn #(= ((-> % :args :f) (-> % :args :x) 1) (:ret %)))
How write spec to ensure correctness in the samples generated using exercise-fn for HOFs? (I want to use spec's check fn to test HOFs on generated inputs based on written spec.)
2017:11:09 14:29:03 kumarshantanu Hi, I’m a spec noob here — can somebody tell me an easy way to automatically (st/instrument) in all namespaces?
2017:11:09 14:30:03 joost-diepenmaat you only need to call st/instrument once
2017:11:09 14:30:19 joost-diepenmaat but you have to make sure you’ve loaded all the namespaces before you do
2017:11:09 14:30:53 joost-diepenmaat if it’s in your dev setup, just call st/instrument in your main or dev namespace at the bottom
2017:11:09 14:31:11 joost-diepenmaat if it’s in tests, your best bet is probably to use a fixture
2017:11:09 15:07:17 kumarshantanu Also, how do I generally make “did not conform to spec” errors easier to read? I’m using Orchestra+Expound and have specified the following, but to no avail:
2017:11:09 15:34:46 kumarshantanu This gist works for me at the REPL. @bbrinck The error reporting I’m dealing with extracts the exception message and stack trace and sends it as HTTP response. Is Expound supposed to work in that use case?
2017:11:09 16:03:35 bbrinck Yes, it should. For instance, building on my previous example, here is code to catch the instrumentation exception and return the message as a string: (try (foo "") (catch Exception e (.getMessage e)))
2017:11:09 16:04:54 bbrinck Another option is to use Expound to validate incoming data directly. For instance, let’s say the HTTP endpoint takes some JSON as incoming data
2017:11:09 16:06:06 bbrinck Then you could validate that data with something like:
2017:11:09 16:08:29 bbrinck By default, Expound writes out all the Clojure specs that failed, but there is an (undocumented, but soon to be documented) option to omit this
2017:11:09 16:09:22 bbrinck if you’re interested in that, I can give some example code
2017:11:09 18:49:35 bbrinck If anyone else is interested, the issue was the dynamic var *explain-out* is only set in the current thread, and servers like HTTP Kit spawn new threads for requests. The fix is to replace (set! s/*explain-out* expound/printer) with (alter-var-root #'s/*explain-out* (constantly expound/printer))
2017:11:09 17:38:11 naomarik there a better way of having a generator that just executes a function with no regard to the input? this does what i want, but notice the argument here is unnecessary #(gen/fmap (fn [_] (utils/uniform 1e12 1e13)) (s/gen pos-int?))
2017:11:09 18:58:54 taylor I was going to suggest gen/return but do you even need the output of the pos-int? generator?
2017:11:09 19:04:37 taylor for example, doesn’t matter what the previous generator is doing if you’re just constantly returning the same thing regardless of input
2017:11:09 19:23:56 gfredericks this is a nonstandard generator construction
The "right way" would be to use combinators somehow rather than calling a function that supplies its own randomness.
That might be a bit more effort, depending on what you want.
If you aren't doing things the "right way", then there's not really a better way.
2017:11:09 19:24:11 gfredericks do you really need a uniform distribution between two doubles?
2017:11:09 19:24:47 gfredericks (gen/double* {:min 1e12 :max 1e13}) would not be very nonuniform I don't think
2017:11:13 05:56:41 naomarik @U3DAE8HMG yeah my point was i didn’t need the output of the pos-int? generator. gen/return was EXACTLY what i was looking for, thanks 😉
2017:11:13 05:57:29 naomarik @U0GN0S72R yeah i need uniform distribution between two large numbers because made a spec that expected a number beteween those two ranges and this issue causes it to fail without a custom generator https://dev.clojure.org/jira/browse/CLJ-2179
2017:11:13 05:58:02 naomarik i also stumbled upon #(gen/choose 1e12 1e13) that does uniform distribution
2017:11:13 12:55:06 gfredericks that ticket is interesting
2017:11:13 12:55:46 gfredericks @U0DHHFEDP what is your spec for exactly?
2017:11:13 12:55:54 gfredericks I'm having trouble seeing why something would fail
2017:11:13 13:27:05 naomarik @U0GN0S72R it’s a ghetto spec for a unix timestamp in ms, the generator fails because it attempts to make ints starting at very low numbers. here’s the code
(s/and pos-int?
(s/int-in 1e12 1e13))
2017:11:13 13:30:04 naomarik it passes most the time when ran by itself but i have it nested in another datastructure and it fails 100% of the time
2017:11:13 13:30:37 gfredericks oooooh; this is an s/and issue, not an s/int-in issue
2017:11:13 13:30:56 gfredericks if you swap the order of those two you should have it succeed
2017:11:13 13:31:20 gfredericks or set the generator to be the default generator that the s/int-in expression generates
2017:11:13 13:31:37 gfredericks I think CLJ-2179 is a red herring here
2017:11:13 13:33:38 naomarik tried swapping the order, didn’t work
2017:11:13 13:35:12 naomarik i think the issue is legit, if you try running the code (gen/sample (s/gen (s/int-in 0 100))) you get a ton of low numbers
2017:11:13 13:36:17 naomarik my final code works well enough though
(s/with-gen (s/and pos-int?
(s/int-in 1e12 1e13))
#(gen/choose 1e12 1e13)
))
2017:11:13 13:36:55 naomarik sorry for formatting 😉
2017:11:13 17:29:22 gfredericks @U0DHHFEDP getting low numbers that are in the range is one thing, but I thought you were describing getting numbers that were out of range
2017:11:14 02:33:43 naomarik @U0GN0S72R that’s what I thought was going on, i think my issue is that i was plugging in doubles using the short notation. It seems (s/int-in) doesn’t cast those for you but will verify that the input is in range, this is why I needed to pair it with pos-int
2017:11:14 02:38:28 naomarik bumping the numbers up in the first example makes it fail a lot more often, I guess pos-int is the only thing generating numbers here?
2017:11:14 02:43:31 naomarik in any case was my fault for plugging doubles in something that is clearly for ints 😉
2017:11:14 12:06:25 gfredericks yeah, when you use s/and the generator is taken from the first spec and the remaining specs are used to filter
2017:11:10 19:02:01 uwo I know this question comes up all the time, and that multi-spec is a way to solve it. but just asking again to see if there’s another sol’n:
we want to ensure that a data structure at some point in our system conforms to a certain shape. The data is nested and the keys already have fixed namespaced names. It’s easy to specify required keys at the top level, but at every nested level the key already has a fully qualified name, and that fully qualified key has different requirements in different system contexts.
The only two sol’ns I see atm are multi-specs and key-name rewriting. Using an empty (s/keys) is not a sol’n in this case, because we need conform to ensure that required fields at nested levels are present.
2017:11:10 19:49:48 alexmiller when you say “fully qualified key has different requirements in different system contexts” you have started off by violating spec’s assumption
2017:11:10 19:49:59 alexmiller unless you spec it to cover all possible contexts
2017:11:10 20:52:01 uwo @alexmiller gotcha. I figured that was the answer. Those fully qualified key names are the ones we place in datomic, and they get shuffled all over the app. It’s just different requirements for the “same” data in different contexts
I'm starting to get the impression spec is NOT ONLY for unit testing / debugging, but also something intended to be used in live production code ?
2017:11:10 20:55:25 qqq It's not clear to me how this s/conform is useful, unless it is meant to be used in production to 'reshape' data
2017:11:10 21:14:36 taylor I don’t think it’s intended to arbitrarily reshape data, but the “tagged” output from conform can be useful. For example, if you’re expecting different types of inputs, s/or will tag the matched spec in the conform output, otherwise it’d be hard to tell which spec matched
2017:11:10 21:15:56 taylor I do use it in production for similar cases, one example is a message handler that receives a few different types of messages that don’t have a natural “key”. conform output tells me exactly which kind of message it is
2017:11:10 21:38:54 seancorfield @qqq We use spec very heavily in production code. We s/conform input values from forms and REST APIs to the desired shape/types.
2017:11:10 21:40:40 seancorfield We define a spec for each API endpoint, and we s/conform the raw Ring :params to the spec. If it's s/invalid? then we use s/explain-data to guide us in returning error code and messages, otherwise we have a validated parameters map.
2017:11:10 21:41:28 seancorfield We limit the coercions tho'... We do string->long, string->boolean, string->date, and very little else.
2017:11:10 23:17:13 qqq @seancorfield: interesting, I was thinking that something vaguely along those lines would be possible ; nice to see someone do it in practice
2017:11:10 23:17:32 qqq unrelated spec question: how does s/or and s/alt differ ?
2017:11:10 23:19:07 qqq I'm running them with s/conform ... and getting identital results
2017:11:10 23:31:10 seancorfield s/alt is for alternatives in a (sequence) regex; s/or is for specific alternatives in a spec.
2017:11:10 23:31:46 seancorfield (so s/alt mostly goes with s/cat etc)
2017:11:10 23:41:06 qqq Is there a way to define a spec, then selectively, on valid? calls turn "recursion on / recursion off" ?
Practically, imagine we are speccing SVG.
For a given SVG node, there are (1) properties of this node and (2) the children of this node.
I want to be able to define the spec once, and then selectively say:
1. check that this node itself is valid (but don't care much about its children)
vs
2. check that this node itself is valid, and that all its children (and all descendants) are valid too
so one is a "shallow, just this level check"
another is a "deep, check the entire tree check"
2017:11:10 23:55:08 bfabry I'm 99.4% sure that the answer is "define 2 specs"
2017:11:11 02:07:30 qqq the problem with two specs is that in the case of maps, the key is used to pic, the spec for the value
2017:11:11 02:07:39 qqq which mean the data 'decides' whether the check is recursive or non-recursive
2017:11:11 05:16:32 seancorfield @qqq But that's exactly the point: with two separate specs you control whether the keys are deep or shallow -- assuming you have unqualified keys in the map and can use different namespace-qualifiers for shallow vs deep.
2017:11:11 05:25:57 qqq @seancorfield: but I want to determine whether to make a shallo/dep check at the function call, not at the point when I construct the data
2017:11:11 05:26:32 qqq I want a single piece of svg data
in some functions, I want to make an expensive deep check taht it's an valid svg tree
ikn other functions, I want to make a shallow check that "current node is good; we know nothing abut chiklren"
2017:11:11 09:30:29 mattiasw I need just 1 running one call to the function to check at a time. Inside the function, there is a db call, so I need to run checking in single-thread, so (check 'test-many)doesn't work. I tried to use check-fn, but this doesn't even start calling the function. (stest/check-fn 'test-many (s/fspec :args (s/cat :samples (s/coll-of :hashdb.db.commands/data)) :ret nil?)) (I changed backquote to quote when publishing to slack.
2017:11:12 01:52:32 James Vickers Is there a 'normal' way to use function specs in your automated test suite? I feel like I haven't found a good solution to this - likely because I haven't written a regular test.check suite either.
2017:11:12 01:54:37 gfredericks wrapping a call in clojure.test/is is pretty easy
2017:11:12 02:08:55 James Vickers I was wondering if there was already (`test.check`?) library function that would do something like that (some 'idiomatic' way).
2017:11:12 02:09:40 James Vickers Is using function specs in the test suite a normal thing that people do, or are folks just using spec for development time help?
2017:11:12 02:15:10 gfredericks I would certainly put them in the test suite if I were using spec
2017:11:12 02:31:38 seancorfield @jamesvickers19515 We use spec in all sorts of ways. We use it heavily in production code for validating and conforming input parameters, specifying data structures (and then leveraging the form of the spec to "reflect" and get the set of keys that are "permitted", e.g., when storing hash maps in a SQL table), in tests via s/instrument, in tests via s/exercise to generate conforming test data.
2017:11:12 02:32:30 seancorfield In some cases we use test.check as part of a regular test, but generally we have those as separate tests we run apart from our unit tests -- generative testing often takes quite a while so it probably shouldn't be part of your unit test suite.
2017:11:12 03:45:26 jeaye That spec catches the errors just fine, on Clojure, for all the arities. On ClojureScript, bad calls don't get caught by instrumentation at all.
2017:11:12 03:50:44 jeaye Updated the gist; looks like a simple defn, with the same spec, throws for both cases (as it should). So this is specific to something happening in ClojureScript's multi-arity defn.
what am I doing wrong here, it is complaining that regex can not be converted into a function
2017:11:12 16:45:32 qqq the last line should be:
(s/valid? ::stroke-color ...)
2017:11:13 04:53:48 qqq is there a way in spec to say
::x is a vector of numbers
::y is a vector of numbers
::s is a string
and the three have the same length (thisxis regarding to an s/keys)
2017:11:13 04:55:42 qqq this is what I have so far:
but it's not clear to me how to specify the three have the same length
2017:11:13 06:24:35 seancorfield @qqq Are you using them in a context together? Sounds like you could s/and a length check on top of whatever structure uses all three?
(There might be more straightforward implementation of vals-same-length?)
2017:11:13 07:38:33 qqq @jumar: I did not realize I can just pass in an arbitrary clojure function.
2017:11:13 07:38:47 qqq This is very useful, thanks!
2017:11:13 08:07:00 seancorfield (sorry, was distracted by the amazing availability of early 80's German electronica on iTunes these days... yes, @qqq what @jumar said... when you use s/and you can have any predicate and it has access to the entire form being processed, so you could access just ::text-x, ::text-y, ::text-str if you wanted, or vals to get all the map's values.
2017:11:13 08:07:39 seancorfield I was thinking you were using this with s/cat for argument specifications, but the same principle applies. Specs are just predicates. It's all functions, all the way down 🙂
2017:11:13 19:37:16 deg I need to define a spec for a map that holds keys owned by multiple parts of my code. So, looking a a tasteful way to build it incrementally from multiple files. (So that each namespace adds in the parts it understands).
This strikes me as non-trivial, especially since I can't just have a defonce'd atom that I assoc into. Because s/keys is a macro, I need to assemble only after I've gathered all the parts.
Sounds like I need to think about file loading order and other issues that I've ignored until now in my Clojure/Script.
Is there a straightforward way to do this? Or am I barking up the wrong trees?
2017:11:13 19:44:53 hiredman I am not sure what you are asking?
2017:11:13 19:45:09 hiredman if each key is owned by a different part of your code, and you want to spec each key do that
2017:11:13 19:45:27 hiredman and then in whatever central place, create your keys spec from each key
2017:11:13 19:46:46 hiredman the structure of clojure requires you to have a central place, the way the ns macro and everything else works requires you to have a defined load order and explicitly require the code you need access to
2017:11:13 19:52:07 qqq does s/coll-of check every element, or does it randomly pick a few and test them ?
2017:11:13 19:52:39 taylor > coll-of will exhaustively conform every value
2017:11:13 19:53:23 taylor as opposed to every:
> Note that ‘every’ does not do exhaustive checking, rather it samples
coll-check-limit elements.
2017:11:13 20:05:46 bbloom been away for a bit - i remember core had some private convenience macro for checking macro invocations. what’s the latest recommendation on that? i actually want to use the spec for parsing inside the macro, but it’s a bit awkward to conform and then if that fails explain, etc
2017:11:13 20:23:30 deg @hiredman - I was looking for a way to avoid having to know about all the parts at one central place.
2017:11:13 20:25:52 deg That is, I don't want to say (s/keys :opt [ns1/key1 ns2/key2 ,,,]), but instead create the spec dynamically.
This seems trickier than it should be, since keys is a macro, so I can't just do a trivial atom/assoc/apply dance.
2017:11:13 20:46:29 hiredman it sounds like you want the already existing feature in spec where a s/keys spec will check specs for keys that exist in the map that have a spec even if they aren't part of the s/keys spec
2017:11:13 21:15:39 qqq I assume this is more appropriate for blog post reading rather than discussion:
is there a good writeup somewhere of qualified vs unqialified names for specs ?
Everything I've done so far in clojure is unqualified, i.e. :foo kws, but looking at spec, it seems like the default / preferred wa yis to be qualified i.e. ::foo or :blah.namespace/foo
2017:11:13 21:45:31 arohner I have a uuid. Is there a smart way to bias the generator so I can control the distribution of uuids? i.e. completely random vs. power law?
2017:11:13 21:46:07 arohner I’m aware of e.g. gen/elements, was wondering if there’s something else
2017:11:13 21:46:33 gfredericks you're trying to get collisions?
2017:11:13 21:46:58 arohner yes, some of the time
2017:11:13 21:47:29 arohner but I’d really like e.g. power laws on repeat UUIDs, so out of 1000 uuids, one appears 100 times, one appears 20 times, etc.
2017:11:13 21:48:47 arohner reading the spec source was informative. I’ll just with-gen with a UUID constructor
2017:11:13 21:49:39 gfredericks where will you get the args for the constructor?
2017:11:13 21:50:57 arohner type 5s are deterministic, and I suspect I can create v1s by specifying a unix time
2017:11:13 21:50:58 gfredericks if you use gen/large-integer and gen/fmap that to the type-3 uuid constructor, that will give you something like the distribution you want
2017:11:13 21:51:20 gfredericks keep in mind how distribution relates to size
2017:11:13 21:51:37 arohner can you expand on that?
2017:11:13 21:55:04 gfredericks I also talked about it some here if you liked faces and voices and slides and things https://youtu.be/F4VZPxLZUdA?t=25m26s
2017:11:13 21:56:47 arohner very useful, thank you
Am I using spec right nere? the vec2-new and bbox-new seem very UN-clojurish 'every is data' approach.
However, given that I'm using ::x ::y ::x0 ::y0 ... I don't know how else to nicely create vec2/bbox objects
2017:11:14 01:17:18 taylor that would obviate the tag-is thing. the *-new fns at the bottom seem like they’re just converting map keys from unnamespaced to the current namespace, why not use the namespaced keys upstream?
2017:11:14 01:17:35 taylor (using unnamespaced keys in your spec is also an option)
2017:11:14 01:33:18 qqq @taylor: because namespace 'object' is constructed != namespace 'object' is specced
2017:11:14 02:08:38 seancorfield @qqq In the upstream code, do you already have maps with unqualified keys? If so, where did they come from?
2017:11:14 02:09:50 seancorfield If you don't want the keys namespaced, change :req to :req-un (then bbox-new becomes just (assoc data :tag ::bbox))
2017:11:14 02:11:00 qqq @seancorfield: all code is code I have written (and can change -- am rewriting with spec -- and trying to figure out best practices)
2017:11:14 02:11:27 qqq I actually think qualified makes more sense than unqialified, since a word may mean different things n different namesapces
2017:11:14 08:16:27 deg @hiredman I did not know that, thanks!. Was it always the case that s/keys checks other keys too? Where is this documented?
Meanwhile, I realized that another part of the solution for me is to use s/merge. That way, the central spec needs to only about the existence of each module, not its behavior.
2017:11:14 20:06:23 ajmagnifico Question: In general, do you find it’s better to define specs in the namespaces that use them? Or in a separate “appname.specs” namespace?
2017:11:14 20:07:15 ghadi personally, separate
2017:11:14 20:35:03 ajmagnifico I think that the advantage of defining them in the namespaces where they are used is one of brevity.
2017:11:14 20:35:14 ajmagnifico For example, with nested data structures
2017:11:14 20:36:28 ajmagnifico this is less verbose than {:longappname.specs/a 1 :longappname.specs/b 2}
2017:11:14 20:36:43 ajmagnifico But the #: syntax only applies to the top level of the map
2017:11:14 20:36:47 taylor you can alias the namespaces for brevity
2017:11:14 20:41:18 ajmagnifico So, I’m loading a config file that looks like this
2017:11:14 20:41:26 ajmagnifico and I’m trying to spec it
2017:11:14 20:41:44 ajmagnifico I know I can use (spec/keys :req-un …)
2017:11:14 20:41:49 ajmagnifico But I’d like to use namespaces
2017:11:14 20:43:13 ajmagnifico Anyway, all of the pieces of this config are currently in my appname.specs namespace
2017:11:14 20:43:28 ajmagnifico When I load it and try to validate it, it fails, because all of my specs are namespaced
2017:11:14 20:43:55 ajmagnifico But I’d rather not have to put #:appname.specs in front of each map in the config
2017:11:14 20:44:55 ajmagnifico So if I use the alias strategy like taylor said, I’d need to choose an arbitrary alias ahead of time that I would use in the config file, and that would need to match up with an alias that I would create in the namespace where I’m actually using and validating this config data.
2017:11:14 20:45:17 ajmagnifico Or, if I defed the specs in the namespace where the data will be loaded and used,
2017:11:14 20:45:33 ajmagnifico then I can just put a :: in front of each of the keys and everything will work as intended
2017:11:14 20:46:05 ajmagnifico But am I going to get burned down the road if I have all of my spec definitions scattered throughout a bunch of namespaces instead of keeping them all in one location?
2017:11:14 20:46:34 taylor I guess I’d ask myself what problem am I solving by namespacing all the keywords in my config map
2017:11:14 20:46:53 taylor or is it instead creating more problems
2017:11:14 20:47:06 taylor where’s your config defined, how is it loaded?
2017:11:14 20:48:02 ajmagnifico Okay, that’s a fair question taylor. I guess at this point it’s me still trying to learn the idiomatic way of doing spec as it was intended by its creators. And they seem to think that namespaced keywords are a beautiful thing. I’m still trying to figure out how they’re beautiful. 🙂
2017:11:14 20:48:21 ajmagnifico best practices
2017:11:14 20:48:37 ajmagnifico so I’m wondering if “best practices” here are, like you said, creating more problems
2017:11:14 20:48:41 ajmagnifico or if I’m just doing it wrong.
2017:11:14 20:49:03 taylor I use specs for maps with unnamespaced keywords all the time, personally. I don’t think there’s anything necessarily wrong with it
2017:11:14 20:49:58 ajmagnifico Anyway, the config is stored in edn format in a hand-edited file, loaded with edn/read-string during app initialization
2017:11:14 20:50:22 taylor but then again most of the maps I work with are deserialized from some format that doesn’t support namespaces
2017:11:14 20:52:38 taylor and FWIW you could use any “made up” namespace for your config keywords, it doesn’t necessarily have to match a real namespace in your project
2017:11:14 20:53:03 ajmagnifico that’s an interesting idea
2017:11:14 20:53:20 ajmagnifico thanks for the discussion, taylor
2017:11:14 20:53:55 ajmagnifico I’ll probably just end up using :req-un
2017:11:14 20:54:11 taylor that’s what I’d do, but I’m lazy 😆
2017:11:14 20:54:20 ajmagnifico the hallmark of every great programmer
2017:11:15 08:40:50 tbaldridge @ajmagnifico something to consider, Spec also recommends flat maps:
2017:11:15 14:41:09 taylor (s/map-of int? map?) but you could get more specific (s/map-of int? ::some-complex-keys-spec)
2017:11:15 14:41:37 slipset I did the second 🙂
2017:11:15 14:50:01 ajmagnifico thanks @tbaldridge. That’s a really good idea, and it will actually solve a couple of other problems for me as well. I didn’t recall that keyword namespaces don’t actually need to correspond to real namespaces, even though that’s exactly what taylor told me! I guess I just needed to see it concretely for it to click.
2017:11:15 14:50:50 ajmagnifico I guess the only thing you miss out on when you make up namespaces is use of the :: syntax
2017:11:15 14:50:59 ajmagnifico but in my case, that’s a small price to pay
2017:11:15 23:27:10 dpkp I'm trying to spec a variadic function that has a gnarly java-esque method signature like (f str int)(f str int bool)(f str int bool int) . I've tried to model this w/ s/cat but I'm having trouble building a spec that says (f str int int) is invalid but (f str int bool int) is valid. Example:
2017:11:15 23:29:45 dpkp this works if args are supplied as nil. but I can't figure out how to also support the smaller arg list. (s/valid? ::foo ["a" 1]) is false
2017:11:15 23:29:48 dpkp any pointers?
2017:11:16 02:45:26 alexmiller Other suggestions were fine but also could use nested s/? for optional stuff
There is something wrong with this "pattern" for spec on multispec? Someone else doing something like this? Other options?
2017:11:16 16:37:07 gklijs Not really sure what your trying to do. @souenzzo
2017:11:16 17:16:15 souenzzo @gklijs I'm trying to make a spec to a multimethod
2017:11:16 20:00:36 jumar @souenzzo I don't have an experience with multi-spec per se, but if you are trying to to instrument multimethod, that's not currently possible with spec. You'll need to create a wrapper function for multimethod and spec that one.
2017:11:16 17:17:59 gklijs ok, no experience with that, I was busy using multi methods handling specs, so I can edit certain data based on it’s spec in the browser
2017:11:17 08:14:54 otfrom when writing the :fn bit of an fdef, doesn anyone have any tips or examples to figure out which bit of the fn failed if there are multiple properties that should hold?
2017:11:17 08:15:41 otfrom or with labels almost gives what I want, but obviously isn't logically correct. I want something like all
would be very tidy for checking a value was copied correctly in the function
2017:11:17 10:43:17 triss is there a way to write a spec that says: a collection containing two or more integers and anything else?
2017:11:17 10:46:03 gklijs would became something like, and it would need to be a collection, and when filtered on number? the size is minimal of 2
2017:11:17 11:09:43 tcoupland @triss sounds like an s/cat which makes use of some of the regexp elements
2017:11:17 12:15:20 Olical Does anyone know if you can spec a map and have it conformed into multiple parts depending on the key. For example, conform {:A 1 :B 2 :a 1 :b 2} into something like {:upper {:A 1 :B2} :lower {...}}. Basically a group-by during a conform.
2017:11:17 12:15:54 Olical I essentially want different rules for different kinds of keys in a map. So certain styles of keys would have different value requirements.
2017:11:17 12:17:51 andre.stylianos From what I've been seeing of the usage of spec, I'd guess not. Seems like it falls under the "data transformation" camp which is not something they intend spec to be used for.
2017:11:17 12:18:20 andre.stylianos but I might be wrong
2017:11:17 12:18:35 Olical Yep, that's how I was beginning to feel about it. I'm currently doing that transformation after the fact, but it's hard to find them in a recursive spec, so I can't do the transform up front ahead of time.
2017:11:17 12:18:43 Olical I don't want to walk the tree multiple times is all.
2017:11:17 12:19:09 Olical I would quite like different specs for different kinds of keys though, not sure if I can do that.
2017:11:17 12:19:36 Olical In my case, if it starts with on- I want it to be restricted to a function value. If it's anything else it should be a string.
2017:11:17 12:20:29 andre.stylianos makes sense. Does it have to be a tree? You could {:upper/A 1 :upper/B 2 :lower/a 1...}
2017:11:17 12:21:33 andre.stylianos and not quite sure what you mean by "different specs for different kinds of keys though, not sure if I can do that."
2017:11:17 12:23:07 Olical Yup, I'm not sure either 🙂 It's like, I don't care about the actual keys, as long as they're keywords. But if it starts with on- it should be a function. I wonder if I can combine map-of and keys. It's almost like I want two stages, conform as much as you can with this spec, THEN conform with this spec.
2017:11:17 12:23:33 Olical Right now I'm just doing map-of keywords to string or function.
2017:11:17 12:24:10 Olical I just need constraints between key and value. Otherwise if I exercised this spec it would generate invalid runtime data, but it would match the spec.
2017:11:17 12:24:55 andre.stylianos Yeah, for what you're saying it seems like map-of would be a fitting choice
2017:11:17 12:26:41 andre.stylianos But I'm not sure if there's a way to treat the key/value pair as a whole with map-of
2017:11:17 12:28:04 Olical I think I need a hybrid that doesn't exist 😅
2017:11:17 12:28:18 andre.stylianos Another option might be coll-of if map-of doesn't work, since that way you could probably spec each entry as [k v] pairs
2017:11:17 12:31:01 Olical For context, this is an attribute map that will be applied to DOM nodes in this little Reagent/React like thing I'm building.
2017:11:17 12:31:30 Olical So most keys will just be strings that go to setAttribute on a DOM node, but event listeners are special things that are handled separately.
2017:11:17 12:31:50 Olical I'm using spec as a parser of my hiccup like data structure basically.
2017:11:18 00:45:32 qqq does spec have a way to capture:
this object is an ATOM and when you deref the ATOM, it should satisfy this spec
2017:11:18 00:57:03 taylor I haven’t seen anything like that. I suppose you could wrap an atom and hook spec on derefs?
2017:11:18 00:57:29 qqq We can set a validator-function on the atom, so it'll auto reject on swap! / reset! if it fails the spec.
2017:11:18 00:57:42 taylor interesting
2017:11:18 00:57:49 qqq However, I now want a way to get the validator function of an atom, so we can say things like "assert that this atom has FOOBAR validator"
2017:11:18 01:01:35 danielcompton You also have the same problem for things like manifold deferreds
2017:11:18 01:03:29 qqq wtf are manifold deferreds? 🙂
3. How is this possible? if 23 is not a valid ::appState, shouldn't assert throw an exception ?
2017:11:18 03:57:39 qqq Resolved: I forgot to use (s/check-asserts .... )
2017:11:18 07:35:39 danielcompton There’s a CLJS port of it somewhere
2017:11:18 07:48:20 mattiasw I have a simple function that takes data, and creates test data from it. Currently, it uses (rand-int ..), what minimum changes do I need to do to this code, in order to use the same seed for random as the rest of the generators.
(defn create-update-operation
"Change or delete or add a new key to `m`.
In 30% of the operations, then value will be nil, i.e. delete."
;; Do not change or delete any of the pre-defined keys
([m] (let [res (create-update-operation (filter #(not (#{:id :tenant :entity :k :updated :version} (key %))) m) {})]
;; (println (count res))
res))
([m changes]
(if (<= (count m) 3) changes
;; find the next key to change (update or delete)
(let [nxt (rand-int (count m))
next (nth m nxt)
rest (nthrest m (min 1 nxt)) ;0 just mean that I am regenerating the first key.
k (key next)
_ (assert (some? k))
v (update-or-delete-value k)
changes2 (assoc changes k v)]
;; see if we should add a new key to the map
(if (< (rand-int 10) 2)
(recur rest (let [k2 (keyword (random-string))
v2 (update-or-delete-value k2)]
(assoc changes2 k2 v2)))
(recur rest changes2))))))
where update-or-delete-value also uses rand-int
2017:11:19 00:00:04 qqq Is there a better way to write: (s/def ::p1 #(s/valid ::vec2 %)) ?
2017:11:19 01:18:57 qqq In the spec guide, we see:
(s/def ::suit #{:club :diamond :heart :spade})
which asserts that item is ELEMENT OF given set
is there a corresponding builtin for
'item is SUBSET OF given set`
2017:11:20 05:08:42 qqq they very act of writing specs is making my code clearer
2017:11:20 05:08:54 qqq it's forcing me to think: what condition does this key have to satisfy to be valid, and documenting it as code
2017:11:20 07:39:47 qqq this is kind of insane, but is there a way to (while only holding weak references), tell clj/cljs to memoize calls to
(s/assert SPEC OBJ)
?
2017:11:20 12:20:08 gfredericks the jvm has a hashmap for that sort of use case I think
dunno how easy monkeypatching s/assert to get that effect would be
2017:11:20 12:20:27 gfredericks why're you calling multiple times?
2017:11:20 20:25:39 johanatan are there any known gotchas involving partial application around a stest/instrumented function ?
2017:11:20 21:15:00 hiredman it depends what you mean
2017:11:20 21:15:59 hiredman instrument works by setting the root value of vars to an instrumented value of the function, but if you have already taken the value of a var before instrumenting, of course that will have effect
2017:11:20 21:17:44 hiredman so if you have something like (def f (partial g 1)) and then later instrument, calling f won't call the instrumented version of g
2017:11:21 01:44:47 johanatan @hiredman yea, that makes sense. unfortunately wasn't my problem but i figured it out anyway (something unrelated)
2017:11:21 01:45:21 johanatan the error message i was getting from spec was unintelligible but i just massaged the code, fixed any issues i saw via manual inspection and it eventually went away
2017:11:21 03:02:16 johanatan narrowed it down to instrument actually doing more than I figured it would. looks like it exercises the instrumented with some values that the user itself isn't passing in. this was a bit surprising to say the least
2017:11:21 03:02:36 johanatan kept trying to find where the ((nil)) value was coming from in my code/ how that was even possible etc etc
2017:11:21 03:40:03 seancorfield @johanatan If you instrument a higher-order function -- one that takes a function as an argument -- then it will use generative testing, as I understand it.
2017:11:21 07:58:00 qqq with coll-of, is there a way to say:
this collection ca be empty, but it can't be nil
i.e. [] is okay, but nil is NOT okay
2017:11:21 10:53:46 triss hey all. Is there a spec for something I can use as a predicate for s/valid?
2017:11:21 10:54:05 triss I want a spec that matches both predicate functions and the names of specs.
2017:11:21 11:06:46 mpenet @qqq it works like this by default
2017:11:21 17:08:09 qqq @mpenet: yeah, thanks, I should have run that test, turns out bug was elsewhere and I was misattributing it
2017:11:21 18:32:44 johanatan @seancorfield it was a higher-order function in this case. any idea the reasoning behind using generative testing for this case?
is it just a 'hack' since the function inputs cannot be inspected to a level of detail necessary for verification?
2017:11:21 18:46:26 seancorfield I thought that was explained in the Spec Guide on the web site @johanatan? I certainly wouldn't say it was a 'hack'. It seems perfectly reasonable to me. How else would you verify that a function you're passing as an argument conforms to its spec in this situation?
2017:11:21 18:47:33 mpenet It s doable at invoke time
2017:11:21 18:47:51 mpenet There is/was a ticket about this
2017:11:21 18:49:29 mpenet I personally wished it was like this, now I almost always spec hof as ifn? for instrumentation because of this, which makes it a bit useless
2017:11:21 18:50:31 mpenet Imho both make sense in some cases, we should just have the choice
2017:11:21 18:58:03 mpenet It was clj-1936 not sure if there is a new ticket started from the discussion there
2017:11:21 19:09:34 johanatan @seancorfield it probably is explained but i haven't read the guide from beginning to end so could've missed it. well, yea, in a dynamic language this probably can't be done (except perhaps by inserting metadata onto anonymous functions at their definition sites and then reading it back out at the spec check sites [which now that I mention it, does sound like a better way if it would be possible to impl])
2017:11:21 19:10:49 johanatan [but by 'hack' i meant, an unfortunate edge where the abstraction leaks due to limitations of the underlying system being modified]
2017:11:21 19:13:44 johanatan @triss isn't spec? what you are asking for?
2017:11:21 22:05:00 triss @johanatan Not quite.
2017:11:21 22:05:14 triss I’m validating on (s/or :fn fn? :keyword keyword? :spec s/spec?) is there a neater way?
2017:11:21 22:09:21 qqq I'm starting to love spec's dynamic attributes.
2017:11:21 22:09:37 qqq I'm doing some GUI layout, and I can write a spec of :assert that the sum of the width of the elemes of this vector is <= width of the screen
2017:11:21 22:09:51 qqq hit is nearly impossible to do in static type systems
2017:11:21 23:16:25 seancorfield @triss Do you want to validate that something is a registered spec or that it looks like it could be a spec?
2017:11:21 23:16:59 triss is a registered spec would be best for me
2017:11:21 23:17:17 seancorfield For the former, you could use #(try (s/get-spec %) true (catch Exception _))
2017:11:21 23:17:35 seancorfield That would be false for predicate functions tho'...
2017:11:21 23:17:55 seancorfield ...but it would validate that you actually had a named, registered spec (keyword).
2017:11:21 23:18:29 triss ah i see. that’s a handy snippet fro something no doubt…
2017:11:21 23:18:53 seancorfield (so I guess you'd still want (s/or :fn fn? :spec #(try ...)) ?)
2017:11:21 23:19:01 triss I’m gonna have to scratch my head for a bit about what I want to match on really… I want anything that woks as the first argument for s/valid?
2017:11:21 23:19:12 triss where is the spec for s/valid?
2017:11:22 03:37:25 alexmiller you can’t actually spec most of the functions in spec without creating an infinite recursion when instrumented
2017:11:21 23:20:36 seancorfield Based on the docstring for s/valid?, it doesn't look like there's a spec registered for it.
2017:11:21 23:22:02 seancorfield The source shows it calls specize on its spec argument and that in turn is
returns false. I expect it to return true. It appears to be eating the assertion error. Is there a way to get a more helpful msg on why it's failing the predicate ?
2017:11:22 03:15:14 qqq reframing question: when using predicates with clojure spec, is there a way, instead of just returning true/false, also return a error msg on false ?
2017:11:22 03:38:26 alexmiller currently, no
2017:11:22 04:01:31 qqq is this a short coming of current spec design, or is this because I'm mis using spec and should be using spec primitives as much as possible instead
2017:11:22 04:16:40 seancorfield One thing I've found helpful @qqq is to break predicates down into smaller components and give them names. That way the explain refers to the named predicates. In your case, I'd write data-is-vector?, data-has-ys-rows?, and data-has-xs-cols? and have those accept the whole hash map and return true/false. Then you'd have (s/and (s/keys :req-un [::xs ::ys ::data]) data-is-vector? data-has-ys-rows? data-has-xs-cols?) and I believe you'll get better error messages -- without using assert.
2017:11:22 04:19:53 seancorfield The other thing we do is take the explain-data and parse it so we can map symbols and forms to English error messages (well, actually to i18n keys so that we can produce error messages in any language!).
2017:11:22 08:50:41 hkjels So, I’ve written quite a lot of ui-components where I use spec for various things. Mostly checking that parameters used with the components are valid. I’ve come to a stage where I’m doing performance-testing and it seems that spec is sucking the life out of this project. Is it just a bad idea to use spec for such a task?
2017:11:22 13:36:39 alexmiller Yes
2017:11:22 09:00:27 andre.stylianos I believe the encouraged way is to use spec for production code only at the boundaries where you receive data from sources you don't control. Beyond that, the validation that you do for internal code should be used only for development.
2017:11:22 09:14:12 hkjels OK.. Then I’ll have to use something like spec/nonconform all over the place in dev then
2017:11:22 09:14:59 hkjels I guess it makes sense to not have it on in production
2017:11:22 10:34:09 ikitommi @hkjels just turn off the validation for real app? e.g. instrument the component functions for dev. Or use assert and disable that for prod. Haven’t used spec for ui but did that at some point with Schema. WIth it, you can disable function validations and the schematized defs and fns will not emit any code related to Schemas => no penalty.
2017:11:22 11:26:32 hkjels When I disabled validation, it definitely got a performance boost
2017:11:22 11:27:24 hkjels conform is still one of the heavier tasks in the performance-tab og chrome, but it’s also doing quite some work, so that makes sense I guess
2017:11:22 11:28:11 hkjels I have to do some more testing, but this might do
2017:11:22 12:00:42 guy When you fdef a function with no args, do you just omit the :args key?
2017:11:22 13:35:49 alexmiller No, you should use (s/cat) to validate 0 arity
2017:11:22 13:45:46 alexmiller Also, you should be suspicious of 0 arity functions. Either they have side effects (and maybe shouldn’t be spec’ed) or you can just use a def instead.
2017:11:22 13:51:13 guy Thanks!
2017:11:22 13:38:25 otfrom @seancorfield and @alexmiller are there any good examples out there of using relatively complex :fn functions in fdefs? I saw that and hints above and wonder if that is the best way to go to figure out which bit of the property is being violated
2017:11:22 13:42:22 alexmiller I’ve written some here and there but not sure if in anything public. If they get too complicated, then you have to question whether that’s the best way to test. Sometimes straight test.check is better
2017:11:22 13:56:54 otfrom cool, that makes sense
2017:11:22 17:31:27 qqq I realize this is a bad idea in most cases, but is it possible to say:
(s/maps ... key :buttonContainer satisfies spec ::container) ?
2017:11:22 18:03:06 alexmiller Don’t understand the question
2017:11:22 18:07:35 bronsa are you looking for :req-un?
2017:11:22 20:55:57 alexmiller FYI, I’ve been working on fix the auto doc junk for Clojure and contrib and the docs have been updated for Clojure, spec.alpha, and links in the spec guide
2017:11:22 21:07:30 seancorfield @alexmiller That's great to hear -- thank you! Is there anything I can do to help get https://clojure.github.io/java.jdbc/ updated too?
2017:11:22 21:10:23 alexmiller nope. I can try to run it manually though…
2017:11:23 16:26:10 qqq I have a number of specs:
::Foo ::Bar ::Cat ::Dog
if I do (s/keys :req-un [::Foo ::Bar ::Cat ::dog]) the keys of the map become :Foo :Bar :Cat :Dog. Is there some way to redo this so the keys become :foo :bar :cat :dog instead ?
2017:11:23 16:50:58 misha (s/def ::foo ::Foo)
2017:11:23 16:52:11 misha And then use ::foo in :req-un
2017:11:23 16:53:26 qqq obvious yet brilliant
2017:11:23 20:32:00 qqq (s/valid? any? nil)
what is idiomatic way to say: this can be anything so long as it is not nik ?
2017:11:23 20:32:29 madstap @qqq some?
2017:11:23 20:33:02 qqq (some? false) => true -- this is surprising
2017:11:23 20:47:25 guy
(defn some?
"Returns true if x is not nil, false otherwise."
{:tag Boolean
:added "1.6"
:static true}
[x] (not (nil? x)))
2017:11:24 02:52:12 johanatan I'm getting ExceptionInfo Couldn't satisfy such-that predicate after 100 tries. clojure.core/ex-info (core.clj:4744) for a spec generator that seemingly should be fairly trivial:
2017:11:24 02:52:47 johanatan [I added the with-gen on snake-cased-alpha-numeric with it being the most complicated primitive in there and still get the error]
2017:11:24 02:58:11 danielcompton @johanatan can you generate any of the leaves?
2017:11:24 03:01:20 johanatan perhaps there is a bug where my regex doesn't match values generated with the with-gen-specified generator
2017:11:24 03:01:42 johanatan i've seen that error when the two are mismatched: i.e., when the spec and the generator for it are not perfectly aligned on what is acceptable or not
2017:11:24 03:02:45 gfredericks I forget what the implications of jinx are
2017:11:24 03:02:51 gfredericks do I have to not talk now for some time period
2017:11:24 08:58:15 danielneal I think you can't talk until someone says your name three times
2017:11:24 12:07:34 gfredericks the same person? back-to-back in quick succession?
2017:11:24 13:49:49 danielneal aw you're right that's totally underspecified
2017:11:24 13:53:45 danielneal oh wait
2017:11:24 13:53:46 danielneal my mistake
2017:11:24 13:53:48 danielneal
Traditionally, a jinx is ended when anyone speaks the jinxed person's name. However, a common variation says that only the jinxer can free the jinxee from their obligation to remain silent. (This is sometimes called a "private jinx" or "jinx personal lock".)
2017:11:24 19:08:21 gfredericks a private jinx sounds a lot less interesting
2017:11:24 03:03:01 johanatan hmm, yea, i'll give that a try. thx!
2017:11:24 03:08:57 johanatan that fixed it. was able to successfully generate 1000 at once
2017:11:24 03:10:52 johanatan there is a bug in my regex though. noticing that pipes are ending up in the output. probably was the original problem
2017:11:24 03:15:48 danielcompton buy me a coke I think, although I probably owe you the drink. Edit: this was about the jinxing
2017:11:24 03:17:03 johanatan haha, definitely. i would even get you a beer if you prefer
2017:11:24 03:17:21 johanatan this was my intended regex, btw: #"[a-z0-9\_]+"
2017:11:24 03:17:42 johanatan [no pipes]
2017:11:24 05:20:29 qqq does clojure have any naming convention regarding fn names that end in a '-' ?
2017:11:24 05:21:08 qqq for ecample, I have
(defn node->svg ...)
and I would like to define a helper function named
(defn node->svg- ...)
but intuitively, functions that end in '-' seem like a bad idea
2017:11:24 05:33:00 seancorfield The most common convention seems to be to end in a *
2017:11:24 05:33:51 seancorfield @qqq I've seen that in a lot of libraries -- and started following it myself.
2017:11:24 12:06:57 gfredericks I have the uncommon preference for foo', imitating mathematical notation
2017:11:24 13:34:56 guy Hi has anyone made a spec for a base64 encoded string before?
2017:11:24 13:35:19 gfredericks that sounds like it could be a regex
2017:11:24 13:35:33 guy ooh yes
2017:11:24 13:35:36 guy thats a smart idea
2017:11:24 13:35:39 gfredericks though perhaps not a trivial one if you want to really follow the spec
That is, my ::field-def expects a tuple of keyword and field-type (a set of keywords). So I'd expect s/cat to consume the first value from a list, then apply ::field-def as a predicate and deconstructor to it. Instead, it is applying ::field-def directly.
From my (mis-?) understanding, I'd expect the following to succeed, but it fails:
Again, my expectation is that s/cat consumes a seq. The first element in the seq should be a ::field-def, which itself is a seq of two elements. Instead, it is matching the treating the entire list [[:foo :text]] as the field-def tuple, and failing because the first value in the list, [:foo :text] is not a keyword.
2017:11:24 23:31:01 hlship Basically, my goal is this:
That is, a single field def, followed by zero or more cluster-def's.
2017:11:24 23:35:52 seancorfield @hlship Regex specs "unwind" when they are combined, unless you wrap them in s/spec (or some other delimiting operation).
2017:11:24 23:36:41 hlship Ok, that could be what I'm missing. It's right there in the docs, once I know what to look for!
2017:11:24 23:37:22 seancorfield s/cat isn't meant for tuples -- there's s/tuple for that
2017:11:24 23:37:52 seancorfield (although you'll get no labels with that I think...)
2017:11:24 23:38:37 hlship Nice. I don't need the conformed value, I'm just validating.
2017:11:24 23:39:08 seancorfield I think it catches everyone out at least once 🙂
2017:11:25 10:17:23 qqq is there a way to hijack s/assert 's error messages?
I often have situations where:
1. obj is a giant data sttructure
2. obj fails spec ::foo
3. the fact that obj fails can be verified by just loloking at (keys obj)
4. s/assert prints out all of obj, which actuall makes it more difficult to see what is going on
2017:11:25 22:48:04 bbrinck You can also customize the printer e.g.
my-printer will take a single argument that is the explain-data (it’s data like the return value from s/explain-data).
2017:11:25 23:22:07 qqq this looks better than what I asked for
2017:11:25 23:22:20 qqq so I get to take the assertion failure, as clojure data, rewrite it, then print it out
2017:11:25 23:22:42 bbrinck Yep, you can format it however you want
2017:11:25 23:23:15 bbrinck Also, if you happen to want an out of the box experience, you can use expound
2017:11:25 23:23:58 bbrinck It doesn’t actually help in this specific case (it prints out the entire value when it is missing keys), but you can provide a custom function to print out the value in this specific case
2017:11:25 23:24:40 bbrinck in that case, you’ll get the spec-name and some other data, which might be enough to detect this particular case.
2017:11:25 23:25:33 bbrinck Or you could even write your own printer which inspects the explain-data and does something custom for keys specs, but passes it along to expound otherwise.
2017:11:25 23:42:29 qqq when an assertion happens, I want it to pop up a new browser window, print the first level of the data, have + buttons next to parts that can be expand (and expands when I click on them), and highlights in a red DOM rectangle the key/value pair that is failing the spec
2017:11:25 23:42:35 qqq someone needs to build that 🙂
2017:11:26 11:36:16 qqq (s/keys ::req []) <- notice the :: instead of :req
is there a way to have that throw an error ?
sometimes, I define that, then I create objects, and they satisfy the spec ... because it's a map with no requirements since ::req is silently ighnroed
2017:11:26 13:42:20 alexmiller Sure, file a jira
2017:11:26 14:50:48 gfredericks would that not go against the "accept arbitrary extra keys" philosophy of spec?
2017:11:26 18:08:34 alexmiller I wouldn’t necessarily put it in the spec for keys, but it’s reasonable to validate it in the function. It’s not like this is getting passed to anything else.
2017:11:26 18:09:24 gfredericks I spose I had always interpreted the openness as partially targeting forwards-compatibility
2017:11:26 18:10:33 alexmiller It’s still forward compatible to accept more stuff in the future
2017:11:26 19:17:59 gfredericks right, but then code targeting the future API can't use the older version if it happens to be around
maybe that's not a useful thing? you just want backwards compatibility so you can use the newest of all the versions that each part of the code wants?
2017:11:26 21:21:08 alexmiller I would say that’s a non-goal
@qqq
@gfredericks sure. But you can do 🙂
Maybe in future versions of spec, this spec will be wrong. But for now, it's a usefull build-time error.
2017:11:26 17:57:06 gfredericks but there's a difference between spec setting that spec on s/keys and a user doing it themselves
2017:11:26 19:54:27 arohner Before I go write one, is there a generator for specs? i..e one that will generate (s/cat :x int?)
2017:11:26 21:24:01 alexmiller The spec specs in CLJ-2112 are imo the right direction for that. The specs there are not yet good enough to be used for this in general
2017:11:26 20:56:13 gfredericks once there's a spec spec you could presumably get a generator from it
2017:11:26 20:56:53 gfredericks shouldn't spectrum have a spec spec already anyhow?
I say that as someone who has never used spectrum
2017:11:26 23:02:00 arohner It only needs s/spec? as a spec when using spectrum. To test it, I want a spec generator
2017:11:26 21:00:34 taylor curious if there’s been any thought/discussion/work for sharing specs (besides putting them in a shared library)
2017:11:26 21:02:55 taylor something akin to XSD? I suppose it’d be limited to what can be expressed in whatever format
2017:11:26 21:22:23 alexmiller The whole point of using fully qualified namespace keys is that everyone can share specs
2017:11:26 21:35:55 taylor yeah, I guess my curiosity is more about alternative mediums/mechanisms for sharing specs?
2017:11:26 21:36:00 gfredericks but as soon as you use custom predicates, you need to share code as well
2017:11:26 22:56:37 alexmiller Which is also fully qualified
2017:11:26 21:38:42 taylor I’m thinking of something conceptually similar to Swagger I guess. And yeah it would be a pretty limited subset of spec’s abilities.
2017:11:27 04:34:19 fedreg Is there a way to get spec information similar to how I can use meta to get info on a function? So if I have a spec like (s/def ::example string?) is there some function I can call to see the spec definition? Even better, is there anything that will give me a list of specs in the namespace? Thanks!!!
2017:11:27 15:59:15 alexmiller yes, you can call (doc ::example)
2017:11:27 15:59:40 alexmiller you’d have to write something to filter specs per namespace
2017:11:27 07:33:22 misha s/form
2017:11:27 07:34:31 misha s/registry
2017:11:27 07:34:45 misha @fedreg
2017:11:27 12:50:25 ikitommi @taylor related to “similar to swagger” - if specs could be read back from their s/form without eval (I think it could be done already with undocumented part of the spec - but Rich might be doing something functional for this?), we could have a pragmatic subset of specs that could be safely used between servers & clients. and build swagger-like docs for them.
2017:11:27 12:54:57 ikitommi we could publish the s/form as vendor extension to the spec-tools swagger-docs. e.f. :x-spec "(clojure.spec/keys :req [:user/x])" + :x-specs on top-level as the spec-registry for the related specs. Non-clojure clients could use swagger/openapi and clojure-clients just the spec-part.
2017:11:27 16:34:31 r0man Is this a bug?
2017:11:27 16:34:33 r0man (if-let [x false] x :my-ns/invalid)
(if-let [x false] x :clojure.spec.alpha/invalid)
2017:11:27 16:34:54 r0man the first works, the second raises an exceptions from spec
2017:11:27 16:35:00 r0man Call to clojure.core/if-let did not conform to spec: In: [2] val: :clojure.spec.alpha/invalid fails at: [:args :else] predicate: any?
2017:11:27 16:36:54 r0man the second form works also in Clojure 1.8
2017:11:27 16:39:10 r0man oh, and I was trying the above with Clojure 1.9.0-RC1
2017:11:27 17:00:44 qqq is there a way to say:
::foo is a COLLECTION
such that (map :get-some-key ...) creates an unique list
and if this fails, print out the entire collection
2017:11:27 21:20:49 fedreg Thanks! Completely missed that registry function.
2017:11:27 22:29:05 j-po Is there a minimally ugly way to compare specs from two projects (like two versions of the same codebase)? One idea that comes to mind is loading one set of specs into a separate registry, but that looks like it's been made deliberately hard. Is there maybe a way to blow away the spec registry entirely to restart from scratch?
2017:11:27 22:31:52 bfabry specs are fully namespaced with the idea that they'll be unique across projects, so caveat is anything you do that's assuming different is going to end up weird. but, if you want, I'm pretty sure you could just reset! the atom that is the spec registry
2017:11:27 22:37:27 gklijs @j-po maybe first load the old one, write all the forms to a map, and serialize. Then start again with the new one, and compare against the stored map?
2017:11:27 23:05:30 j-po Thanks!
2017:11:28 10:37:50 otfrom I'm creating a look up hash map from a vector of records where the keys will be strings based on what I want to look up. Anyone seen an example of how to spec this? I can't think of how to do it with s/keys
2017:11:28 11:23:59 guy hopefully thats helpfully! 👍
2017:11:28 11:25:03 otfrom it is
2017:11:28 21:45:46 sekao is there any library that can "unroll" a spec so i can display a given spec with all its component specs in a single data structure?
2017:11:29 16:19:52 acron Re spec, should Clojure ensure that keys referenced in s/keys are registered? I just noticed an unregistered key in one of mine as was surprised I hadn't been made aware by the compiler
2017:11:29 16:20:54 mpenet it allows that so that you can define key sets before the key specs themselves, most of the stuff in spec in wrapped in delays
2017:11:29 16:21:22 mpenet I could be wrong, but this sounds like the "issue"
2017:11:29 16:22:04 acron I understand, kinda makes sense.
2017:11:29 16:24:48 acron Thanks @madstap this is useful
2017:11:29 19:13:09 misha is there a good example of a generator, which generates map (record) where values of 2 fields are dependent?
multispec solves opposite problem: conform/validate records with 2+ dependent fields, can it be leveraged for generating generator?
2017:11:29 19:17:59 misha the nuance in a data model I have, is that :type key on record determines not presence/absence of an attribute, but shape of a particular attribute:
so I am already jumping some hoops to create multispec for this (extra level of spec indirection between data shape and attribute name)
2017:11:29 19:20:23 misha would like to avoid having custom monster generator, if possible at all
2017:11:30 00:01:40 shaun-mahood I'm trying to hook a custom generator to a spec and I can't quite get it working - I suspect it's something simple that I'm missing. Details in thread.
2017:11:30 00:01:40 shaun-mahood I'm trying to hook a custom generator to a spec and I can't quite get it working - I suspect it's something simple that I'm missing. Details in thread.
2017:11:30 00:01:57 shaun-mahood My custom generator is
2017:11:30 00:04:24 shaun-mahood Spec is (s/def ::plan (s/with-gen string? gen-7-digit-plan))
2017:11:30 00:05:26 shaun-mahood Error message starts with #object[TypeError TypeError: self__.gfn.call is not a function]
2017:11:30 00:06:27 shaun-mahood What I'm actually looking for is a 7 digit string with a range from 0000000 to 9999999 - probably there's a way better way to do this.
2017:11:30 00:06:59 taylor (s/def ::plan (s/with-gen string? (constantly gen-7-digit-plan)))
2017:11:30 06:59:19 misha @U054BUGT4 I might be wrong, but it seems like every gen override place in spec expects no args fn returning actual generator, hence “constantly” above.
2017:11:30 07:00:19 misha e.g. s/exercise is one of those places
2017:11:30 05:09:37 quan I'm trying to spec clojure.string/index-of
but running into this weird exception when instrument
(stest/instrument)
(clojure.string/index-of "abc" "a" 0)
;=> CompilerException java.lang.ClassCastException: clojure.spec.test.alpha$spec_checking_fn$fn__2943 cannot be cast to clojure.lang.IFn$OOLO
2017:11:30 05:09:44 quan not sure what's wrong here?
2017:11:30 06:13:37 seancorfield @quan I believe that happens when you try to instrument a function that has arguments type-hinted to primitives. The IFn$OOLO type is (Object, Object, long) -> Object and is optimized for the primitive argument and that breaks spec. It's a known issue (I don't recall the JIRA issue number).
2017:11:30 06:17:35 quan ah yes, from-index has ^long hint
2017:11:30 06:25:49 shaun-mahood I want to use spec generators to build sample data when I'm prototyping a front end app. The generators return nil when I try to use them at runtime after requiring the file, but they work fine if I run them directly in the repl. Is there a good way to get these to work at runtime?
2017:11:30 14:55:55 mikeyjcat Hi. I’m using spec to validate data being prepared for transactions on datomic, including tempid’s. All works good. There is now a spec registered for :db/id that checks to make sure it is a instance of datomic.db.DbId. Wonderful, and I even created a custom generator. However, now when I am using spec with data read in, I have data with :db/id 123345, rather than a instance of datomic.db.DbId. This obviously fails the spec in instrumentation. It seems as though I need two versions of the spec for :db/id but this cannot be, in my understanding. Anyone tried anything similar?
2017:11:30 15:59:46 alexmiller your spec is not capturing all the possible forms of :db/id
2017:11:30 19:18:15 tcoupland has anyone got a tidy way of writing core.test is checks of spec/check? I've got something that works, but it's pretty ugly!
2017:11:30 20:00:53 taylor is it uglier than this?
2017:11:30 21:16:09 tcoupland that's what i'm using at the moment
2017:11:30 22:01:25 taylor nice
2017:12:01 09:21:31 tcoupland when i get a minute i might have a play with sumarize, might give a clearer failure message
2017:12:01 05:25:28 kumarshantanu Hi, has anybody successfully used spec for web validation and translated the validation failures to HTTP 400 for nested data models? Any example or references would be great to have.
2017:12:01 06:47:34 ikitommi Don’t know about success, but there are routing libs like compojure-api and reitit, which support spec as web (backend) validation. There is an example in https://github.com/metosin/c2. For invalid input, produces http 400 with this kind of body:
2017:12:01 06:54:02 kumarshantanu We are using a different data-driven router, so looking for a standalone validation mechanism using spec. Is there any such library?
2017:12:01 06:59:26 ikitommi Well, reitit is kinda modular, if you take the metosin/reitit-spec is’s flattened deps tree is: spec-tools, clojure.spec & meta-merge
2017:12:01 07:00:32 ikitommi it bring the ring-module but it has no dependencies. Could move the Coercion protocols into own module.
2017:12:01 07:02:35 ikitommi do you think the coercion should be totally out of reitit (modules)? I would be more polite towards other routing libs at least.
2017:12:01 07:04:06 kumarshantanu Thanks, let me try out Reitit-spec. I’m sort of new to spec. And yes, compat with other routing libs would be very useful.
2017:12:01 07:05:05 kumarshantanu Though can’t comment now about code organization.
2017:12:01 07:08:56 ikitommi there is already compat, but needing to include another routing lib to get coercion for another is kinda odd. Plan is to ship coercion interceptors too (as they don’t require any new deps), so they should be usable from “pedestal-style interceptor web libs”
2017:12:01 07:10:12 ikitommi are you using mw or interceptors? and if interceptors, which kind?
2017:12:01 07:11:05 kumarshantanu I am using Ring with a routing lib, but right now just using plain app code to do validation (it’ a new project) - no m/w for now.
2017:12:01 14:13:06 mpenet another thing I prefer not to use lately: custom conformers 🙂
2017:12:01 14:15:21 gfredericks speaking of schema-related transformations, I've been experimenting with reviving the stuff in https://github.com/gfredericks/schema-bijections as a library that only deals with bijections, not referring directly to schema/spec at all
I'm curious how useful something like that would be for transforming data to the format that a spec expects (and back? e.g. jdbc)
2017:12:01 14:16:04 gfredericks I think generating bijections from specs would be the ideal result
2017:12:01 14:16:11 mpenet that'd be useful
2017:12:01 14:16:30 gfredericks I still have to decide if/how to incorporate surjections though
2017:12:01 14:16:34 mpenet I tend to write this kind of stuff with specter these days, but that's quite gory
2017:12:01 14:16:37 gfredericks since those are also useful in some cases
2017:12:01 14:19:09 mpenet tbh I didn't really like the form it took in schema, but that was practical. I am not sure I have seen a solution I like for these kind of transformations yet
2017:12:01 14:19:36 gfredericks the form what took in schema?
2017:12:01 14:19:42 mpenet coercers
2017:12:01 14:20:56 gfredericks oh right -- I don't like them because they represent surjections everywhere, even for uses when bijections can work
and it makes it easy to test your function-that-coerces-its-inputs with the canonical form of the data, since that's easier, and never actually test the production form that has to be coerced
2017:12:01 14:22:25 danielneal bijections from specs would be awesome ➕
2017:12:01 14:23:16 gfredericks I am envisioning that you can define different styles for other representations, like json and jdbc, and then get bidirectional functions for converting your canonical spec form from one to the other
2017:12:01 14:23:35 gfredericks e.g., "I want camel-cased keys, I want date-times serialized like this" etc.
2017:12:01 14:24:06 gfredericks that category of things, but hopefully arbitrarily flexible
2017:12:01 14:24:59 gfredericks so if you say your json representation has camelCased keys while your internal representation is kebab-cased, then a json input with kebab-cased keys would be considered incorrect and cause an exception rather than just silently be accepted
2017:12:01 14:27:37 mpenet There was a hint of a next iteration of spec coming out at some point (I think in Rich's talk), I am wondering if there are changes related to these kind of things (and others often mentioned).
2017:12:01 14:28:23 mpenet I personally am curious to see how spec forms conforming (or whatever replaces it) will take form
2017:12:01 14:29:39 mpenet related to CLJ-2112
2017:12:01 14:33:51 gfredericks oh yeah, I forgot about that
2017:12:01 14:59:56 mpenet and specs metadata too, could help for a lot of things
2017:12:01 15:17:57 gfredericks you mean enabling specs to have metadata attached?
2017:12:01 15:18:14 gfredericks I assume they already can so you must mean something else
2017:12:01 15:19:16 mpenet you can't attach metadata to specs atm
2017:12:01 17:35:38 ikitommi @alexmiller related to the s/merge problem, there is a PR request in spec-tools of a merge that seems to fix the problem. Could that be pushed to Spec itself? Comments welcome: https://github.com/metosin/spec-tools/pull/91
2017:12:01 17:36:10 alexmiller fixes what problem?
2017:12:01 17:37:49 ikitommi My comment on 15:53, that can’t be right?
2017:12:01 17:38:57 ikitommi e.g. merge merges that values, so the unconformed value override the conformed ones.
2017:12:01 17:41:34 alexmiller it doesn’t override, it’s just that only the last spec in the merge controls the conformed result
2017:12:01 17:42:32 alexmiller that is the intent, not a bug, but it can have some non-obvious effects, particularly with unqualified keys
2017:12:01 17:43:14 alexmiller I think there is a ticket for this already in jira, but not sure what should be done with it
2017:12:01 17:44:17 ikitommi oh, so that works with qualified keys. That’s interesting.
2017:12:01 17:48:28 ikitommi Woudn’t @arttuka’s PR fix that? Could the issue be re-opened?
2017:12:01 17:51:43 alexmiller I wouldn’t re-open that, but if there is a good statement of a problem and a patch, we can take a look at it
2017:12:01 17:54:44 alexmiller that is, in a new ticket
2017:12:01 17:52:13 alexmiller I have no idea what that PR is doing at a glance
2017:12:01 17:53:42 alexmiller Rich has a pretty extensive re-work coming for spec (post 1.9 release) and I suspect we won’t really look at any fixes till we’re on the other side of that
2017:12:03 16:32:09 danielneal Ah interesting! What kind of rework? Is it on the implementation side or is there going to be some kind of change to the interface?
2017:12:01 17:58:10 ikitommi ok, I’ll ask Arttu to post a new issue out of that. And looking forward to the re-work, whatever that is. thanks!
2017:12:02 11:53:01 gklijs I had an issue with a spec I defined in a .cljc file. I use an or to make one of two keys required. That’s working fine. But when I inspect the form in clj the or is without namespace, while I was expecting the namespace to be clojure.core.
2017:12:02 13:43:41 alexmiller What do you mean by “inspect”?
2017:12:02 14:15:05 gklijs I debugged by adding logging. (namespace x) is empty in the .cljc one, but clojure.core when the spec is defined in a .clj file.
2017:12:02 15:52:07 alexmiller I still don’t understand a) what you’re doing b) what you’re seeing or c) what you expect to see
2017:12:02 16:28:00 gklijs What I do is have a library which for a subset of what’s possible to spec, transform the namespaced data into a vector without namespaces in such a way that I’m able to get back to namespaced map from the vector. Part of this is checking the form of the spec used. When you use clojure.spec.alpha/keys, you can use clojure.core/or to be able to spec “one of these should be there”. Now when the spec id defined in a .clj file the or is namespaces eith clojure.core, but when the spec is defined in a .cljc file the or has no namespace. I now solved it by looking only at the name of the or symbol. So there is no real problem anymore, but I was surprised this was happening.
2017:12:03 15:56:55 ikitommi @kumarshantanu did an example of normal ring app with the spec-coercion via a middleware
2017:12:05 00:03:41 drewverlee Can anyone recommend any good resources on clojure spec, or property based testing concerning side effects (like reading, writing from files).
I just watched “A Deep Specification for Dropbox - Benjamin Pierce”
: https://youtu.be/Y2jQe8DFzUM and have to assume the ideas from it were a motivation for clojure spec (given the timing). In the video, their is an argument that that deep specifications lead to more reliable software, which i agree with. He also argues, that this requires a lot of time and effort, which, also makes sense.
My experience with spec so far has been that its hard to see how you can adapt them to handle side effects (writing to files, calling apis, dbs, etc…) In the talk, the side effects are handled by observing them, and checking if the observations follow a specification. In general, is the approach you need to take if you want a specification over side effects?
My intuition is that spec is more about modeling the non side effect parts of your application. But it feels like so much work is generally done as side effects, that the cost is fairly high to create the specs and generators that “stub” out the side effect. Or really, what an “effective” stub would be.
2017:12:05 00:22:40 hiredman that youtube talk is more a long the lines of using test.check directly, not using spec
2017:12:05 00:24:54 hiredman I am sure some people are using spec for that kind of thing, and you might use spec as a nice dsl for creating generators for that, but you wouldn't use spec's instrumentation stuff to do that kind of testing
2017:12:05 00:30:34 hiredman in general, when I am trying to test a whole process, I use test.check to generate operations, feed them in to the process, then compare the result with what my model says it should be. spec (at least how I use it, and there are a lot of parts to spec so it can be used in a lot of different ways) is more about describing data that is moving around in the system. and I end up using those descriptions in assertions to make sure what I expect is there, to parse/recognize which of some alternative data a function was passed, or to generate other descriptions of the data for other tools (schemas, serializers, etc). I pretty much never use spec to annotate a function and then instrument that function.
2017:12:05 00:36:59 hiredman I think generative testing splits in to two classes. Class 1 is what you see in simple examples on line, you have a pure function that does something to a list of integers, so you generate a list of integers and pass it in and assert some property. Class 2 is like what you see in that dropbox video (which is very good), you end up generating a program (a sequence of operations) that you run on the system you are testing and then also against some reference implementation (say for a distributed key value store, the ref impl might be an in memory hashmap), and compare the results.
2017:12:05 00:37:27 hiredman spec's instrument and generative stuff seems mostly aimed at class 1
2017:12:05 00:37:42 hiredman class 2 is definitely a big investment in testing
2017:12:05 00:52:12 drewverlee @hiredman thanks, I saved your comment and I'll get around to reading it in a bit 🐕
2017:12:06 00:03:56 zalky Hey all: I'd like make some spec assertions regardless of whether check-asserts has been set. It's important these assertions are always made. To that end I'm looking to use assert*. It's part of the public api, but the doc-string says Do not call this directly, use 'assert'.. Is there something important that I need to know? Seems like it fits the bill otherwise.
2017:12:06 00:23:29 olivergeorge Could you do something like (asset (s/valid? xxx)) or (binding [s/*assert-flag-thing* true] (s/assert ::x 1)).
2017:12:06 04:02:10 alexmiller you can use assert* but it’s api is subject to change as part of the implementation - it’s public because it needs to be accessible by the macro when expanded
2017:12:06 19:26:23 hlolli I'm a complete beginner in clojure-spec, my question is, why don't I get spec error here?
2017:12:06 19:27:54 taylor if you’re looking to assert that calls to silly have valid inputs, you can use instrument. Although that’s not going to care about your :ret or :fn function specs
2017:12:06 19:29:08 hlolli I thought that was done with :args within fdef? I was hoping it would tell me that the number 1 is not string
2017:12:06 19:29:27 taylor that’s right, but there’s no :args spec in your example
2017:12:06 19:29:40 taylor and you still have to instrument the function for those assertions to happen
2017:12:06 19:30:18 hlolli ah ok! thanks, I'm reading trough the manual and therefore had this question, I'll continue to read further.
2017:12:06 19:31:48 taylor oh, if you’re only concerned about the return value of your function, instrument isn’t going to assert on that. You can use check for that though.
2017:12:06 19:32:44 hlolli yes that's what I was looking for, but in real world cases I'd see spec being most helpful in the args.
2017:12:06 21:23:11 bbrinck FWIW, Orchestra can be used to check ret/fn specs on instrumentation
2017:12:06 22:18:56 hlolli @U053S2W0V how is it different from cljs.spec.test.alpha/instrument that said, that's exacly the reason why I want to use spec, to get these errors right away.
2017:12:06 22:19:14 taylor instrument only checks the :args
2017:12:06 22:19:46 taylor looks like you can use Orchestra to get the same effect for :ret and :fn specs though
2017:12:06 22:19:50 hlolli ah, nice I see
2017:12:06 22:22:05 hlolli also like their macros, very neat!
2017:12:07 02:56:45 eriktjacobsen Does anyone have any spec->documentation projects? Something that would, for instance, render in a browser a map spec as a map with the values as the underlying spec types, that hides some complexity via being expandable? Thinking something like https://github.com/jebberjeb/specviz but that is more geared towards in-place documentation.
2017:12:07 15:33:57 joost-diepenmaat @hlolli because: you need to use clojure.spec.test.alpha/check a function to test its :ret and :fn specs
2017:12:07 15:34:23 joost-diepenmaat if you switch on instrumentation it will only test the :args spec for afn
2017:12:07 15:35:10 joost-diepenmaat never mind. I just noticed you have a bunch of answers.
2017:12:08 13:44:56 triss Hi all… anyone got a nicer way of doing this?
(defn cat-specs
"Takes a spec and finds the highest level 's/cat' from it and extracts the
specs of the values it matches."
[spec]
(->> (s/form spec)
(tree-seq seq? identity)
(filter seq?)
(filter #(= 'clojure.spec.alpha/cat (first %)))
(first)
(rest)
(partition 2)
(map second)))
2017:12:08 14:20:31 souenzzo
(defn cat-specs
"Takes a spec and finds the highest level 's/cat' from it and extracts the
specs of the values it matches."
[spec]
(->> (s/form spec)
(s/conform (s/cat :op '#{clojure.spec.alpha/cat}
:args (s/* (s/cat :name keyword?
:value any?))))
:args
(map :value)))
2017:12:21 11:30:14 triss Fantastic thankyou! Apologies I just got round to understanding this
2017:12:09 11:40:51 mbjarland I have a question, I’m using spec to validate the format of a “layout string” and in the spec I use conformers to convert from string to seq of chars:
, is conformers the way to go or is there some cleaner way to deal with strings as sequences of chars?
2017:12:09 11:44:03 mbjarland I was hoping to use https://github.com/bhb/expound to make the error messages more readable, but expound does not seem to support conformers
2017:12:09 11:52:26 mbjarland I think I’m landing at using instaparse for parsing strings instead of spec. Seems like coercing proper error messages (including the location in the string etc) from spec would be more work than it’s worth for this problem
2017:12:09 13:10:54 alexmiller While spec can be twisted into parsing strings, it’s not designed for that and it’s going to be way slower than either a proper regex or a parser.
2017:12:09 17:09:59 drewverlee does anyone have an example of how you would conform a string into a data structure, like a hasmap: (conform ::foo "8/blue") => {:age 8 :eye-color "blue"} It seems i would need to specify almost like a regex over the string. This feels doable, but i'm not sure.
2017:12:09 17:24:21 mpenet See s/conformer. You basically pass a fn to it that does the conforming or returns ::s/invalid
2017:12:09 17:24:47 mpenet It can also take another arg to do the inverse (aka unform)
2017:12:09 19:35:39 alexmiller I would say, don’t use spec for this, use a parser like instaparse
2017:12:09 19:36:08 aengelberg :+1:
2017:12:09 19:36:27 alexmiller spec is about describing the structure of Clojure data, not strings
2017:12:09 20:01:39 borkdude I was going to ask here, Instaparse allows to hide output or hide tags to prevent extra nesting. Is such a thing with Spec also possible?
2017:12:09 20:05:05 aengelberg You could maybe use s/and and s/conformer to flatten things as they are being parsed / conformed.
2017:12:09 20:06:47 genec what's the correct set of dependencies for using spec?
2017:12:09 20:10:32 alexmiller Use of generators is optional. If you want that, you’ll also need [org.clojure/tools.check “0.9.0”]
2017:12:09 20:13:12 genec @alexmiller thanks. I'm going to use spec to validate csv files, seems like a good fit
2017:12:09 20:15:34 alexmiller I don’t think that’s a good fit
2017:12:09 20:15:54 alexmiller CSV is text with tricky quoting rules
2017:12:09 20:16:11 alexmiller The right tool is a parser, like instaparse
2017:12:09 20:16:40 alexmiller Spec is for Clojure data
2017:12:09 20:18:01 genec hmm... I'll take a look at instaparse. I have fields that can either be an int or "Missing", which I need to create a log of an errors for the user to correct before importing the file, which is why I thought spec would be good for that
2017:12:09 20:21:08 guy Could you use instaparse to parse the data, then use clojure spec to just check it after?
2017:12:09 20:21:49 genec I suppose I could, I was just going to use one of the csv libs
2017:12:09 20:36:07 borkdude @aengelberg thank you — and thanks for Instaparse, it’s a really amazing tool 🙂
2017:12:09 21:46:02 aengelberg Glad you find it useful!
2017:12:09 22:17:27 borkdude Yes, just used it for the first time to solve a puzzle, inspired by someone who did it in PureScript using parser combinators.
2017:12:10 00:12:10 drewverlee I'm somewhat confused about the exchange here:
https://dev.clojure.org/jira/browse/CLJ-2116
If specs aren't meant for coercion from non clojure data structures to clojure datastructures. Then what is the recommended solution for coercion? Is the message that we shouldn't mix coercion to clojure data with validation of that data? Someone with more experience chim in please, but i feel like those things might overlap a great degree and if you cant compose the tool that is doing one with the other, then it feels like there is going to be a lot of confusing repetition.
2017:12:10 00:39:34 seancorfield @drewverlee The philosophical reason is that if your spec does coercion, no clients of your spec can avoid that coercion, and that can lead to problems.
2017:12:10 00:42:50 seancorfield FWIW, we use spec at work to validate form parameters which all come in as strings, so we have some very limited coercion in a few of our specs. We have ::long and ::double specs that accept both a number and a string that can be parsed to a number, a ::boolean spec that accepts both a Boolean and a string that can be parsed to a Boolean (under a documented set of rules), and a ::date spec that accepts both an instant and a string that can be parsed to an instant (again, under a documented, and limited, set of rules).
2017:12:10 00:43:39 seancorfield That's about as far as I'd go with coercions in specs. Anything beyond that should separate the parsing operation out (and use spec to validate the resulting data structure), IMO.
2017:12:10 00:50:14 seancorfield We also have a REST API where the payloads are parsed (with Cheshire, via wrap-json-params and wrap-json-body) and then validated -- separately -- which makes the specs much more robust and more usable, and makes explain-data much easier to process (for example, using it to turn spec failures into human readable messages). Coercion in a spec tends to make that last part harder (since you have more operations to pick apart).
2017:12:10 00:51:09 seancorfield Coercion in specs also often causes more work in unform and generators since you need to be able to "de-coerce" validated data.
2017:12:10 00:56:19 drewverlee @seancorfield That's reasonable. I suppose it works against what compjuri-api is going for. More to the point, i can't get compojure api to work anyway. Is there another restful api service i should look at? The more hand holding solution the better probably, as my use case is really bare bones.
2017:12:10 00:57:34 seancorfield We've built our REST APIs with just bare bones Ring and Compojure. I haven't looked at compojure-api.
2017:12:10 00:57:34 seancorfield We've built our REST APIs with just bare bones Ring and Compojure. I haven't looked at compojure-api.
2017:12:10 08:08:34 misha @borkdude have a look at s/nonconforming
2017:12:10 09:03:29 ikitommi @seancorfield @drewverlee The CLJ-2116 tries to solve just that, by separating the coercion from specs. Specs to define the form and apply different coercion when needed, e.g. in the borders. int? should be coerced from string->number If provided as a path-parameter, not if provided as JSON body or within your own domain code. Solution to enable this would be simple.
2017:12:10 09:04:59 ikitommi there is also CLJ-2251 on too If that which would make things fast.
2017:12:10 09:06:30 ikitommi @drewverlee If you have trouble with the spec coercion with c-api, happy to help.
2017:12:10 14:23:56 drewverlee Thanks @ikitommi i'll ask in another channel.
2017:12:10 14:37:15 drewverlee @seancorfield i suppose its hard to see why i would use a spec on a string i had already successful parsed in most cases
(parse "drew,2")
=> {:name "a" :siblings 2}
I have already discovered this line meets most of the specification
* has two entries
* second entry is a number
If i want to build a spec about the clojure data to assert it has two entries and the first one is a string and the second is a number, i feel like i'm repeating information. I think the key point here is that, if i build a spec that does coercion on a string it wouldn't verify on the resulting structure. So its not really any more flexible or less work then separating out the coercion function and the spec.
2017:12:10 15:01:14 genec Is there a way to use spec so that it can type hint function parameters during development? Coming from F#, it's something I really miss during development. Or is there a good post about spec demonstrating the Clojure way to keep track of data / types during development?
2017:12:10 15:01:14 genec Is there a way to use spec so that it can type hint function parameters during development? Coming from F#, it's something I really miss during development. Or is there a good post about spec demonstrating the Clojure way to keep track of data / types during development?
2017:12:10 15:06:47 taylor I don’t have a good answer, but I also came to Clojure from F# ❤️ Personally, there were lots of things I missed early on from F# but over time I found myself not missing them so much
2017:12:10 15:08:13 taylor I imagine you’re thinking of something that integrates into the editor? Spec does solve this problem with instrument but it happens as your code is executed, not when you’re typing it in
2017:12:10 19:06:23 genec thanks - yes, I'm missing the type checking from F#. I don't see any reason why there could not be type hints for things that have been spec'd in Clojure. I'll keep looking into this. Out of curiosity, what's your development env? I was using Atom/Proto-Repl but found it a bit flaky so I've switched to Intellij / Cursive and really like it.
2017:12:10 21:40:09 taylor I mostly use IntelliJ w/Cursive
2017:12:10 22:05:54 drewverlee Ok some high level questions:
1. Is there value in exchanging name-spaced data across different systems? I feel like rich might have made a big deal about this and its gotten lost in the noise of me working with spec.
2. What would be the best way to transform a map with one namespace into another one with different namespaces.
e.g
This feels like what conform does, only i want it to not care about the namespace. Also that idea might be imply i'm doing something else wrong, so i'm open to suggestions.
2017:12:10 22:10:07 drewverlee another way to handle my problem would be to unnamespace the map
2017:12:10 22:12:17 drewverlee or i could not apply the namespace in the first place. I feel like these approaches have tradeoffs that probably dont matter in my situation but are interesting to think about.
2017:12:10 22:26:56 drewverlee While i'm thinking about it. Is there a way to combine two specs for a hash-map into one?
2017:12:10 23:04:11 alexmiller s/merge
2017:12:10 23:05:55 drewverlee it would be that easy wouldnt it 😕 thanks @alexmiller
2017:12:11 10:59:30 guy at the bottom of this entity map part
2017:12:11 11:14:53 ikitommi @gfredericks @mpenet bumped into the need of bijections too: In routing, if a path-parameter is declared as keyword? we can easily coerce it from string with string->keyword. But with reverse-routing, we need to turn that keyword param back to string to create a path. I seem to have resolved that earlier by using a json-encoder to write things to string-like, but this can’t be the right way to do this.
2017:12:11 11:18:52 ikitommi I guess a simple (and fast) way would be to introduce a IntoString Protocol. But I guess there can be N different formats to read from and same N formats to write to. Hmm.
2017:12:11 11:24:48 ikitommi Actually, I think I can just introduce a new type-conformer for spec-tools for the opposite direction type->string.
2017:12:11 11:30:09 ikitommi No, using the conform requires the end value to be valid. This would require something like the CLJ-2251 with a “just transform without validating” options. Or something totally different.
2017:12:11 12:21:11 gfredericks so would still be a good use case for a bijections library?
2017:12:12 18:04:58 ikitommi @gfredericks Would be best if spec itself provided the needed hooks to implement the bijections, but not likely to happen. I need something now, so will hack something. If you have good ideas how to do this properly, I’m all ears!
2017:12:11 11:45:16 misha @alexmiller when will clojure.core/long? be released? it is in master, but is not in clojure 1.9.0 as I expected (or am I missing something?). Thanks
https://dev.clojure.org/jira/browse/CLJ-1298
2017:12:11 12:04:49 misha is there out of the box generator for System.currentTimeMillis?
(apart from clojure.test.check.impl/gen-current-time-millis) opieop
2017:12:11 12:11:29 gfredericks @misha how is what you're describing different from generating a long?
2017:12:11 12:11:52 misha far greater than 0? :D
2017:12:11 12:12:12 gfredericks they weren't far greater than 0 in 1970
2017:12:11 12:12:22 misha honestly, I did not think it through yet
2017:12:11 12:12:36 misha pos-int? then?
2017:12:11 12:12:53 gfredericks they were negative in 1969
2017:12:11 12:13:18 gfredericks is there a (not clojure) spec for these numbers? I've never heard of one
2017:12:11 12:13:25 misha oh, okay.
2017:12:11 12:14:17 misha I think the "much greater than zero" part of a spec – is a specialization for my case.
2017:12:11 12:14:41 gfredericks anyhow, with time generators you have three options
A) make a generator that is not focused on your current now (cleanest if you can get away with it)
B) call (time/now) somehow before constructing your generators and pass that in (lots of boilerplate potentially)
C) make a nondeterministic generators (with associated damage to reproducibility and shrinking)
2017:12:11 12:15:09 gfredericks for B) you'd want to log the time you're using, for reproducibility
2017:12:11 12:18:51 misha thank you.
at this point I think I might just use int? with s/with-gen, and supply (System.currentTimeMillis) as a generator, as my only goal (so far) with this particular value - is plausible human-readable exercise data.
2017:12:11 12:19:55 misha (to the extent time-millis could be "human-readable")
2017:12:11 12:20:24 gfredericks converting to/from time-mills would be a cool savant mental trick
2017:12:11 12:20:54 misha doubt I would want to master it opieop
2017:12:11 12:50:14 tjtolton Is there a megathread somewhere about 1.9 discussions? Did spec get pulled from the release as a finished product? Are macros using spec in the stable release? what's going on?
2017:12:11 13:07:26 bronsa spec got pulled out, it's still alpha but no major breaking API changes are expected, currently there's specs for ns, let, fn, defn
2017:12:11 14:15:32 tjtolton Thanks @alexmiller, I'll bet that was a bummer of an internal conversation 😕
2017:12:12 00:43:54 athos As you might already see it on the Clojure ML, I just released Pinpointer, a clojure.spec error reporter similar to Expound, Inspectable, etc https://github.com/athos/Pinpointer
2017:12:12 00:44:14 athos Pinpointer formats (and even colorizes!) spec errors in a human-readable way. The difference between Expound and Pinpointer is very subtle from the outside perspective, but Pinpointer is based on a systematic error analysis rather than heuristic approaches, and this makes it possible to report more complicated errors, such that s/conformer would transform part of the input value.
2017:12:12 00:46:43 athos Have a look at it if you're interested, and any feedback is welcome 😉
2017:12:12 10:25:41 stathissideris @athos I tried pinpointer and it gave up on my complex error, it just used explain as a fallback. Under what circumstances does it do that?
2017:12:12 11:22:29 athos Thank you for giving a try to Pinpointer, @stathissideris! 😆
By default, Pinpointer falls back to s/explain if something bad happens during the error analysis. That mechanism exists to prevent Pinpointer itself from ruining the original spec error info.
2017:12:12 11:23:33 athos If you would like to turn off the fallback behavior, call pinpoint with the option {:fallback-on-error false} like (pinpoint <spec> <input> {:fallback-on-error false})
2017:12:12 11:23:49 stathissideris ok 🙂 in my case I have a pretty loose spec that describes a DSL, so if an expression is wrong, the errors include a lot or different options
2017:12:12 11:24:23 stathissideris because the actual error is one of many things that can go wrong
2017:12:12 11:24:36 stathissideris alright, I’ll try again with this option and see what happens
2017:12:12 11:47:45 stathissideris thanks!
2017:12:12 12:18:17 rickmoynihan can pinpointer/expound/etc… handle exceptions with spec errors in them?
2017:12:12 12:18:35 rickmoynihan or would I need to hook into the REPLs exception printer?
2017:12:12 12:24:16 deg In a re-frame CLJS app, my app-db holds entries created by multiple parts of my system. I want to check total validity at start of each re-frame event. So, I have (s/def ::db (s/merge :ns1/db-keys :ns2/db-keys ,,,)) in a central db.cljs file and I validate it in each event.
Most of these parts are small and can be reasonably each held by a single file holding its specs, subs, and events.
But, this creates a problem. The events need to require the namespace of the merged spec, while that namespace needs the namespaces of each part. What is the cleanest way to tastefully break this namespace circularity?
2017:12:12 12:44:03 rickmoynihan Ok… partly wanting this for exceptions integrant raises from pre-init-spec… but it looks like integrant actually calls s/explain-out to generate the message in the exception so it does indeed work
2017:12:12 12:51:24 athos Yes, s/*explain-out* plugin mechanism should handle exceptions well.
If you rather want something like clojure.repl/pst, pinpointer.core/ppt would be useful.
2017:12:12 12:51:52 rickmoynihan cool
2017:12:12 14:47:53 Andreas Liljeqvist What is the recommended approach for shared specs in several projects? One solution is to make the spec a separate project and then require it.
2017:12:12 15:03:50 taylor I’ve been curious about the same. I guess the options are 1) share a project 2) serialize to some other format which obv. limits what you can do in spec.
2017:12:12 15:09:03 Andreas Liljeqvist Probably going to use a project, on the plus side I get versioned specs
2017:12:12 16:23:14 misha 1) monorepo 2) shared lib 3) send/receive as edn (you’d need to make sure specs are cljs compatible, if sharing with ui)
2017:12:12 19:00:43 jeaye We have a separate repo for specs shared between the client and server (edn). It also contains common functions shared between the two, defs, etc.
2017:12:13 10:27:04 Andreas Liljeqvist Thanks for your input, quite helpfull
2017:12:12 14:48:07 Andreas Liljeqvist But I am not completely happy with that idea
2017:12:12 14:53:32 stathissideris why not?
2017:12:12 15:15:21 Andreas Liljeqvist I suspect it is the best solution, but I am worried about code reloading and such. -- I will just have to try it out
2017:12:12 15:29:01 stathissideris @andreas862 don’t forget about the leiningen checkouts setting (if that’s what you’re using)
2017:12:12 15:29:18 stathissideris it allows you to hack on multiple non-deployed projects in parallel
2017:12:12 15:30:49 Andreas Liljeqvist @stathissideris ah, that is a nice feature!
2017:12:12 21:09:27 bfabry spec is meant for validating ranges of possibilities, a def'd var is "permanent" in clojure land. but yeah putting your right hand side inside a function and then fdefing that would make the most sense to me
2017:12:12 21:09:59 johanatan mm, ok. i'll try that. i think last time i tried it couldn't get it to work with anonymous functions (and wasn't really excited about polluting the global namespace with another defn)
2017:12:12 21:11:06 johanatan do you know if it is possible to fspec an anonymous function?
2017:12:12 21:11:23 bfabry I mean seeing as this is all only happening once you could just do (def foo (s/assert ::spec (your code here)))
2017:12:12 21:11:51 johanatan hmm, except that it would probably need a doto so that the result is returned?
2017:12:12 21:12:45 bfabry assert returns the result if the result was valid
2017:12:12 21:12:53 johanatan oh, ok
2017:12:12 21:12:59 johanatan yea, that works. thx!
2017:12:12 21:13:02 bfabry no worries
2017:12:12 21:18:40 hlolli How does one do or in spec, I've tried googling
I also don't need these keys, but this function is probably wrong. Im makeing a transpiler to a language that has it's own dispatch functions, and since I'm automatically generating the metadata, I end up with different possibilites. Essentally, I'd want to be able to say; this vector can be a sequence of specs ::x ::y or sequence of specs ::foo ::bar.
2017:12:12 21:19:24 taylor s/tuple?
2017:12:12 21:20:19 hlolli troughout the ocean of functions I want to spec, they can be tuple/octet or whatever number of possibilites...
2017:12:12 21:20:59 alexmiller probably want to look at the regex specs (cat * + ? alt)
2017:12:12 21:22:05 hlolli ok, will look there. Lot to take in when first stepping into spec 🙂
2017:12:12 21:24:20 alexmiller if I read what you want to do there for example you could say (s/cat :x ::ar :y (s/alt :a ::ar :k ::kr))
2017:12:12 21:25:14 hlolli cat defines a sequence?
2017:12:12 21:25:21 alexmiller to match sequential collections that start with ar and then have ar or kr. x, y, a, and k are all made up keys that affect the conformed result
2017:12:12 21:27:03 hlolli ok nice, this is actually for the input parameters so it's importent, I'm tempted to do this check at the very beginning of my functions.
2017:12:12 21:27:25 alexmiller they are most commonly used in s/fdef to spec args to functions
2017:12:12 21:27:36 alexmiller so that is a good match
2017:12:12 21:29:08 hlolli yes I may end up doing that, it's just this instrument function that I'm skeptical of, but I should not be. I want immediate error in runtime on wrong spec, or no matching patterns of a provided arguments, it's kinda pattern matching, therefore looking at or
2017:12:12 21:29:54 alexmiller s/assert might be useful too
2017:12:12 21:30:30 hlolli On extra question since I'm rambling, is there a way to have any effect on the error message
In: [1] val: #object[csound.core.ControlSignal] fails spec: :csound.core/ar at: [:k1 :0] predicate: audio-signal?
I'm in clojurescript and I know that the object has keys which could help the user locate his problem, as this could be within a nested ast.
2017:12:12 21:32:33 taylor s/explain-data gives you a structure you can get useful stuff out of, and there are some libs for transforming that into something that could be useful for you
2017:12:12 21:33:33 hlolli ah yes! in fact the object itself in question.
2017:12:12 21:35:02 hlolli Thanks @U3DAE8HMG
2017:12:12 21:35:57 hlolli Or explain has it too, just didn't consider takeing advantage of that and throw my own error message, suited to help the user find the spec mismatch.
2017:12:14 20:12:49 hlolli Who's asking @U0MF00YRX? I'm working on some interoperability between clojurescript and csound, mainly transpiler, that would be a good start, already been live-coding with this combination for few years
2017:12:14 20:16:19 bfay Oh awesome @U0CAUAKCG! I'm trying to do some similar stuff with clojurescript/supercollider. I tried using Overtone for a bit, but found that the startup time is too painful, especially when I tried running on a Raspberry Pi
2017:12:14 20:24:15 hlolli Yes, Overtone is bit over engineerd for Raspberry Pi. But it's way better software than Sonic Pi in my opinion.
2017:12:14 20:34:17 bfay I think Sam Aaron had a different vision for both. The vision for Sonic Pi is about live-coding and education, it's a fun way to teach newcomers how to code. But yeah possibilities are pretty limited compared to Overtone and plain SuperCollider
2017:12:13 04:37:23 johanatan hi, does anyone know how to diagnose the following error?
1. Unhandled java.lang.IllegalArgumentException
No implementation of method: :specize* of protocol:
#'clojure.spec.alpha/Specize found for class: nil
2017:12:13 13:20:46 guy i think s/or takes keyword, predicate pairs
2017:12:13 13:20:53 lopalghost yes, this was in response to @johanatan
2017:12:13 13:21:19 guy ah sorry
2017:12:13 13:21:26 guy 👍
2017:12:13 13:21:40 guy Good spot then!
2017:12:13 17:25:42 johanatan @lopalghost good catch! thx!
2017:12:13 17:43:20 johanatan does anyone know a way for an fspec to consider relations between its target's inputs? like if one of the parameters is functionally constrained by the value of the other one?
2017:12:13 17:44:16 johanatan i could do it by stuffing the args into a nested vector:
(defn f [[a1 a2]] ...)
and then writing a spec/generator for that vector which combines two other [independent] spec/generators but i was kind of hoping that there is a better story for this...
2017:12:13 17:52:51 lopalghost the :args argument already treats the args as a sequence, so you can use (s/and ...) to spec both individual inputs and relations between them
2017:12:13 19:57:23 lopalghost or otherwise, pad the vector with a nil before conforming, if it only has one element
2017:12:13 19:57:37 lopalghost I think cat would be most idiomatic though
2017:12:13 20:19:49 johanatan @lopalghost actually s/and for that purpose will result in a such-that and you can be at risk of a couldn't satisfy such-that predicate after 100 tries error
2017:12:13 20:20:08 johanatan there needs to be a way to generate the two values together so that we know the two will conform with one another
2017:12:13 20:20:14 johanatan hence my suggestion to use a nested vector of args
2017:12:13 20:21:30 johanatan do you see any other way?
2017:12:13 20:38:03 tristefigure Hi. This is my first time using spec, and taking inspiration from clojure.core.specs.alpha, I decided to write my own version to parse/produce macro definitions. I wanted it to return something less detailed in a flatter hash and I wasn't happy with the fact results from conform mirror the spec' structure and with other petty details, so this is what I came up with:
2017:12:13 20:38:46 tristefigure To which extent is this a misuse of this library ?
2017:12:13 21:35:55 lopalghost @johanatan not sure how a nested vector would solve the problem, as any spec you apply to the nested vector could also be applied to :args
2017:12:13 21:36:23 johanatan @lopalghost it solves it by letting me introduce a generator specifically for that tuple
2017:12:13 21:38:13 lopalghost You can't do that with the :arts spec? Sorry if I'm misunderstanding you
2017:12:13 21:38:28 lopalghost :args*
2017:12:13 21:38:51 johanatan e.g.,
(s/def ::args-tup
(s/with-gen
(s/tuple ::arg1 ::arg2)
#( ... enforce relations here )))
(s/fdef -refinement-kwd->validator
:args (s/cat :tup ::args-tup)
:ret ::ret-val)
(defn- -helper [[arg1 arg2]] ; tuple allows both inputs to be generated simult.
;; omitted
)
(defn- original [arg1 arg2]
(-helper [arg1 arg2]))
2017:12:13 21:39:19 johanatan you can do it in :args but you are liable to hit the cannot satisfy such-that problem
2017:12:13 21:47:59 lopalghost I mean why not just use ::args-tup as the :args spec? Does that cause a problem? I'd try it out myself but I don't have a repl at the moment
2017:12:13 21:50:13 johanatan s/cat is required to name the arg
2017:12:13 21:50:26 johanatan yea, without s/cat, stest/check tries to send in the wrong number of arguments
2017:12:13 21:50:41 johanatan [i.e., sends in 2 where 1 is expected]
2017:12:13 22:09:11 lopalghost ok I cobbled together a stupid example, let me know if this is relevant:
2017:12:13 22:10:14 lopalghost didn't supply a generator of course, but stest/check works with no problems here
2017:12:13 22:32:59 hlolli Would it be wise to spec no argument, if just for the metadata, if so, how would someone spec :args for no args?
2017:12:14 06:59:26 curlyfry Just (s/cat)!
2017:12:14 10:08:34 hlolli Thusund takk!
2017:12:13 22:35:11 johanatan @lopalghost that use of s/and has an implied underlying gen/such-that which depending on circumstances may be impossible or too difficult to satisfy. thus it is always better to generate exactly the right structure you want from scratch rather than relying on such-that.
2017:12:13 22:49:19 johanatan @lopalghost also, your fdef won't work: :args is an alternating sequence of name and type/spec. thus you need s/cat
2017:12:14 15:03:26 lopalghost @johanatan seems to work fine
2017:12:14 15:04:02 lopalghost is there a reason you need the args vector to conform to a map?
2017:12:14 15:05:12 lopalghost if you supply a generator for ::args-tup, you shouldn't have to worry about failure during generative testing
2017:12:14 15:13:47 gfredericks easy generator for a strictly descending pair of numbers: (gen/let [[a b] (gen/tuple gen/large-integer gen/large-integer)] (let [[a b] (sort [a b])] [(inc a) b]))
2017:12:14 15:47:05 andy.fingerhut I was looking at updating the Clojure cheat sheet for v1.9, now that it is out. I was wondering -- is there a list somewhere of all Clojure predicates for which test.check has generators included, vs. those it does not have generators for?
2017:12:14 15:48:06 gfredericks I expect there's a data structure in the spec code mapping specs to generators
2017:12:14 15:49:06 andy.fingerhut If someone beats me to finding a link to that, let me know. I may want to link to it.
2017:12:14 15:50:46 gfredericks if you have a list of predicates, then programmatically figuring out which have generators should be easy
2017:12:14 15:52:08 andy.fingerhut Is there some small expression you can evaluate in a REPL to determine if a predicate has a generator already?
2017:12:14 15:53:11 gfredericks I thought there was a function in the spec API that takes a spec and returns a generator
2017:12:14 17:03:03 alexmiller s/gen
2017:12:14 17:03:40 alexmiller There are some things in gen namespace too
2017:12:14 17:04:11 alexmiller And that’s where the built in mapping is set
2017:12:14 17:05:00 lopalghost gen/gen-for-pred
2017:12:14 17:47:04 andy.fingerhut Thanks, s/gen is very helpful. And dang, I could have really added a lot of value by helping with the generator for zero? 🙂
2017:12:14 17:50:35 andy.fingerhut I couldn't have been very helpful, of course. Simply my way of expressing surprise that there was such a generator.
2017:12:14 20:56:33 andy.fingerhut Yes, the most complex part about it is if you want to generate every value that can satisfy zero?
2017:12:14 20:57:10 andy.fingerhut I probably should have made my snide remark about the predicate nil? instead, since it would be more obvious.
2017:12:14 20:57:51 andy.fingerhut I am probably having a senior moment here on my memory -- is there any reasonably common term for any type that isn't a collection?
2017:12:14 20:58:34 taylor it’s not scalar is it
2017:12:14 20:58:58 andy.fingerhut It is, and I am understanding more every day what memory lapses feel like 🙂
2017:12:14 20:59:46 taylor I increasingly find myself forgetting what I’m about to do as I walk across my tiny apartment to do it
2017:12:14 21:44:07 gfredericks tattoos are the answer
put "things that aren't collections are scalars" on your left arm
2017:12:14 21:47:46 cgore And “there are no scalars” on your right arm 😄
2017:12:14 22:00:09 andy.fingerhut On a related note, I watched the movie "Memento" for the first time recently. Sound advice.
2017:12:14 22:00:23 taylor are you sure it was the first time??
2017:12:14 22:00:48 gfredericks more likely he will soon watch it for the last time
2017:12:14 22:02:15 andy.fingerhut It felt like the first time, anyway
2017:12:14 22:02:59 andy.fingerhut If only I can keep forgetting my favorite movies and books, I can get full enjoyment from them multiple times -- if I can remember which ones they are.
2017:12:14 22:04:41 taylor if ignorance is bliss then forgetfulness is… personal growth?
2017:12:14 22:04:47 andy.fingerhut (Laugh Out Loud And Had To Explain My Laughter To My Wife)
2017:12:15 07:30:34 andy.fingerhut Are there any handy examples someone can link to for writing a spec for a function, where one of its arguments is a predicate function, e.g. a spec for clojure.core/filter ?
2017:12:15 07:59:13 andy.fingerhut Thanks. Nicely written examples there.
2017:12:15 07:56:07 andy.fingerhut I've got the doc strings for seq? and seqable?, but I can't for the life of me think what the difference is between them, i.e. why would you pick one over the other in a spec?
2017:12:15 08:05:38 curlyfry @andy.fingerhut seqable? is for things that can be made into a seq (but aren't necessarily seqs themselves). For example, (seq? []) returns false, but (seqable? []) returns true.
2017:12:15 08:08:46 andy.fingerhut Got it. Thanks.
2017:12:15 08:09:13 andy.fingerhut seqable? seems far more often useful in :args specs, then.
2017:12:15 08:14:18 andy.fingerhut Looks like a spec like this probably needs a custom generator, if I see an error message like "Couldn't satisfy such-that predicate after 100 tries."? (s/def ::set/relation (s/and set? #(every? map? %)))
2017:12:15 08:19:36 taylor yeah with s/and I believe the generator is based off just the first spec
2017:12:15 08:20:08 taylor so, very unlikely it will generate sets with all maps
2017:12:15 08:26:41 andy.fingerhut When I try to def sorted-vec in my REPL I get an assertion error "Arg to vector must be a generator"
2017:12:15 08:28:37 andy.fingerhut Sorry, pilot error. I had my gen alias'ed to clojure.spec.gen.alpha, not clojure.test.check.generators as stated at the top of that examples page
2017:12:15 08:44:47 andy.fingerhut Does a generator for predicate any? ever produce values that aren't equal to themselves in Clojure, e.g. Double/NaN ?
2017:12:15 12:10:29 gfredericks test.check's gen/any sure can
not sure what any? got wired up to
2017:12:15 12:10:54 gfredericks I feel like in both cases we need a secondary concept, like gen/any-sane
2017:12:15 12:11:25 gfredericks just making NaN opt-in all the time seems like encouraging bad testing
2017:12:15 12:30:34 lopalghost @andy.fingerhut have you tried using coll-of to spec a set of maps?
2017:12:15 13:03:53 andy.fingerhut because, you know, it is not, not a number
2017:12:15 13:09:40 gfredericks yeah but I'm talking about for something like gen/any where you are potentially generating large data structures and if there are any NaNs hiding anywhere in that giant tree then the whole thing becomes sometimes equal to itself but sometimes not
2017:12:15 13:11:49 andy.fingerhut right. gen/any-anumber was a bad joke 🙂
2017:12:15 13:12:33 lopalghost reminds me of one of my favorite clojure expressions:
> (number? ##NaN)
> true
2017:12:15 13:13:54 andy.fingerhut I feel a JIRA coming on ...
2017:12:15 13:14:33 andy.fingerhut @lopalghost I have not tried coll-of, but thanks for the suggestion. Will do.
2017:12:15 13:16:40 andy.fingerhut I guess the generator for any? also sticks with things considered "values" in Clojure, i.e. it doesn't generate instances of java.util.Set ?
2017:12:15 13:17:22 andy.fingerhut those also play havoc with one's usual ideas for clojure.core/= if they are inside of other sets, or keys of maps.
2017:12:15 13:59:55 gfredericks yeah, definitely
2017:12:15 14:00:09 gfredericks generating literally anything would be quite a tall order
2017:12:15 21:28:05 didibus Is there a way to override the generator of the args spec of an fspec? Using instrument and :gen I thought would be able to do it, but I can't figure how.
2017:12:15 21:33:26 alexmiller I don't think so, or at least it's not easy (and I think there is a ticket about this)
2017:12:15 21:35:21 didibus Ok, thanks
2017:12:16 09:40:56 andy.fingerhut Are there any common practices on how detailed people like to make their specs, in terms of trying to specify precisely which :args should be allowed, vs. which should not? In particular, I've been looking at clojure.set/rename-keys and how some values of the second argument lead to hard-to-predict return values.
2017:12:16 09:42:02 andy.fingerhut If you see happy faces, those should be keywords with "b"
2017:12:16 09:42:44 andy.fingerhut I could write a spec simply saying that the second parameter is a map?, but that would allow maps like the above to be included, which have unpredictable behavior.
2017:12:16 09:43:22 andy.fingerhut I could also write a more restrictive spec for the 2nd arg that prevents any 2 values in the 2nd argument map from being equal to each other, which would avoid the unpredictable behavior.
2017:12:16 09:45:13 andy.fingerhut For that same function, it is also clear that for generating random inputs that are "interesting", i.e. that don't simply always return their first argument with no changes, the keys in the first and second maps should often contain common values. So a useful generator that tries to exercise bugs in the implementation needs to steer inputs in that direction with high probability.
2017:12:16 09:46:11 andy.fingerhut Anyway, looking for thoughts and recommendations there.
2017:12:16 10:13:29 gklijs @andy.fingerhut I think a great deal comes down to how much effort you want to put into it. As sone as you start using very specific specs, you will also need to supply generators for those. While if you just want to have some edge cases tested, it would probably be less work to write specific tests for those. Depending on how you use specs, making them complex might also slow things down.
2017:12:16 11:51:16 guy you can use ` to make it ignore emoji’s @andy.fingerhut
2017:12:16 11:51:28 guy do one ` then another after you have finished
2017:12:16 11:51:40 guy it should look like {:a 2 :b 3}
2017:12:18 20:12:31 donaldball I’m struggling to figure out a good way to express a spec category that has come up before, but I can’t recall what the best recommendations are any longer. The value is a map with namespace keyword keys, two of which each require values in their own sets, but further the pair of values must occur its own set. For example, :pet/type could be #{:cat :dog :snek} and :pet/fur? could be boolean?, but {:pet/type :snek :pet/fur? true} is not allowed.
This is trivial with a predicate spec but then I lose any hope that s/explain can identify the paths to the keys contributing to the violation.
2017:12:18 21:27:57 alexmiller multispec?
2017:12:18 21:35:56 misha @donaldball yeah, doable with multispec, but verbose. Especially if you'd overload second value based on type of the first (different value sets for :pet/fur? depending on :pet/type
2017:12:18 21:38:52 misha plus, you'd need to supply custom generators to avoid "could not satisfy predicate after # tries", when amount of valid value invariants is noticeably smaller than count of combinations in cartesian product of both value sets.
2017:12:18 21:44:18 donaldball I thought about multispec but I wasn’t even sure how I could tell spec “hey use a smaller spec for this namespaced key in s/keys this time” and couldn’t figure out a good way to express it using s/map-of🙂
2017:12:18 21:44:49 misha speaking of "verbose", how you, ladies and gentlemen, organize your specs, when there is fair amount of "intermediate" specs, which are not exactly useful in and of themselves, but are used to build larger composite specs. E.g. when "leaf" specs are "useful", and "root" spec is "useful" (for eyeballing), but "branch" specs in between are sort of just glue and are noisy.
"Just suck it up" – is an acceptable answer, I guess opieop
2017:12:18 21:45:27 misha @donaldball you can "alias" specs
2017:12:18 21:51:24 misha the thing is, when you want to overload a :pet/foo key value based on :pet/type value, you can get away with this in spec by using unqualified keywords in your entity maps {:foo ..., :type}.
but if you want to overload value of a namespaced key – it will be a mess to spec it, if possible at all
2017:12:18 21:57:46 misha @donaldball you still have an option to return these from multispec
(s/keys :req [:pet/type :cat/fur?]) ;; for type=cat
(s/keys :req [:pet/type :snake/fur?]) ;; for type=snake
2017:12:18 22:34:47 donaldball Yeah, but all these options seem to require adjusting the form of my inputs. While the input form is not necessarily desirable, I’d like to try to spec is it is, not as I have transformed it to be.
2017:12:19 19:38:20 falak I am trying to write specs for a function whose argument can be nil or a collection. However, I am unable to combine those two using s/or. This is what I tried -
This definitely won't work but it'd be great to have the option -
..... (s/or nil? list?)
2017:12:19 19:42:55 guy can’t you do
(s/def ::arg1 (s/nilable list?)) then just use that with s/cat :arg1 ::arg1 ?
2017:12:19 19:50:58 falak Ah! I wasn't aware of s/nilable. Thanks!
However, I am not sure why but (s/nilable coll?) works and s/nilable list? doesn't when the data structure is actually a list of maps (or nil).
2017:12:19 19:52:11 falak When I run type on the args, it says clojure.lang.LazySeq
2017:12:19 20:10:21 seancorfield @fravani because list? is only true for things that implement IPersistentList -- which a lazy sequence does not.
2017:12:19 20:11:35 seancorfield But coll? is true for IPersistentCollection and LazySeq does implement that.
2017:12:19 20:18:27 falak That makes sense. No wonder it worked when I converted the lazy seq to non-lazy using into () ... Thank you so much for the help.
2017:12:20 21:10:18 sparkofreason If I want to have a macro that will be able to process spec forms defined in ClojureScript, how do I get the forms in the macro definition? s/form will only get the form at run time, and I need it at compile time.
2017:12:20 21:30:44 misha will you use it instead of s/def?
2017:12:20 23:15:42 sparkofreason No, after s/def. Basically trying to auto-generate destructuring forms based on the spec.
2017:12:20 22:02:20 ag Can someone give me an insight to the issue I'm having with our project please.
Because of the way how clojure.spec.alpha interacts with clojure.test.check, specifically I'm guessing this part https://github.com/clojure/spec.alpha/blob/master/src/main/clojure/clojure/spec/gen/alpha.clj#L38
it breaks on attempt to use clojure.tools.namespace.repl/refresh, in a project where clojure.test.check.generators being used in .cljc file(s).
And that's because Generator record gets redefined, resulting in two different classes.
Here's a sample repo that repros that bug (created by my colleague) https://github.com/xiongtx/reload-error-boot.
I thought the easy way of fixing it is to stop using clojure.test.check and just use clojure.spec.gen.alpha everywhere. But that doesn't seem to include bunch of things: macros like gen/let and gen/bind, gen/return, gen/pos-int, etc. Basically not using it at all - seems not possible (?)
Because of that, CIDER's refresh and clojure.tools.namespace.repl/refrsh are broken, that makes it sort of painful to work with. Can someone suggest a workaround?
2017:12:20 22:03:09 gfredericks can you use refresh-all instead?
2017:12:20 22:03:25 gfredericks wait why does Generator get redefined at all?
2017:12:20 22:08:32 gfredericks refresh shouldn't be reloading library code
2017:12:21 01:00:18 ag can you please take a look at https://github.com/xiongtx/reload-error-boot
there's something strange going on, I think this deserves to be a JIRA ticket (unless someone already filed one)
I can't formulate though exactly what's happening
2017:12:21 01:12:08 gfredericks at a glance, my guess is that cider is more enthusiastic about reloading things than tools.namespace is?
certainly seems like bad behavior. I don't think tools should reload libraries under normal circumstances. I've never seen tools.namespace do that.
2017:12:21 01:19:40 ag it happens also with tools.namespace
2017:12:21 12:06:32 gfredericks you're saying the lack of a standard require makes tools.namespace act as if t.c.generators was a project namespace?
2017:12:21 16:07:23 ag ¯\(ツ)/¯
2017:12:21 16:07:49 ag it seems failing only for .cljc files
2017:12:21 16:08:29 ag with shared generators
2017:12:21 17:23:47 gfredericks does tools.namespace explicitly announce that it's reloading the generators namespace?
2017:12:21 17:23:59 gfredericks (it usually prints which things it's reloading)
2017:12:21 18:02:35 ag this is in our project, when you do
(require '[clojure.tools.namespace.repl :refer [refresh]])
(refresh)
:reloading ... then list of namespaces
:error-while-loading finops-admin.specs.shared
#error {
:cause "Assert failed: Second arg to such-that must be a generator\n(generator? gen)"
:via
[{:type clojure.lang.Compiler$CompilerException
:message "java.lang.AssertionError: Assert failed: Second arg to such-that must be a generator\n(generator? gen), compiling:(shared.cljc:72:34)"
:at [clojure.lang.Compiler$InvokeExpr eval "Compiler.java" 3700]}
{:type java.lang.AssertionError
:message "Assert failed: Second arg to such-that must be a generator\n(generator? gen)"
:at [clojure.test.check.generators$such_that invokeStatic "generators.cljc" 346]}]
:trace
and long stacktrace
2017:12:21 18:02:58 ag note, there's nothing wrong with that generator in finops-admin.specs.shared
2017:12:21 18:17:02 ag this is a bug that appears in the .cljc code that's trying to use clojure.test.check.generators
2017:12:21 00:15:15 sparkofreason Found an answer, albeit hacky-feeling, to getting compile-time spec forms for CLJS. At compile time, the CLJS form of the spec is stored in cljs.spec.alpha/registry-ref, an atom containing a map keyed by spec-name. This works, but feels like I'm coupling to internal implementation details, so if there's a better way, I'd be thankful.
2017:12:21 00:15:16 johanatan @lopalghost ah, yea, sorry re: the s/cat: it apparently is redundant/ unnecessary when the spec itself is already a tuple
2017:12:21 00:15:25 johanatan and yea, generating the entire tuple at once is what i'm doing in fact
2017:12:21 00:15:46 johanatan was just hoping there'd be a better way because that implies that i need wrapper functions that take the untupled inputs and call the tupled ones
2017:12:21 00:16:15 johanatan i'm hitting a weird error where spec itself is trying to construct an ExceptionInfo in a haram way
2017:12:21 00:20:55 johanatan if when-not returns nil, ExceptionInfo doesn't like that
2017:12:21 00:26:50 johanatan (apply ex-info (remove nil? ["Specification-based check failed" (when-not ...)])) would seem to be a fix
2017:12:21 00:27:25 johanatan although that's going to seriously limit the person on the receiving end's ability to debug 🙂
2017:12:21 00:27:36 johanatan so there is likely a higher-level explanation for why we got here
2017:12:21 00:27:54 johanatan which might be good to pass on to the user
2017:12:21 07:24:54 misha @dave.dixon you can call any defined function inside macro as part of the data transformation, can’t you? Which happens at compile time. It is the form macro returns who is evaluated at run time
2017:12:21 07:25:59 misha Did you try to use s/form inside macro, but outside of quoted return value?
2017:12:21 11:29:22 triss hey all - how do I turn something like (clojure.spec.alpha/coll-of clojure.core/number?) back in to spec I can use in s/valid?. I’m deconstructing my specs and using bits of them elsewhere.
2017:12:21 11:42:10 guy Could you just do (s/def ::your-spec (s/coll-of number?)) Then you can do (s/valid? ::your-spec ..)
2017:12:21 11:42:41 guy Or can’t you just use it in s/valid?
2017:12:21 11:42:57 guy (s/valid? (clojure.spec.alpha/coll-of clojure.core/number?) …)
2017:12:21 11:48:13 triss thanks guy… I’ll check that in a moment…. I just realised a quick call to eval does the job for me.
2017:12:21 11:49:35 guy kk!
2017:12:21 14:18:52 sparkofreason @misha Yes. That doesn't find the spec. Looking at the CLojureScript spec code, the form function is only defined at run-time, and basically just wraps the specize* method on the Specize protocol. The reified instance of Specize wouldn't be available at compile-time anyway. The registry-ref atom defined in cljs/spec/alpha.cljc appears to be the mechanism by which the s/def macro communicates between compile and run-time. I don't see any function which would abstract access to registry-ref for use from macros.
2017:12:21 21:14:35 ag possible to have a "parametrized spec", where you can specify generator parameters? e.g.:
making the following more flexible, by letting it generate dates in specified range (instead of hardcoded values):
2017:12:21 21:15:28 alexmiller no, other than via a macro wrapping this
2017:12:21 21:16:09 alexmiller there is s/inst-in which does ranges?
2017:12:21 21:16:28 alexmiller not sure if that would serve your needs
2017:12:21 21:17:06 ag alright... thanks @alexmiller
2017:12:21 21:56:24 seancorfield @ag Yeah, that's been a bugbear for us at World Singles since we have to track a constantly moving time window for certain valid time ranges.
2017:12:21 22:11:54 ag so this (gen/sample (s/gen (s/inst-in #inst "2017-10-01" #inst "2018-12-31"))) returns bunch of dates, but they are mostly are in 2017-10, what gives?
2017:12:21 22:12:40 shaun-mahood @ag: There was a great talk by @gfredericks at this years conj about generators that had a good section on better generation of datetimes (the most relevant part starts at about 30 mins in if you want to skip right to it) - https://www.youtube.com/watch?v=F4VZPxLZUdA
2017:12:21 22:12:56 seancorfield @ag generators for ranges tend to start at the beginning and use small increments at first. Yup, what @shaun-mahood said!
2017:12:21 22:13:29 seancorfield That talk was super helpful for us at World Singles!
2017:12:21 22:13:48 ag we've been also using test.chuck I think it's @gfredericks project, right?
2017:12:21 22:19:55 shaun-mahood Yep - I haven't tried it yet but more than once someone has pointed me to it to answer a question about doing something more complicated. I guess I should try it 🙂
2017:12:21 22:21:08 seancorfield We use it for regex generators. Wonderful!
2017:12:21 23:08:09 misha @ag you can provide map of override generators on spec "call" site
...Optionally an overrides map can be provided which
should map spec names or paths (vectors of keywords) to no-arg
generator-creating fns. These will be used instead of the generators at those
names/paths. Note that parent generator (in the spec or overrides
map) will supersede those of any subtrees...
2017:12:21 23:14:24 misha this way you can sort of dynamically inject generators built specifically for the current circumstances. But I yet to see (or spend time building myself) complex enough dependent generator "trees", so can't point to any caveats or best practices yet
2017:12:21 23:15:38 misha (e.g. imagine ETL pipeline: you generate random input file, generate ETL config based on generated input file, and then property-test extracted result against input)
2017:12:21 23:18:41 misha more detailed your specs become – more branchy and fragile (I guess?) your generators become. And managing that is noticeable overhead in and of itself. Throw in proper shrinking capabilities support, and random seed honoring, and it is suddenly a full time job opieop
2017:12:21 23:59:11 cfleming Just to double check my understanding here - if I’m wanting to use spec for macro grammars in a library which also needs to support clients on older versions of Clojure, I can just have the specs and the fdefs in a separate namespace which is loaded dynamically somehow if Clojure 1.9+ is detected, is that right?
2017:12:21 23:59:43 cfleming Or perhaps users of the library can just require that namespace if they’re using Clojure 1.9.
2017:12:22 00:03:11 bfabry seems to be exactly what clojure.java.jdbc does
2017:12:22 00:03:26 cfleming Thanks, I’ll check it out.
2017:12:22 00:03:52 cfleming in fact, the specs could even be in their own library I imagine.
2017:12:22 00:04:11 bfabry well if it works for core 🙂
2017:12:22 00:04:29 cfleming Yeah, but core is special 🙂
2017:12:22 00:06:58 cfleming Ok, great - looks like that will work fine - thanks!
2017:12:22 00:54:47 bbloom what’s the idiom for negating a spec? ie if i want to greedly parse excluding something
2017:12:22 00:55:48 bbloom just use #(not …) ?
2017:12:22 00:58:34 gfredericks complement?
2017:12:22 00:58:46 bbloom yeah or that
2017:12:22 00:59:02 bbloom just wondering if there was any negative match regexes or anything like that
2017:12:22 01:05:33 bbloom i guess i’m struggling with a non-greedy (reluctant?) s/* pattern
2017:12:22 01:08:44 andy.fingerhut Has anyone attempted to write code that determines the 'difference' between two specs yet? That this is possible to do for "vanilla" regular expressions was, I recall, mentioned as a motivating factor for spec to be include regex matching. (I realize, of course, that spec generalizes vanilla regex's in a way, with extra side conditions via and/or, such that it might not always be possible to apply those regex diff algorithms in those cases).
2017:12:22 01:08:59 andy.fingerhut s/be include/be based on/
2017:12:22 01:11:29 alexmiller I don’t think it’s going to be efficient to do so without leveraging the guts of the regex spec impl
2017:12:22 02:48:35 cfleming Is it considered idiomatic to name things that I don’t care about capturing :_?
2017:12:22 09:46:06 misha @cfleming I've seen it used. I prefer :_actual-name-of-thing-you-ignore both for keywords and symbols (documentations is hard to come by these days)
2017:12:22 09:51:06 cfleming @misha Thanks. This is for constant symbols and keywords in macro grammars, I had things like (s/cat <whatever> :from #{:from} <whatever>) which seems a bit redundant.
2017:12:22 09:51:21 cfleming (i.e. when capturing a constant :from)
2017:12:22 10:03:54 misha of course there are things which are impractical to document (in any way)
2017:12:22 16:00:52 triss is it possible to look up the spec for a function?
2017:12:22 16:22:46 athos You probably mean s/get-spec rather than s/spec ?
2017:12:22 21:15:37 curlyfry @triss clojure.repl/doc will show you any specs for a function
2017:12:22 22:36:39 pablore hello, is there any way to define a spec for a sorted collection?
2017:12:23 11:34:53 stathissideris I gave a talk on Spec and what you can do with it for the Athens Clojure Meetup, it’s in Greek with English captions: https://www.youtube.com/watch?v=T1qpIaB6_vM
2017:12:23 20:02:36 bbrinck @stathissideris your talk was excellent. I particularly liked the descriptions and diagrams of more advanced generative-testing techniques. Good stuff.
2017:12:23 20:06:27 stathissideris @bbrinck thanks a lot, I’m glad you enjoyed it! :)
2017:12:24 08:08:19 misha @pablore yes, expensive one would be something like
2017:12:24 08:11:26 misha basically, write a predicate function, which will answer your "is it sorted?" question with true or false.
2017:12:24 08:11:46 rauh #(apply <= %) is probably faster.
2017:12:24 08:11:52 misha (the one above changes collection type, and will not work most of the time)
2017:12:24 08:17:48 misha yeah, ~400times faster opieop
the point is – basic spec is just a predicate function
2017:12:24 14:27:14 alexmiller just use sorted?
2017:12:24 14:27:55 alexmiller if you are checking whether it’s a sorted collection or not
2017:12:24 16:28:24 taylor is there a predicate for checking if something is a lazy seq
2017:12:24 16:37:32 taylor disregard, I really just want to convert all sequences to vectors
2017:12:25 23:53:22 flyboarder Hey guys, trying to figure out how I can serialize a clojure spec exception. java.io.NotSerializableException: clojure.spec.alpha$regex_spec_impl$reify__2436
2017:12:26 07:45:27 gklijs @flyboarder The Class java.lang.Exception extends Throwable implements Serializable, to get it to work you probably need to create your own exception which either extend from the java exception, or implement Serializable, that’s how far I can help you with my java background, never did anything yet in clojure for exception handling, so I don’t know how it’s different from java.
2017:12:26 09:11:04 stathissideris @flyboarder looks like the serialization breaks for the spec itself which I believe is included in the ex-data of the ex-info, under the key :spec
2017:12:26 09:11:23 stathissideris could you dissoc it as a quick test and try serializing again?
2017:12:26 19:58:35 flyboarder @stathissideris that seems to work, I need to remove the entire namespaced key :clojure.spec.alpha/spec is there a way to future proof the removal for when the namespace changes?
2017:12:27 13:23:32 alexmiller Alias clojure.spec.alpha to s in your namespace definition, then use ::s/spec. That way, you will just need to change the namespace in one spot.
2017:12:27 18:54:46 flyboarder @U064X3EF3 thanks!
2017:12:26 19:59:46 flyboarder @gklijs I needed to wrap the exception in my own ex-info while removing the non-serializable parts, thanks!
2017:12:27 04:40:12 shdzzl I'm trying to work out what is happening with the following snippet, I think I must be misunderstanding how s/cat works:
2017:12:27 04:43:19 shdzzl It makes no sense to me that it would be matching the first vector element on ::bound, so I'm not sure how that's the spec it's failing.
2017:12:27 04:43:59 flyboarder @shdzzl I had issues wrapping my head around s/cat also, it kinda explodes the argument, is the best way I can describe it
2017:12:27 04:47:13 shdzzl I'm not sure I follow, it explodes which argument? I visualize it as kind of zipping predicates to a sequence, so my s/cat example should match any seq of length three and check the first element against ::needs, the second against ::bound and the third against ::body. What am I missing?
2017:12:27 05:15:51 shdzzl Uhuh, that does fix it. ::needs wasn't failing when tested on it's own. Is there some flattening or something going on? I'm going to have read over s/* again. Thanks a ton.
2017:12:27 05:16:39 flyboarder @shdzzl thats what s/cat is doing (s/cat :needs (s/* symbol?)) is different than when you move that into it’s own spec
2017:12:27 05:17:39 shdzzl Ooohhh! When written like that it's more obvious.
2017:12:27 05:18:18 flyboarder rule of thumb for (s/cat) build it up one arg at a time
2017:12:27 05:18:31 flyboarder otherwise it’s hard to figure out where it fails
2017:12:27 05:19:07 shdzzl Good advice.
2017:12:27 05:29:17 shdzzl While I'm here, I had another question. Is it possible/advisable to spec constructors (`MyRecord.` for example)?
2017:12:27 05:35:55 shdzzl It touches on it in the spec guide, but I was curious if someone had tried it with fdef or similar.
does anyone have an explanation for why the above completes almost instantly but my repl does hang during the 10s of seconds that it actually takes to check the symbol in question?
2017:12:27 05:40:10 tbaldridge @johanatan what does that code print?
2017:12:27 05:40:40 johanatan which code? it just prints the elapsed time line, then hangs until the check actually completes
2017:12:27 05:40:49 johanatan then i get my REPL prompt back to execute more lines
2017:12:27 05:40:52 tbaldridge and what does it return when it completes?
2017:12:27 05:42:12 johanatan let me check. didn't wait for it to complete before because i was trying to debug why a loop/recur was spinning constantly instead of waiting for check to complete
2017:12:27 05:45:36 johanatan @tbaldridge it returned the typical map result from check
2017:12:27 05:48:57 tbaldridge So one last thing to try:
2017:12:27 05:49:08 tbaldridge (do (time (stest/check a-symbol)) nil)`
2017:12:27 05:49:43 tbaldridge bleh you get the point, wrap it in a do, nil. that will remove any questions about REPL printing taking a long time
2017:12:27 05:49:49 tbaldridge which is normally what it is in cases like this
2017:12:27 05:50:55 johanatan hmm, that actually returned immediately like before but also gave me back control at the REPL prompt (i'm assuming the check is happening in the bkg and i do have a process called main doing a lot of CPU work in my procmon)
2017:12:27 05:51:43 johanatan [i think main is the cider repl server if i'm not mistaken-- last time i killed it, cider immediately printed out a bunch of stuff indicating unhappiness lol]
2017:12:27 06:04:14 johanatan ah
2017:12:27 06:04:15 johanatan found it
2017:12:27 06:04:23 johanatan check returns a lazyseq
2017:12:27 06:04:39 johanatan so wrapping it in doall seems to have done the trick
2017:12:27 06:05:16 johanatan [i suppose the result wasn't being dropped on the floor previously because the repl itself was waiting for the value to be returned so that it could print it]
2017:12:27 06:19:26 johanatan thanks for the help!
2017:12:27 06:19:54 flyboarder @shdzzl sorry I havn’t tried that one
2017:12:27 06:21:59 shdzzl No problem, I'm going to look into it when I get the time.
2017:12:28 14:57:15 romni is there a way to spec a map using a coll of types/keywords (instead of a single type/keyword as with multi-spec)? the idea is that all specs corresponding to matched types are merged and used to validate the map
2017:12:28 15:02:55 romni what i have is { ::types [:first :second] } and what i'm looking for is a way to create something like a multi-spec which dispatches on the values in the coll corresponding to ::types and which will build a spec like (s/merge ::first ::second)
2017:12:28 15:12:19 ikitommi Hi. I have a vector of s/keys specs and I would like to merge them, without eval. As s/merge is a macro, I guess I should use merge-spec-impl?
2017:12:28 19:53:52 ikitommi I’m trying to combine stuff with s/merge. Is this intentional or a bug?
2017:12:28 20:04:43 ikitommi But as the s/spec is macro too, with my arbitrary vector-of-specs input, I would have to call s/spec-impl instead to avoid eval and figure out the forms somehow. Any news on the new functional core for clojure.spec?
2017:12:29 13:31:47 souenzzo I think that this is a bug @ikitommi
2017:12:29 14:34:18 ikitommi thanks @souenzzo, wrote CLJ-2302. As a workaround, I’m forcing all the incoming spec to be Specs (or names of Specs):
2018:12:31 10:09:04 ikitommi Hi. Is there or is someone working with a spec error reporter that could hint on misspelled keys or non-defined (directly on s/keys) map keys?
2018:12:31 10:11:26 ikitommi I have a s/keys spec for configuration with all the keys optional. Normal s/keys is not helping much here.
2018:12:31 10:15:02 ikitommi spec-tools has the utilities for pulling out all the keys & merges and could built on that, but would rather use something ready for this.
2018:12:31 11:21:22 dominicm @ikitommi figwheel has one
2018:01:01 15:34:58 ikitommi Thanks @dominicm for the pointer. Looks great, seems to lean on custom strict-keys Spec to catch the typos. Coudn’t get it working as it uses really old version of spec, but will poke around to see if I could get it working. Different artifact name btw in Clojars and in Github.
2018:01:01 19:12:50 drcode Hi everyone- I have a clojure DSL that comes in two variants that are about 95% the same and I want to write a single spec that validates both variants- What is the best way of doing this? Basically what I want to do is write about a hundred clojure.spec/def declarations that are the same for both variants, but a few deeply nested items will need idiosyncratic logic, looking something like:
I know I can solve this through some nasty macrology (that maintains two separate namespaces of specs for the two variants, so that the idiosyncratic differences can be "spliced in") but usually it is a bad idea to lean heavily on complex macros... Thanks for any tips!
2018:01:01 19:43:20 gfredericks aaron brooks was just talking to me on twitter about this sort of thing
2018:01:02 15:46:10 pablore I thought that could solve my validation problem with my function, but it still failing :c
2018:01:02 15:46:26 pablore
Couldn't satisfy such-that predicate after 100 tries. {}
2018:01:02 16:00:04 taylor that could be due to one of your specs using an s/and and not being able to generate something that conforms to the whole spec
2018:01:02 16:03:52 taylor for example ☝️ that s/and is generating based off that first map? predicate, but the generator is unlikely to produce a map with :foo and so it gives up after so many tries
2018:01:02 16:29:00 pablore Cannot fix it for now. Will put it on hold until errors are more specific
2018:01:02 16:29:13 pablore thanks anyway for the info
2018:01:04 02:34:24 cfleming Is there a way to make the spec optional and star operators greedy?
2018:01:04 02:34:42 cfleming I have the following spec:
2018:01:04 02:35:10 cfleming This is horribly ambiguous as written, due to the use of any? and repeated/optional elements.
2018:01:04 02:35:50 cfleming I’d like to make the first s/? greedy, so to say if it’s possible to match the :binding part then that’s what should happen.
2018:01:04 02:36:26 cfleming The only alternative I can think of is to do this, taking advantage of ordering precedence for alternates:
2018:01:04 02:36:36 cfleming But that’s kind of ugly.
2018:01:04 02:38:20 cfleming Is there any defined precedence of optional elements inside an s/cat?
2018:01:04 04:11:14 alexmiller In short, no. The way the regex derivative stuff works is kind of doing all the options in parallel
2018:01:05 02:56:30 cfleming FWIW other regex parallel implementations allow this even though they work in lockstep with no backtracking.
2018:01:05 02:56:59 cfleming e.g. NFAs or VM approaches
2018:01:05 03:23:45 cfleming I think it would be worth specifying in the doc if ops like ? and * are greedy or not - it’s not mentioned right now and is useful information in ambiguous cases.
2018:01:04 04:18:34 alexmiller I don’t know that I understand what you’re trying to spec well enough but the latter seems more readable to me
2018:01:04 04:20:45 alexmiller One thing I’ve found is that spec’ing a dsl is a pretty good way to judge how good your dsl is :)
2018:01:04 06:17:03 cfleming This is for Clara Rules
2018:01:04 06:17:13 cfleming So I can’t easily modify how it works.
2018:01:04 13:00:14 mikerod > One thing I’ve found is that spec’ing a dsl is a pretty good way to judge how good your dsl is :)
Yeah, in the case of Clara, there is a bit too much ambiguity in this case I’d say. Have to figure out a way to tighten it down a bit perhaps. Spec to the rescue.
2018:01:04 16:34:32 bbrinck I seem to recall that someone published a repository with some specs for core functions (beyond what is currently included in https://github.com/clojure/core.specs.alpha), but now I can’t find it. Anyone else remember this?
2018:01:04 16:35:23 bbrinck I realize that these specs may be incomplete or buggy, and that’s OK for my use case. This is just for toying around with instrumentation.
2018:01:04 19:21:19 alexmiller the repos I’ve looked at were both mostly wrong and not very good examples of specs, so ymmv
2018:01:04 19:23:15 bbrinck @alexmiller Good to know. I’m not that concerned about accuracy for this case, but if they are not well-written specs, that’s more of an issue, since I’m using these as a temporary proxy for the real clojure.core specs
2018:01:04 19:24:37 bbrinck I should clarify: I’m not worried about completeness in the sense that if, say, some functions are missing specs for certain less common arities, that’s OK. If the specs are just buggy in general then that would cause an issue.
2018:01:04 19:26:31 alexmiller from what I’ve looked at, the specs are wrong, as in will both allow bad things and reject good things
2018:01:04 19:27:22 alexmiller and most of them spec a bunch of things that instrument won’t work with (inline functions and functions with primitive args), which is not bad, but also not useful
2018:01:04 19:27:23 bbrinck Gotcha. Yeah, for my case, rejecting good things will cause issues. I’ll take a look.
2018:01:04 19:29:05 bbrinck > bunch of things that instrument won’t work with (inline functions and functions with primitive args)
I’m afraid I don’t understand. Can you expand on this a little?
2018:01:04 19:39:23 alexmiller instrument currently doesn’t work with: functions taking/returning primitives (fixable), inlines, multimethods (fixable), and protocols
2018:01:04 19:39:44 alexmiller so spec’ing + will buy you nothing for example
2018:01:04 19:40:32 alexmiller but it’s not harmful and does get included in docs, which might be valuable in and of itself
2018:01:04 20:16:36 bbrinck @seancorfield @alexmiller Ah, thanks. I did not know about that. If this works, I’ll be using instrumentation in CLJS, so presumably the primitive issue will be different
2018:01:04 20:29:15 bbrinck Well, I didn’t mention the CLJS part before 🙂 and in any case, those specs are in CLJ, so I can see why you thought CLJ (I’ll need to copy them into a cljs file if I want to use them).
2018:01:04 22:10:26 cfleming @kenny I think because a map is a coll. Presumably s/coll-of calls seq on the map to get at the elements, which in this case are entries. Then s/cat calls seq and gets the entry key and val.
2018:01:04 22:10:41 kenny Shit, that always gets me.
2018:01:04 22:10:44 noprompt 😄
2018:01:04 22:11:58 kenny Thanks guys.
2018:01:05 00:34:36 andy.fingerhut I am guessing that if some function A included in Clojure calls function B, and I instrument B, then because of direct linking within default Clojure JAR, A will still call un-instremented version of B? And building my own Clojure with direct linking disabled is probably best way to enable A to call instrumented version of B?
2018:01:05 00:36:15 alexmiller That is all correct
2018:01:05 00:36:50 andy.fingerhut Makes sense. Thx
2018:01:05 00:37:07 alexmiller There is a build property you can set in the Clojure build (we run the tests both with and without on CI)
2018:01:05 00:52:46 andy.fingerhut I must be doing it wrong. I've tried "mvn -Ddirectlinking=false clean package install", and even edited build.xml to change directlinking to false in there, but instrument'd B isn't causing A-to-B call to fail as I think it should. I'll keep experimenting to see where I'm going wrong.
2018:01:05 00:58:14 bronsa @andy.fingerhut beware of inlining maybe
2018:01:05 00:59:11 andy.fingerhut A is clojure.data/diff, or actually a function/protocol implementation that it calls, that calls B = clojure.set/union, where clojure.set/union is the function I'm trying to instrument.
2018:01:05 01:01:39 andy.fingerhut Ugh. I think I forgot that build.xml only affect ant builds, not mvn builds.
2018:01:05 01:13:39 seancorfield (or at least that's what we have in our script for production processes -- and we assume it's false by default elsewhere)
2018:01:05 01:16:13 andy.fingerhut I believe that flag is what tells the Clojure compiler whether to direct link user Clojure programs, or not. It doesn't seem to affect whether Clojure core code is compiled with direct linking (i.e. it still seems to have direct linking between my A and B functions that are both within Clojure itself)
2018:01:05 01:16:40 andy.fingerhut I can avoid direct linking within Clojure itself by building it from source after changing true to false for directlinking inside of pom.xml
2018:01:05 01:16:56 andy.fingerhut I haven't found the right incantation to make that happen from the command line, without editing pom.xml
2018:01:05 01:16:57 seancorfield Ah, OK, misread what you were attempting then. Sorry.
2018:01:05 01:21:19 alexmiller You can override it (that’s what the clojure matrix builds do). Maybe that flag is only for test compilation. Sorry, on my phone and haven’t looked at it in a couple years.
2018:01:05 01:22:21 andy.fingerhut @alexmiller No worries. Hand-editing pom.xml is doing the trick for me, and I don't need it automated for any personal reasons. I suspect you may be correct that the command line options might only be affecting compilation of Clojure tests.
2018:01:05 01:23:57 alexmiller Oh, it is just for testing - see the profile definitions in the pom
2018:01:05 01:24:10 alexmiller Activating the different profiles will change the flag
2018:01:05 01:25:05 alexmiller You change the compilation for Clojure itself in build.xml
2018:01:05 01:25:23 alexmiller In the compile-clojure target
2018:01:05 01:25:38 alexmiller There is a sysproperty there
2018:01:05 01:25:59 alexmiller I think you’ll have to hard code that change
2018:01:05 01:26:22 alexmiller Tis a tangled web
2018:01:05 01:34:39 andy.fingerhut If I get a reproducible way to build Clojure in this way, is it of any use to add it to dev wiki ?
2018:01:05 01:58:34 alexmiller Sure, although I’m not sure how wide the need is for it
2018:01:05 01:59:17 alexmiller Stu sometimes runs with just the src too, no aot at all
2018:01:05 08:29:44 romni is there any way to write a spec for a multimethod?
2018:01:05 13:19:33 alexmiller Currently, no. You can separate the dispatch function though and spec that if you like
2018:01:05 15:02:03 romni thanks, that's what i've been doing so far but i was wondering if spec contained some syntax sugar for this
2018:01:05 09:28:19 andy.fingerhut @alexmiller I understand the need is not very wide for building Clojure without direct linking, but I took a shot at adding a short section with instructions for how to do so at the end of this wiki page. Feel free to delete it if it is not useful enough, or edit otherwise to your heart's contents: https://dev.clojure.org/display/community/Developing+Patches
2018:01:05 12:36:25 misha how do I both have
(s/and vector? (s/cat :a int? :b (s/* string?)))
and generator for free? this exact form gives up on generating vector? part
2018:01:05 12:38:36 misha I do have to use gen/let, etc. don't I?
2018:01:05 12:41:20 gfredericks ideally the API would be such that you could specify that the default generator for (s/cat :a int? :b (s/* string?)) be transformed with (gen/fmap vec ___)
2018:01:05 12:41:28 gfredericks I don't think the spec API makes that easy though
2018:01:05 12:44:51 misha so I need to extract s/cat as dedicated spec, and use it both in s/and spec definition and generator
2018:01:05 12:45:02 misha or is there a "shorter" way?
2018:01:05 13:18:39 alexmiller cat + vector is something that is quite difficult to do atm while retaining conform, generator, form, etc and a known gap. There are various ways to get parts of those but it’s hard to get all of them. This is certainly something I’ve run into and you can see it in some of the core dsl specs. I’ve come to the conclusion that a built in s/vcat would solve all of the cases I’ve run into but I think Rich is looking for a more composable solution still
2018:01:05 13:30:46 misha s/vcat is definitely an ok solution for an application code, but smells like "giving up" in a library opieop
2018:01:05 13:34:18 alexmiller Which is one reason this is still unresolved :)
2018:01:05 13:43:55 alexmiller There is overlap here between regex ops and coll-of which both work on colls. Ideally those features could be more easily composed
2018:01:05 16:21:09 misha Is there an "easy" way to overwrite :min-count:max-count of the s/coll-ofs/map-of specs' generators "inline"? e.g. for REPLing. Or will I have to provide brand new generators for those and use them in s/with-gen?
2018:01:05 18:56:14 alexmiller the generators will respect the min/max count you gave them
2018:01:05 18:56:52 alexmiller but you can do something like (s/with-gen (s/coll-of int? :max-count 100) #(s/gen (s/coll-of int? :max-count 3)))
2018:01:05 19:30:02 misha I am asking, if there is shorter way than you wrote. Many colls-of I speced as a standalone specs, and it would be cool to reuse those with something like
2018:01:05 19:33:30 misha 2 main use cases for this for me, are:
- do not generate too many, because I want to eyeball it.
- do not generate too few, like (s/exercise) tends to do for first few samples (or all, if you give it small sample size (s/exercise ::my-spec 2))
2018:01:05 19:41:14 alexmiller in short, no. you can give exercise a generator override map though
2018:01:05 19:48:09 misha I remember that, but it is basically the same effort stuff-typing- and copypaste-error-wise for these 2 usecases.
thank you
2018:01:05 22:45:39 seancorfield When I have (s/conform (s/merge ::spec-1 ::spec-2) my-data) this only seems to conform values in the last merged spec, not the first one. If you swap the spec order, you get conformedvalues from ::spec-1 instead but not ::spec-2 -- is that by design or is it a bug in s/merge? /cc @alexmiller
2018:01:05 22:47:05 seancorfield (if I change s/merge to s/and I get conformed values from both specs)
2018:01:05 22:51:29 alexmiller By design s/merge does not flow conformed values
2018:01:05 22:57:25 seancorfield OK. That was just surprising, given the docstring.
2018:01:05 22:59:13 seancorfield So I'm going to need (s/and (s/merge ::spec-1 ::spec-2) ::spec-1) for my particular use case (so that it still generates), right @alexmiller?
2018:01:05 22:59:33 seancorfield (I know that checks ::spec-1 twice)
2018:01:05 23:02:46 alexmiller The generator for merge will produce maps that validate for both specs
2018:01:05 23:03:47 alexmiller So you shouldn’t need the and here for generation
2018:01:05 23:05:16 seancorfield Right, but I need the s/and to get conformance flowing through the specs.
2018:01:05 23:05:49 seancorfield If I want both conform-flow and generation, I need both the s/and and s/merge.
2018:01:05 23:06:09 seancorfield The s/merge on its own doesn't flow conformance (but will generate fine).
2018:01:06 08:42:16 ikitommi spec-tools is already doing the rogue coercion stuff, so woudn't make the lib any more bad in that sense.
2018:01:05 23:06:31 seancorfield The s/and on its own doesn't generate properly (but will flow conformance).
2018:01:05 23:08:12 seancorfield This is because we have specs that conform API parameter strings to long, double, date etc -- and I want to be able to combine them and get full conformed values across the combination and get generation across the combination 🙂
2018:01:05 23:13:58 alexmiller Didn’t anyone ever tell you not to do coercion?
2018:01:06 11:12:41 misha is there anything to read at length about "don't do the coercion"? It makes sense here and there, but I'd like to see the bigger picture
2018:01:06 14:38:19 alexmiller If you build coercion into your spec, particularly lossy coercion, then you have made a decision for all consumers of your spec that they have no choice over
2018:01:06 17:02:52 gfredericks is the recommendation for coercion use-cases "do it by hand outside the spec'd zone"?
2018:01:06 18:40:52 misha so objection is not about coercion, but about complecting it with specs? Or even about lossy (in any way) specs?
2018:01:06 20:28:46 ikitommi “they (spec consumers) have no choise over” => wouldn’t CLJ-2116 fix just that?
2018:01:06 22:57:08 alexmiller @misha right. I have no problem with coercion as a concept. :)
2018:01:06 23:23:41 gfredericks I on the other hand
2018:01:07 03:33:10 seancorfield As someone who uses coercions in some specs, I wouldn't support CLJ-2116. It enshrines pluggable coercion into clojure.spec which is not the right approach IMO.
2018:01:08 01:34:00 aengelberg I'm trying to write a util that prints out a more friendly error message based on the data returned from s/explain-data. I'm noticing that :in contains misleading data for certain cases:
So for a map-of, it is giving me the key as well as 1 (to indicate the val of the map pair), and for a set, it is giving me an index into some arbitrary ordered version of that set. Neither of these would work if I tried to get-in on those paths.
If these values were somehow prefaced with an indicator of what special case triggered them, I could work with that, but the fact that the data may or may not be misleading with no additional context makes the overall data unhelpful for writing a universal tool. Is that a bug? Or is there another way of introspecting the failed specs that I'm missing?
2018:01:08 01:58:30 alexmiller on the first case, this is a side effect of treating maps as a sequence of tuples. there is at least one ticket about this particular one with a pending patch.
2018:01:08 01:58:48 alexmiller for the set case, I don’t think anything is logged but a ticket for that would be fine
2018:01:08 09:21:53 aengelberg Thanks, I'll try to find time to write up a ticket. I'll definitely check out the other ticket you mentioned for map-of, since I'm curious what the proposed solution was, so I know what the expected behavior should be for sets.
2018:01:08 16:08:16 bbrinck IIRC, https://github.com/athos/Pinpointer takes a different approach to the problem: it analyzed the specs directly to help understand the explain-data (I could be mis-stating something here, @U0508956F please correct me if I’m wrong 🙂 )
2018:01:08 19:36:50 aengelberg @U08EKSQMS thanks! do you have any specific thoughts on the set issue as well? are you also handling that specially from expound?
2018:01:08 19:58:27 bbrinck This could have bugs though. It took awhile to figure out an approach that would work for most :in paths that I could find.
2018:01:08 20:00:05 bbrinck Re: sets, I would vote for any JIRA ticket that makes :in paths non-ambiguous, because I think this is one of the biggest hurdles with building pretty-printers for spec. If you make an issue for the set case, send me the link 😄
2018:01:08 21:10:01 cfleming Thanks @U08EKSQMS, I’ve been meaning to take a closer look at Expound actually
2018:01:08 22:09:37 bbrinck Cool! Feedback is welcome and appreciated
2018:01:08 10:38:21 cfleming @aengelberg I’d like to see that too if you find it.
2018:01:08 12:02:23 misha is it overkill?
(s/nonconforming (s/or :i pos-int? :z #{0}))
2018:01:08 12:04:10 misha - zero? throws on strings, keywords, etc.
- just (complement neg-int?) accepts everything but neg ints: strings, maps, etc.
2018:01:08 12:11:13 misha is this in any "under-the-hood" way lighter?
(s/and int? (complement neg-int?))
2018:01:08 13:34:09 alexmiller What about nat-int?
2018:01:08 14:54:48 misha missed it opieop
2018:01:08 20:46:07 dadair Given the following data: {:a {:name "a"} :b {:name "b"} :c {:name "not-c"}}, how could I construct a spec to say "the key to a value-map must be the keywordized name within the value-map"? This spec would complain that (keyword "not-c") does not match :c
2018:01:08 21:01:49 alexmiller You can use any arbitrary predicate like #(= (keys %) (->> % vals (map :name) (map keyword))) to check validity
2018:01:08 21:02:11 alexmiller but it’s not going to give you that specific error
2018:01:08 21:09:05 dadair great thank you
2018:01:09 07:11:07 athos In that case, I'd prefer something like (s/coll-of (fn [[k v]] (= k (keyword (:name v)))))
2018:01:09 18:30:13 dadair cheers! I like that too
2018:01:09 12:24:24 Charles Fourdrignier Hello,
I would like to write a client for testing some servers' responses.
Not for production purpose, but for coding events like coderetreats (inspired by https://github.com/rchatley/extreme_startup).
I'm trying to implement this with Clojure (http-kit) and think to Clojure Spec as a validator.
I'm a beginner with Clojure and Clojure Spec and I don't know if it's a "good" idea, absolutely ridiculous or in the "Clojure's spirit".
I write two specs to demonstrate the concept.
https://gist.github.com/Charlynux/35c754f3f213012397da8ae61b97cb1c
I would appreciate any feedback about the idea, code...
2018:01:09 15:12:20 bbrinck I don’t see any problem with the idea of using spec to validate responses. Depending on how you intend to use the spec, there may be some ways you can improve it so you get better error messages when it fails.
2018:01:09 15:27:33 bbrinck For instance, it might be useful to use a multi-spec here https://clojure.org/guides/spec#_multi_spec or maybe use s/or so you can figure out which case matched with conform. It depends on whether you want to just make sure the response is any type of valid response (i.e. 200 and 404 will both be OK, as long as well formed) or if you want to use spec to determine if a response was successful or not
2018:01:09 12:51:51 mpenet oh, maybe not then, it just broke in my repl, hmm
2018:01:09 13:53:00 madstap That's why it works here @mpenet
2018:01:09 15:59:26 bhauman @alexmiller I heard Rich mention in his last talk that next version Spec would be more "programmable" or something like that. Is there anything more I can read about that? Or can I ask about specifics?
2018:01:09 16:07:50 mpenet my guess: either it's something related to https://dev.clojure.org/jira/browse/CLJ-2112 with unforming that re-creates a new spec, or something entirely new
2018:01:09 16:36:11 bhauman @mpenet Thanks, for the pointer.
2018:01:09 16:53:16 alexmiller Nothing to read yet and not related to 2112
2018:01:10 08:10:07 mpenet Would it be considered ok to promote s/assert* to good for "public" use (atm the docstring says : "Do not call this directly, use 'assert'.")
2018:01:10 08:10:27 mpenet in cases where you want the check to always happen no matter the value of check-assert/compile-asserts
2018:01:10 08:10:55 mpenet or is there a better way? ping @alexmiller
2018:01:10 08:13:36 mpenet I guess if it would be used that way it could warrant a rename maybe
2018:01:10 10:33:52 Andreas Liljeqvist I seem to remember that I could generate with a provided example. Something like (gen/fill-in ::spec {::id "all generated will have this id, other values are generated"})
2018:01:10 10:34:12 Andreas Liljeqvist Can't remember what it was called though... anyone?
2018:01:10 11:16:28 Andreas Liljeqvist @misha yes, thank you!
2018:01:10 16:56:12 misha
(defn gen
"... Optionally an overrides map can be provided which
should map spec names or paths (vectors of keywords) to no-arg
generator-creating fns. These will be used instead of the generators at those
names/paths. Note that parent generator (in the spec or overrides
map) will supersede those of any subtrees.
parent generator will supersede those of any subtrees part just bit me in the ass harold
2018:01:10 16:57:51 misha but it forces me to keep generators and specs separate, which ends up to be cleaner
2018:01:10 21:39:39 pablore I have some function
(defn foo
[a]
{:pre [(s/valid? ::bar a)]}
...)
That asserts if the input conforms to a spec. It works well, but I’m having trouble to debug when it fails, I want to know what input I’m passing that doesnt conform and why.
2018:01:10 21:47:27 pablore I just get an AssertionError
2018:01:10 22:18:27 misha @pablore I use s/explain-data instead of s/valid. it returns nil if everything is ok, and a datastructure, if there are any errors.
2018:01:10 22:19:23 misha So you can use it in if or when.
2018:01:10 22:24:29 pablore s/explain works better than I expected!
I know of s/fdef but I’m writing a really complex reducing function with lot of if-cases.
2018:01:11 01:36:22 andy.fingerhut Does s/every on a map check all key/value pairs as 2-tuples (or at least any that it does check, since I know that s/every does sampling)
2018:01:11 02:47:08 alexmiller Yes - map entries are 2-tuples
2018:01:11 02:47:38 alexmiller I wrote a blog about the destructuring spec btw
2018:01:11 15:03:32 pablore but conform is returning {:id 1} instead of {::id 1}.
2018:01:11 15:14:41 alexmiller s/keys conform doesn’t modify keys. because you’re conforming a map with unqualified keys, the conformed map will have unqualified keys
2018:01:11 15:15:20 alexmiller you could also use :req with data that looks like {::id 1}
2018:01:11 15:42:37 Andreas Liljeqvist I have a problem with multi-spec and overriding generators
2018:01:11 16:57:20 alexmiller Isn’t the question really why the generator override doesn’t work in the first exercise?
2018:01:11 17:03:00 Andreas Liljeqvist Sorry, running on low sleep - Yes
2018:01:11 17:06:40 alexmiller doesn’t seem like the generator overrides make their way to the per-method spec generators, not sure why
2018:01:11 17:07:08 alexmiller looking at the code, it certainly seems like things are done with that intent, not sure why that’s not working, feel free to file a jira
2018:01:11 17:09:01 Andreas Liljeqvist ok, thanks. Will file a report tomorrow
2018:01:11 17:09:45 alexmiller the multispec generator drives from the known dispatch method values, not from the method itself, so that may also play into it
2018:01:11 22:32:47 dadair anyone know why a call to s/exercise would throw FileNotFoundException Could not locate clojure/test/check/generators__init.class or clojure/test/check/generators.clj on classpath. clojure.lang.RT.load (RT.java:458)?
2018:01:12 15:33:06 stijn how can you define or alter specs programmatically? I have my data model defined in an edn file and like to generate specs based on that
2018:01:12 23:58:16 gklijs It's because in the first one you suply a symbol, while in the second a list. Not really sure why though.
2018:01:13 00:23:32 kenny The asymmetry does not make sense to me. Either both should throw or both be allowed.
2018:01:13 02:11:26 seancorfield The (s/keys :req (conj my-keys ::b)) is treated as (s/keys :req [conj my-keys ::b]) so it requires ::b but the other two elements are symbols so s/keys probably doesn't handle them as expected (it certainly does not evaluate them).
2018:01:13 02:11:54 seancorfield Hence, when you supply a map containing ::b it validates and when it doesn't contain ::b it is invalid.
2018:01:13 02:13:15 seancorfield I expect that deep within the implementation of s/keys, the (unevaluated) symbols conj and my-keys are processed in such a way that they fail to be treated as keys at all. @kenny
2018:01:13 02:14:49 seancorfield Ah, looking at the source of s/keys it filters out anything that isn't a keyword:
2018:01:13 02:16:42 kenny I always forget the implicit checking of keys.
2018:01:13 02:17:30 seancorfield It looks like s/keys tries to treat anything that isn't a keyword in the sequence supplied to :req or :req-un as some sort of predicate but I'm not quite sure I follow all the parts of the macro...
2018:01:13 02:18:24 kenny Doesn't Spec auto-check any keys in a map against any specs defined for those keys?
2018:01:13 02:18:43 kenny Which would explain why
{::a "a"
::b "b"}
is invalid
2018:01:13 02:19:29 seancorfield Ah, I see what it's doing with those symbols:
2018:01:15 03:02:49 gfredericks this makes me think the answer will be "you can't use sorted sets as specs"
2018:01:15 03:03:11 andy.fingerhut I looked for a CLJ issue that I thought existed about making set lookup on sorted sets never return an exception because of exceptions thrown by the comparison function, but not sure if I am imagining it.
2018:01:15 03:03:19 gfredericks even if it didn't fail at that point, a sorted set is not a general predicate 😕
2018:01:15 03:04:05 gfredericks making a change like that would be weird because what if you supply a buggy comparison function to sorted-set-by, for instance? it would just swallow them
2018:01:15 03:04:28 gfredericks it could throw them on insert I guess, but would swallow on lookup
2018:01:15 03:11:13 andy.fingerhut Found several related tickets on sorted sets, but not the one I was imagining. My imagination can be vivid.
2018:01:15 09:36:48 bronsa @andy.fingerhut yeah I know which ticket you’re talking about
I can do this by using (s/form ::banana) but that just gives me a sequence so seems very brittle
2018:01:15 15:14:34 andre.stylianos I don't think there's a better way for now, there's an issue in Jira that seems likely to be what you want but it's still open.
https://dev.clojure.org/jira/browse/CLJ-2112
2018:01:15 15:31:09 ikitommi @U053032QC forms have been stable, so (->> (s/form ::banana) rest (apply hash-map)) while waiting the specs for specs.
2018:01:15 15:37:01 conan @U485ZRA58 yes that does look like what I want.
@U055NJ5CC yep, it certainly gets me exactly what i want, it just seems a bit wonky
2018:01:15 17:00:30 alexmiller there is likely to be more support for this in the next major update for spec
@U064X3EF3, you always say to "not use conform to coerce values". In this case, we are using conform to "parse" a struct. It's a valid use?
read the topic
2018:01:15 17:19:50 alexmiller I don’t think I’ve said that. conform is designed to conform values so I’m not even sure what you mean.
2018:01:15 17:21:11 alexmiller this seems like a pretty ugly workaround (the embedded s/def seems like an obvious place where you could have issues too)
2018:01:15 17:24:36 souenzzo (it's just to fit on let to easy copy'n'paste on repl)
I think that you say do not use conform to conform values related to CLJ-2116
2018:01:15 17:25:32 alexmiller do you mean “do not use conformers to coerce values” which is a very different statement?
2018:01:15 17:25:55 alexmiller that seems wholly unrelated to your example if so
2018:01:15 17:27:09 souenzzo YEP. Sorry. I got lost with the words 😞
2018:01:15 17:30:02 alexmiller I don’t think what you’re doing here is related to coercion
2018:01:15 17:30:12 alexmiller which is independent from whether it’s good :)
2018:01:15 17:30:53 alexmiller my real question is why you want to ask this question in the first place and whether you can arrange things to not ask it
2018:01:15 23:04:09 Steve Peterson Hi Spec-cers! Which Clojure libraries are using Spec as a protection layer in their APIs, i.e. to validate args? Are they instrumenting API fns “by default” so input args are checked on each call?
2018:01:16 00:06:57 seancorfield @stevep I can tell you what we're doing in production with clojure.spec at World Singles as far as REST APIs are concerned: we have a spec for each handler (i.e., each API endpoint) and the general pattern is roughly:
2018:01:16 20:44:35 arnaud_bos (how-)do you prevent un-spec-ed keys to propagate further down the normal-process?
2018:01:16 22:15:27 seancorfield @U1DLD0WNR In keeping with Clojure's open-for-extension approach, normal-process will just ignore keys it doesn't care about. Similarly, if I only spec certain keys, others don't get checked -- so the constraints are open.
2018:01:16 00:08:24 seancorfield We have that wrapped up in a macro to reduce boilerplate but that's the essence of it. The error-response function decodes the spec failure based on a set of heuristics and "expected" failure points so we can return documented error codes and localized messages to the client.
2018:01:16 00:09:19 seancorfield We only instrument functions in our dev/test cycle, not in release code. We also have specific "tests" that run clojure.spec.test/check on certain functions.
2018:01:16 00:10:01 seancorfield The argument against just relying on instrument to check arguments is that you want better error messages than spec failures for your clients!
2018:01:16 10:31:00 vikeri Can I modify to a spec after I’ve defined it? My use case is to add the :fn argument for an fdef but only when I run my tests.
2018:01:16 11:22:37 conan @vikeri why can't you just add it anyway? instrument doesn't run the :fn and :ret bits by default anyway
2018:01:16 11:22:52 conan i use orchestra.spec.test/instrument to do this
2018:01:16 11:24:27 vikeri @conan I also use orchestra and it runs the :fn by default so that’s why I have that issue
2018:01:16 11:25:03 conan ah, can you swap out which you use in different circumstances? so you can switch on/off the :fn execution
2018:01:16 11:25:35 conan i think you can just call clojure.spec/fdef again fine, and it'll update the spec registry with whatever you give it
2018:01:16 12:55:16 alexmiller You can specify an alternate spec to use with instrument
2018:01:17 19:30:56 borkdude I cannot pass a keyword as function to s/conformer, since it will try to resolve it as a spec. e.g.: (s/conformer (fn [parsed] (:foo parsed))) is needed right?
2018:01:17 19:31:50 borkdude Other question: I vaguely remember something about not using spec as a string/text parser from this channel, but what exactly was the argument against it?
2018:01:17 19:48:42 seancorfield The main objection is that is complects parsing and validation -- and "forces" the parsing effect on all clients of the spec.
2018:01:17 19:49:37 borkdude forces the parsing effect?
2018:01:17 19:51:29 seancorfield (what we tend to do is have a spec that accepts both the target type as well as a string that could represent the target type ... if the parse/conversion fails, the result is ::s/invalid; if the parse/conversion succeeds or if the target type was provided, the validation is performed)
2018:01:17 20:26:14 donaldball Do you have an example you can share of this style?
2018:01:17 21:13:08 seancorfield I've shared it before here but it'll take me a while to de-tangle the generic code from the proprietary code so don't hold your breath for me to get around to it (too busy this afternoon and I'll probably forget after that).
2018:01:17 21:16:45 seancorfield I probably ought to just open source it at some point...
2018:01:17 21:20:36 donaldball 🙂
2018:01:17 20:59:14 alexmiller my main objection to it is that the regex op capabilities of spec are designed to support the structure of data and only incidentally can be used as a way to describe the structure of strings. Inevitably, I suspect you will find either the capabilities or performance of “spec as string parser” to be lacking and will be disappointed in our lack of interest in addressing those. There are very good tools already in existence for creating parsers on strings and I those are the right tool for the job.
2018:01:17 21:15:30 seancorfield And to be clear, what we do at World Singles is not any sort of spec-as-parser -- we just have a few, very specific specs that accept a string or a target type and, for the string, do a simple conversion. Those target types are Long, Double, Boolean, and Date. We don't attempt to do anything else via spec -- this is just for simple form or query string data.
2018:01:17 21:18:20 borkdude I was surprised spec’s performance was even better than Instaparse (see link above). Handwritten parser is fastest, Kern in between.
2018:01:17 21:19:10 borkdude But yeah, I understand there is no interesting in optimizing spec for strings, although it does quite well
2018:01:17 21:21:47 donaldball I was also fairly delighted at how both expressive and fast spec was when validating strings as seqs of chars, at least for my silly use cases.
2018:01:17 21:26:34 aengelberg to be fair, Instaparse has enough processing overhead that parsing one or two characters at a time in a 47 KB string isn't the best (or most common) use of it 🙂
2018:01:17 21:28:31 borkdude @aengelberg yeah, I’ve discussed this example with you before in #instaparse I believe, it’s part of Advent of Code. Nothing conclusive, just very contextual.
2018:01:17 22:41:44 alexmiller I think the perf differences are going to be highly dependent on the complexity of the grammar and the size of the data so it’s hard to make predictions from a single example.
2018:01:17 22:53:31 rickmoynihan Also instaparse is a far more general purpose parsing solution, supporting both left and right recursion; I’d be surprised if that didn’t incur an overhead.
2018:01:17 23:46:12 dpsutton i've written some specs and i put some calls to s/valid? in comment macros in the namespaces where they are. is there a good way to take this out of a comment macro and have them run at some convenient but non-production time?
2018:01:17 23:47:14 dpsutton clojurescript is the target if there's a compiler option to prevent spec checks on advanced compilation
2018:01:18 15:39:39 kingmob In Clojurescript, what are people doing if they want to remove spec from production builds? Is there a lib for it? Macros wrapping fdef in the same file? Or placing all spec stuff in a dev-only namespace?
2018:01:18 15:48:10 borkdude @kingmob just curious, but why would you want to remove them?
2018:01:18 15:48:26 kingmob Download size
2018:01:18 15:49:05 kingmob If I’m only validating/testing with spec, I don’t want it to take up sie in the resulting Js artifact
2018:01:18 15:49:36 borkdude right. putting them in a dev only ns is what I would do then
2018:01:18 15:54:49 kingmob Thx @borkdude. Thing is, I prefer fdefs be defined by their function, so I was hoping someone had already handled this. We have some dev-time-only logging macros I could repurpose… though I suspect dev-only namespaces would still be tricky, assuming it’s even possible
2018:01:18 15:56:56 borkdude @kingmob this project has an example how dev only namespaces are possible (both lein and boot): https://github.com/borkdude/lein2boot
2018:01:18 15:57:27 borkdude @kingmob you could also try in #clojurescript
2018:01:18 16:00:04 dpsutton yeah it would be nice if you could elide all spec related stuff under advanced compilation
2018:01:18 16:01:07 kingmob @borkdude Ahh, it looks like you factored out everything common into its own file and then use different :source-paths. We do that, too, but if, e.g., I added fdefs in animals.crud, they’d still end up in prod builds
2018:01:18 16:01:41 kingmob We could add it to the dev source path, but then it’s separate from the definition…which is fine, it’s not really that big a deal
2018:01:18 16:01:58 borkdude yeah, not getting around that when putting in a different ns
2018:01:18 16:02:47 borkdude have you tried what difference in size you get for the production js with and without? it may not even be worth it
2018:01:18 16:02:49 kingmob @dpsutton Yeah, it would be nice. Does spec use asserts under the hood so that the :elide-asserts option would remove it?
2018:01:18 16:04:19 kingmob @borkdude Yeah, I have actually, it’s substantial 🙂 Lemme recompile and check with source-map-explorer again
2018:01:18 16:04:35 borkdude I believe you 🙂
2018:01:18 16:09:28 kingmob @borkdude OK, bare-bones spec is adding 39.84 kb to the output .js. It’s not major, but this is just for spec itself, and a sprinkling of initial spec defs. It’s not extensive at all, and since none of it is used at run-time, we might as well ditch it.
2018:01:18 18:05:06 shaun-mahood I've been thinking about building some company/app wide specs. Say I spec something as :com.myco.app.service/id - is there any way that I can apply that spec to some data where everything is named as :service/id?
2018:01:18 20:43:41 seancorfield (whereas s/assert returns the original thing, not the conformed value)
2018:01:18 20:44:25 schmee yeah, I copied my conform thing from another project where I needed the conformed value, now I just want to validate so s/assert is perfect 👍
2018:01:18 20:46:14 seancorfield Just bear in mind it can be turned on/off by command line options and by code (including 3rd party libraries!).
2018:01:18 20:49:12 schmee hmm… using assert for config validation sounds kinda sketchy then
2018:01:18 20:50:55 schmee ahh, putting it in :pre is what I want, forgot that it existed 😛
2018:01:18 20:52:52 schmee no, that just gives an AssertionError, it doesn’t show you whats wrong….
2018:01:18 20:53:25 seancorfield (and asserts can be turned on/off by calling code via *assert*)
2018:01:18 20:53:33 uwo so, when s/keys fails on a missing required key, there’s no path information in explain-data. We’re attempting to dispatch on the result of explain-data, but the information we need about the missing key is in a :pred function. So is the only way to parse this data to parse the s expression used internally to describe required keys?
2018:01:18 20:55:40 seancorfield @uwo Yeah, that's ugly. We wrote a heuristic-based explain-data walker to figure that stuff out. It recognizes and picks apart the :pred value...
2018:01:18 20:56:45 uwo yikes. off the top of your head do you know if there’s jira ticket along these lines.
2018:01:18 20:57:09 uwo thanks, btw!
2018:01:18 21:04:00 seancorfield Off the top of my head, nope, sorry.
2018:01:18 21:05:17 seancorfield But we needed to pick apart spec errors in general and map them back to our domain for reporting to end users... it just took us a while to flesh out the walker (it's... non-trivial).
2018:01:18 21:52:07 uwo same for us
2018:01:18 21:35:58 schmee how do you go about spec-ing anonymous functions with side-effects? since conform et al actually calls the function it’s validating, I guess the answer is “don’t do it?”
2018:01:18 21:40:05 seancorfield Not sure what situation you're talking about @schmee?
2018:01:18 21:42:02 schmee I have a piece of code that takes an array of functions, and those functions should return a particular type of record
2018:01:18 21:42:57 schmee I call s/valid? on the input to this code, and if I use fspec to check the functions in the array, it will actually call all those fns to ensure that they return the correct thing
2018:01:18 21:43:37 schmee but those fns could grab values from a database, launch nuclear missiles or whatever, so I’m thinking I should just spec it as (s/+ fn?) instead
2018:01:18 21:44:38 seancorfield Yes, best not to use fspec for side-effecting functions-as-arguments because instrumentation uses generative testing on them.
2018:01:18 21:44:56 alexmiller Use ifn?, not fn?
2018:01:18 21:45:11 seancorfield An array of functions sounds more like (s/coll-of ifn?)
2018:01:18 21:46:35 schmee noted on ifn?, but why s/coll-of instead of s/+?
2018:01:18 21:48:15 alexmiller + is a regex op used for describing the internal structure of sequential collections (like an arg list). coll-of is for sequential collections. Either may be applicable here, depends on context.
2018:01:18 21:48:41 alexmiller In particular, what if anything is including this spec
2018:01:18 21:49:54 alexmiller If, for example, you’re including it as one arg in an args spec, using + would be bad as it would combine in the same sequential parent context
2018:01:18 21:50:41 schmee I see, I’m using it as part of a s/keys spec so that shouldn’t be a problem then 🙂
2018:01:18 21:51:25 alexmiller Either is prob ok then but I would prob use coll-of
2018:01:18 21:51:44 alexmiller With a :min-count constraint
2018:01:18 21:52:11 schmee I’ll go with that, thanks for the help 👍
2018:01:18 21:53:03 schmee I’m a bit torn about the fspec thing though, I think it would be much more pleasant to see (s/fspec :args (s/cat) :ret (partial instance? MyRecord) in a doc-string instead of ifn?, but I can’t due to generative testing
2018:01:18 21:53:22 schmee I can just write it out in the docstring anyway of course 🙂
2018:01:18 21:54:03 schmee I’m still new to spec so I’m kinda feeling it out, it’s very easy to overdo it and use it as a type system
2018:01:18 22:00:12 seancorfield Note that you can still run clojure.spec.test.alpha/check on your function with the full fspec -- independent of what the function's actual fdef says. (right @alexmiller)
2018:01:18 22:00:40 seancorfield Or you could instrument it with a different fspec too I believe?
2018:01:18 22:01:04 alexmiller yes
2018:01:18 22:01:48 seancorfield I'd probably fdef it with a spec that is safe to instrument, and then check it with a more detailed fspec...
2018:01:18 22:02:29 seancorfield (because it might, potentially, get instrumented by code far, far away in your code base that might "forget" to override the spec for it)
2018:01:18 23:55:40 arohner Is there a library for an s/keys-alike but keys are strings rather than keywords, yet?
2018:01:19 02:23:36 gfredericks (s/keys :req [::foo]) or (s/keys :req ["foo"])?
2018:01:19 02:24:09 gfredericks if the former, is it not syntactically possible to express all string keys? if the latter, how are the strings tied to the spec registry?
2018:01:19 20:15:26 pablore Do specs replace deftype and reify? If not, how do I use them together?
2018:01:19 20:43:09 schmee no. deftype is used for performance and polymorphism, reify is mostly used for Java interop, and spec is about specifying your data. so it’s three different use-cases really 🙂
2018:01:22 09:10:27 witek I want to spec a command vector, given (s/def ::command-name keyword?). Now the command-vector needs to be a vector where the first element must be the ::command-name. All other elements are optional arguments. Any suggestions? Thank you.
If :use-delivery-address is true, :street and :zip should not be there. If :use-delivery-address is false, :street and :zip should be there
I find it really difficult to write a spec for this. Anyone who have any ideas?
2018:01:22 09:25:18 stathissideris @karl.jakob.lind I guess :use-delivery-address is :req, the others are :opt and then you wrap your keys with an s/and to enforce the rest of the logic that you described
2018:01:22 09:31:59 jakob Well, the others are required if :use-delivery-address is true. So it's not that simple
2018:01:22 09:36:48 stathissideris yeah, but I’m proposing that this is enforced by the extra predicate that you will pass to s/and
2018:01:22 09:39:54 jakob hm.. something like this?
2018:01:22 09:58:13 jakob good suggestion @karl.jakob.lind. will read about that!
2018:01:22 16:33:24 carkh let's say i want to write a spec for a string that should be splitted in two, imagine i have a split function that can be called with that string and returns a pair of strings... i have a ::check-whole-thing spec and also a ::check-part spec ...how would i write that ::check-whole-thing part ?
2018:01:22 16:35:29 carkh there are numerous examples, but these all work on clojure data structures
2018:01:22 16:36:28 carkh i think what i'm asking here is how to transform the data being specced before speccing it further
2018:01:22 17:04:54 alexmiller have you read the spec guide? https://clojure.org/guides/spec If not, maybe take a spin through that first and then refine the question.
2018:01:22 17:06:04 carkh i did ... actually found a related question to which you responded on the forum ... the conformer function looks like it fits the use case
2018:01:22 17:06:35 alexmiller that does not seem like a use case where conformers should be used to me
2018:01:22 17:07:19 alexmiller I’m confused by several aspects of your question so not sure how best to answer it
2018:01:22 17:08:12 carkh i'm conforming a string, i want to have it transformed into a vector of strings before further conforming
2018:01:22 17:08:23 carkh could be any kind of transformation really
2018:01:22 17:08:25 alexmiller well, I would recommend not doing that
2018:01:22 17:08:29 carkh ah !
2018:01:22 17:08:32 alexmiller do that in code explicitly
2018:01:22 17:08:37 alexmiller don’t use spec for it
2018:01:22 17:08:59 carkh but semantically i'm really validating the string there
2018:01:22 17:09:08 carkh which is what spec is for isn't it ?
2018:01:22 17:09:40 alexmiller I think you are better off if you spec the inputs and outputs of the transformation
2018:01:22 17:09:47 alexmiller and explicitly invoke the transformation
2018:01:22 17:10:18 alexmiller and don’t use spec to actually do the transformation part
2018:01:22 17:11:24 carkh the thing is that i'm not really interested in that transformation at all, i'm only validating it to pass the whole thing merilly along
2018:01:22 17:14:44 alexmiller if you have to do that transformation in able to pass it along, then it seems like you are interested in it after all
2018:01:22 17:16:47 carkh let's imagine a use case for one of those html dsls ... they have this way of specifying class attributes like so [:div.app-container ...] ...that's not what i'm doing at all but that's a good example... are you saying that spec wouldn't fit to do the "destructuring" of that keyword ?
2018:01:22 17:19:08 carkh or maybe to destructure an udp packet or you know go inside a data structure and destructure it for me
2018:01:22 17:20:12 carkh that last one was a bad example =)
2018:01:22 17:23:07 carkh what's the issue here ? the fact that one could not rebuild the data structure from the conformed result ?
2018:01:22 17:23:40 ghadi to rephrase slightly, don't conflate spec/validation with transformation in one phase
2018:01:22 17:23:47 ghadi you'll be perpetually unhappy
2018:01:22 17:24:11 ghadi do your validation, do transformation, then do validation again if you want
2018:01:22 17:24:43 ghadi there are other gotcha with conform to look into
2018:01:22 17:24:58 ghadi like it forces conformance on all consumers of the spec
2018:01:22 17:25:02 ghadi which you don't want
2018:01:22 17:25:51 carkh ok i need to think a little bit on this
2018:01:22 17:25:59 carkh thansk for your time the both of you
2018:01:22 17:26:31 ghadi no problem. Lots of other validation libraries provide "mutative validation".... conflating transformation into its API
2018:01:22 17:26:36 ghadi really impedes reuse
2018:01:22 17:27:03 ghadi Typical example is 'validate this string into a UUID'
2018:01:22 17:27:17 carkh that's a good example
2018:01:22 17:28:28 carkh but now i first need to conform a big data structure, then walk it and validate parts of it
2018:01:22 17:28:41 carkh that's a lot more code ><
2018:01:22 17:28:57 carkh anyways i'll think on it, thanks again !
2018:01:22 20:58:35 golin I'm trying to spec polymorphic maps where a particular key is used for dispatch. Is there a way to spec the dispatch key to specify the dispatch value?
2018:01:22 21:28:37 golin Thanks, I'll look into it. That retag argument looks interesting.
2018:01:23 12:01:09 stathissideris here’s a tricky one: I have a data structure and I use conform with a regex spec to extract information from it. If the data structure is invalid in two places, I only get the first problem reported, because the regex stops. Is there any way to get the second problem too? (I suspect it’s a no)
2018:01:23 13:04:31 alexmiller No
2018:01:23 13:05:03 alexmiller To get a deeper level for something like this you would really need to write your own custom spec
2018:01:23 13:43:07 stathissideris @alexmiller ok, makes sense, thanks. I’ll see if I can circumvent the problem somehow
2018:01:23 15:54:26 vikeri Is there any way of making sure that the specs I write point to other specs? I can write (s/def ::my-spec (s/keys [:not-a-spec/key]) And :not-a-spec/key doesn’t have to be a spec. Can I somehow enforce that it really is a spec that it’s pointing to?
2018:01:24 11:36:49 misha you might check if form is the same as in registry, and it will not blow up on ns reload for unchanged specs, but still will on the ones you are iterating over (developing)
2018:01:24 11:43:15 vikeri @misha I only want to deny overrides when I run the tests to make sure I haven’t duplicated it in the code. (Loading some specs from an external lib). Yeah I was considering writing my own macro but then I’d like to override the macro so that normally it uses the normal s/def but when I run my tests it throws the error. I’ll probably go with my own macro actually.
2018:01:24 11:45:01 misha sounds like too custom use case for an out of the box solution
2018:01:24 15:53:58 pablore Is there a way to use s/assert and s/explain when the assert fails?
2018:01:24 16:23:49 pablore is it still good practice to use {:pre [(s/assert …)]} on functions?
2018:01:24 16:24:06 pablore for some reason assert is failing but explain is giving me success
2018:01:24 16:27:12 schmee @pablore just keep in mind that asserts can be toggled off
2018:01:24 16:27:39 pablore Yes but thats not the idea, I need it to fail when I give it a bad input
2018:01:24 16:28:21 schmee here’s a thing I’ve found useful many times:
2018:01:24 16:29:02 ghadi there's another (similar) idea there
2018:01:24 21:05:46 mrchance Hi, I want to generate test data based on a swagger spec, but subject to some constraints. Do you think spec is a good tool for that? If yes, how would one best go about that?
If not, what would be a better way to do it?
2018:01:24 22:46:04 kingmob Quick sanity check: in Clojurescript, is cljs.spec.test.alpha working for anyone else? I keep getting compilation errors about clojure.test.check.
2018:01:24 22:47:14 kingmob (Since it’s still alpha, I wasn’t sure)
2018:01:24 23:04:20 dadair In some cases you need to explicitly import [org.clojure/test.check <ver>], at least on the clj side
2018:01:25 08:17:12 mrchance @ikitommi Thanks, but I don't want to generate swagger, but the data consumed by the interfaces specified in a swagger file. It doesn't look like that is possible with swagger-spec?
2018:01:25 09:38:58 ikitommi oh, the endpoint data. With vanilla swagger spec, there needs to be a JSON-Schema -> Spec converter, I don’t know if such exists yet. But if you describe your endpoints with spec, you get the data generation for free.
2018:01:25 10:09:51 mrchance Yes, already doing that, unfortunately some of the other services in my project aren't written in Clojure. If one were to write such a converter, how would one go about it? I invested an afternoon because I didn't really find anything too, but it seemed really hard and messy to generate specs at runtime, due to its reliance on global mutable state 😞
2018:01:25 10:12:43 mrchance @ikitommi I saw that you had schema like maps in your spec tools, which could be handled like normal data. Would that be an option for this case?
2018:01:25 10:23:29 ikitommi most/all of the core specs have functional versions already in the core, but currently not documented and the apis are bit hairy. Maybe those will be polished and published as part of the public api? The forms need to be created manually but for most core predicates, it’s easy to resolve those (and spec-tools does that already)
2018:01:25 12:49:35 mrchance Hm, I see. Thanks, I'll give it a go, and hope for the release of the functional api in core.
2018:01:25 15:43:49 mpenet is there something like s/cat but that doesn't do any labeling? need to match a subsequence within a sequence without too much ceremony
2018:01:25 15:44:53 mpenet it's a seq of chars (just playing), so nothing generic, I need equality matching (ex match (b a r) in (f o o b a r b a z))
2018:01:25 15:45:16 mpenet I know I can mix s/cat and sets of chars or predicates, but that's quite ugly
2018:01:25 15:46:19 mpenet I am also forced to use regex context (my question is part of a wider parsing thing)
2018:01:25 15:46:48 alexmiller if you don’t want labeling, just use s/valid?
2018:01:25 15:47:33 mpenet I dont need labeling per char, I do for the whole subseq (up one level)
2018:01:25 15:47:56 mpenet maybe I didn't understand what you mean
2018:01:25 15:49:05 mpenet all the surounding value have a meaning that I need to retain also, I can't just check "does this seq contains this sub-seq"
2018:01:25 16:01:52 alexmiller you can use s/& to apply a custom predicate to a regex spec
2018:01:25 16:02:29 alexmiller which could be helpful here
2018:01:25 16:04:04 mpenet right, forgot about &
2018:01:25 16:08:16 mpenet that's brilliant, I was messing with conformers but I think & might just do all this for me
2018:01:25 21:26:08 schmee is it just me or do tools.namespace and instrument not get along?
2018:01:25 22:05:11 schmee or does it not check things that are defed?
2018:01:25 22:05:53 schmee that is, if I reload a namespace that has (def foo (some-instrumented-fn invalid-value)), does reloading the ns throw an exception?
2018:01:25 22:08:10 alexmiller Reloading is going to remove your instrumented vars
2018:01:25 22:11:12 schmee ok, so those checks should be done through tests instead?
2018:01:25 22:35:27 alexmiller what checks?
2018:01:25 22:39:55 schmee let me change the question: what’s the best way to check specs with clojure.test?
2018:01:25 22:40:13 alexmiller that seems pretty far from the original questions :)
2018:01:25 22:40:13 schmee (is (s/valid? ::my-spec (some-fn)) works, but it doesn’t give helpful output
2018:01:25 22:40:40 schmee I’m not very familiar with either clojure.test or spec, so I’m fumbling around a bit 🙂
2018:01:25 22:40:45 alexmiller there used to be a pin here to an example, but maybe it has aged out
2018:01:25 22:41:45 alexmiller by “check specs”, I’m assuming you mean to run clojure.spec.test.alpha/check on one or more spec’ed functions
2018:01:25 22:42:41 schmee actually, I want to check with s/valid?, but use s/explain instead of the usual error message provided by is
2018:01:25 22:43:54 alexmiller so s/valid? is not going to be as useful in checking spec’ed functions as is calling check
2018:01:25 22:44:19 alexmiller unless ::my-spec here refers to a data spec
2018:01:25 22:44:28 alexmiller I can’t really tell
2018:01:25 22:45:16 schmee ::my-spec in this case is a data spec, yes
2018:01:25 22:45:30 alexmiller ok
2018:01:25 22:45:35 schmee sorry if my questions are a bit confused, it’s hard to ask good questions when you’re not familiar with something
2018:01:25 22:45:43 alexmiller yep, no worries
2018:01:25 22:46:09 alexmiller I am similarly trying to re-interpret your questions :)
2018:01:25 22:46:38 alexmiller so you want to check if some data matches a spec and if not, return a message that is s/explain
2018:01:25 22:46:43 alexmiller one way would be to:
2018:01:25 22:47:35 alexmiller
(is (nil? (s/explain ::spec data)))
2018:01:25 22:48:33 alexmiller although I guess you want the explain result as a value not as a print
2018:01:25 22:49:30 alexmiller (is (not= "Success!\n" (s/explain ::spec data))) would do that I think
2018:01:25 22:49:38 alexmiller that’s quite ugly of course :)
2018:01:25 22:49:50 schmee explain-data did the trick!
2018:01:25 22:51:22 alexmiller yeah, can do same idea with explain-data
2018:01:25 22:51:50 alexmiller you can supply a final custom error message to is too
2018:01:26 14:56:26 alexmiller I used back-tick there - syntax quote will resolve symbols so that’s an easy way, but I also could have done (s/get-spec 'user/foo)
2018:01:26 14:56:55 alexmiller note that what you get back is a function spec instance, but that spec supports keyword lookup of :args, :ret, and :fn
2018:01:26 14:57:22 alexmiller or you can use s/form on it directly
2018:01:26 14:57:30 triss ah I see now. wonderful thank you Alex!
2018:01:26 15:52:35 don.dwoske Rich mentioned in a podcast (I think) that there was some sugar forthcoming which allowed a map to specify/lift the namespace for all the keys in the map (if they all had the same namespace) to the map level such that each individual key in the map could be printed / declared as un namespaced for cleanliness? Is there news on this, or a preview of what it might look like? I'm working on something similar and an example would be great to see. This was mentioned in the context of using spec, so that's why I'm putting it here vs. another channel.
2018:01:26 15:55:07 ghadi It's syntax sugar -- already in 1.9
2018:01:26 17:00:08 triss ok, so if I have a function `f’ thats been passed in to another function and I don’t know it’s fully qualified name is there a way to find it?
2018:01:26 17:00:54 bronsa no
2018:01:26 17:01:07 bronsa if by function you mean var then yes
2018:01:26 17:02:01 triss ah I do mean var I think - how do I look up a fully qualified name/spec for a function referenced by a variable?
2018:01:26 17:02:47 triss oh hang on `var?' just gave me a false...
2018:01:26 17:09:37 bronsa maybe if you give a code example of what you’re trying to do it might be clearer
2018:01:26 17:11:00 triss I’m trying to look up if a particular function has had a spec written for it.
2018:01:26 17:11:23 triss i.e does a function have a spec specified for it.
2018:01:26 17:11:45 ghadi if you have a fully-qualified symbol, you can do this
2018:01:26 17:12:17 ghadi if you have a function, you cannot
2018:01:26 17:12:17 triss I won’t have the fully qualified symbol when I come to do it - does this mean it’s a no go?
2018:01:26 17:12:43 ghadi "So what are you actually trying to do?"
2018:01:26 17:12:46 triss ah ok - this is a shame - The REPL seems to be able to look up the fully-qualified symbol for function
2018:01:26 17:13:46 triss @gihadi - it is a bit convoluted - I’m experimenting with agent based computing - and turning functions in to agents using only the information in there type signatures
2018:01:26 17:14:39 triss (well there spec’d signatures)
2018:01:26 17:16:02 ghadi can you turn on instrumentation prior a priori?
2018:01:26 17:18:19 triss not what I want unfortuantely - I’m doing some crazy stuff with the spec - I unpack info about each argument and tehn search a collection for it.
2018:01:26 17:19:07 triss the only thing I can’t do is look up wether or a not a function has had a spec written for it (when I don’t have its fully wualified name)
2018:01:26 17:26:05 alexmiller If you don’t have the qualifier, then it’s ambiguous what you’re referring to
2018:01:26 17:26:42 alexmiller A function of the same name could be spec ed in many nses
2018:01:26 17:27:16 alexmiller That said, the registry is just a map with either qualified symbols or qualified keywords as keys - you could search for it
2018:01:26 17:32:19 triss ok - thanks Alex. So is there a way to look up a functions source namespace/name?
2018:01:26 17:33:08 alexmiller Given what info?
2018:01:26 17:33:18 triss just the function…
2018:01:26 17:33:29 triss the REPL seems to know hwere it came from
2018:01:26 17:33:45 alexmiller Example?
2018:01:26 17:34:26 alexmiller A function instance doesn’t hold an explicit name for its source var
2018:01:26 17:35:02 triss
bb-simple.core> (defn function [a] a)
#'bb-simple.core/function
bb-simple.core> (def z function)
#'bb-simple.core/z
bb-simple.core> z
#function[bb-simple.core/function]
bb-simple.core>
2018:01:26 17:35:25 triss so I can store a function in z - the REPL tells me where I defined it
2018:01:26 17:38:13 triss when all I have is z is there a way to find that it refers to bb-simple.core/function?
2018:01:26 17:38:47 bronsa not reliably
2018:01:26 17:39:01 alexmiller that’s not the default printer - you must be running with something addition in your repl
2018:01:26 17:39:40 triss ah - it’s the CIDER repl in emacs
2018:01:26 17:39:41 alexmiller the function class name is a munged version of the original function name (assuming it was from a var, can be from other places too)
2018:01:26 17:40:01 bronsa what that’s doing is using demunge to get from the class name to an approximation of the original var
2018:01:26 17:40:14 bronsa but that is not relibale and can only be done if the function was compiled from a defn
2018:01:26 17:40:22 alexmiller you can demunge to recover the class name, but the munge operation is not a true function (mathematically) in that multiple inputs map to the same class name
2018:01:26 17:40:34 alexmiller that’s where the unreliability comes from
2018:01:26 17:41:13 triss oh ok this all sounds a bit yuck. Think I’ll steer clear for now. It’s a lot of extra typing for me but such islife
2018:01:29 17:09:56 conan what is it you want to generate exactly?
2018:01:29 17:13:21 conan are you trying to generate a 2-value vector containing ints that can be used with apply as the args for sbtrct?
2018:01:29 17:14:16 conan generators generate values, so by providing a spec for each of your input args and for your return value, you've already provided generators (because spec provides a built-in generator for the int? predicate)
2018:01:29 17:19:31 roman01la @U053032QC I want to generate a sequence of args/ret values such that they satisfy relation defined in :fn spec
2018:01:29 17:20:16 conan ah, ok:
(s/exercise-fn `sbtrct)
2018:01:29 17:21:12 conan that'll generate a bunch of input values for a and b (using the generators provided by their specs, i.e. int?), put them into your function, and return you a sequence of those input pairs with the calculated output
2018:01:29 17:21:32 conan if your function doesn't match your :fn spec, it'll blow up
2018:01:29 17:22:36 conan if you want to do a larger number of tests of your function, you can do generative testing using clojure.spec.test.alpha/check
2018:01:29 17:22:51 conan so
(stest/check `sbtrct`)
2018:01:29 17:23:22 conan that'll generate 1000 different inputs and check all the outputs for conformance with your :ret spec and :fn predicate
2018:01:29 17:23:24 conan neat, huh?
2018:01:29 17:23:56 roman01la > if you want to do a larger number of tests of your function..
that’s exactly what I need, exercise-fn doesn’t seem to be able to generate huge sequences 🙂
2018:01:29 17:24:27 conan you can do
(s/exercise-fn `sbtrct 100000)
2018:01:29 17:24:41 conan oh no, looks like that blows up
2018:01:29 17:29:12 conan that's specifying a range that the ints can be in, so you won't get huge numbers generated
2018:01:29 17:29:39 roman01la Should I use with-gen if I don’t want to modify the original spec?
2018:01:29 17:30:40 conan with-gen is for when you need to create your own generator because the existing ones can't capture your set of values
2018:01:29 17:31:04 conan i guess you'll have to look into what's actually causing the error
2018:01:29 17:32:28 conan that approach above works for me though, with s/int-in as the spec for the inputs to your function
2018:01:29 17:33:39 conan i used a different name for my function though so don't just copy/paste
2018:01:29 17:34:04 roman01la works for me as well, thanks!
2018:01:29 17:39:54 conan no worries, docs are often a bit sparse around spec, but it is alpha, so /shrug
2018:01:29 18:05:04 roman01la I ended up with (s/int-in Integer/MIN_VALUE Integer/MAX_VALUE)
2018:01:29 18:21:59 john Does spec.test in any way supersede test.check? I couldn't find docs explaining the difference. There seems to be some overlap between the two.
2018:01:29 18:39:23 arrdem Not that I know of. spec.test leverages test.check extensively and seems designed to complement test.check by enabling you to derive many generators automatically rather than specify them by hand.
2018:01:29 18:43:34 alexmiller correct. test.check is useful for writing arbitrary property-based tests and creating different kinds of custom generators
2018:01:29 18:44:40 bronsa and spec.test uses test.check internally
2018:01:29 19:08:03 john Understood. Thanks
2018:01:30 20:11:50 rafael I'm trying to pass :gen overrides for clojure.spec.test.alpha/check, but I get the sense my understanding of the documentation.
2018:01:30 20:12:08 rafael I expected the following to succeed:
2018:01:30 20:13:05 rafael I'm trying to pass in a generator for the my-kw? spec, but it fails with the cause "Unable to construct gen at: [:a-keyword] for: my-kw?"
2018:01:30 20:15:00 rafael Is overriding generators on the call to check possible and I'm getting some detail wrong, or did I just misunderstand the docs for check and there is no way to override generators like that?
2018:01:30 20:26:31 alexmiller the values in that map should be 0-arg functions returning a generator
2018:01:30 20:26:48 alexmiller so change (gen/return :kw) to #(gen/return :kw)
2018:01:30 20:28:17 alexmiller although now that I back up a bit, I don’t think you can do what you’re trying to do with my-kw? either
2018:01:30 20:28:43 alexmiller you should instead register a named spec instead of my-kw?:
2018:01:30 20:29:46 alexmiller (s/def ::my-kw keyword?) and then use ::my-kw instead of my-kw? and as the key in the gen map
2018:01:30 20:35:31 alexmiller Putting it all together:
2018:01:31 18:43:15 rafael That works. I had tried the 0-arg generator function, but I was missing the fact that the spec ident had to be a keyword. Thank you!
2018:01:30 22:06:34 jqmtor hey all, I've been meddling with spec and I came across something I don't know how to represent. I want to create a spec for the s/keys spec itself and I have this:
the problem is that the keys :req:opt and so on can be repeated. is there any way to avoid this? (any other unrelated suggestion is also welcome)
2018:01:30 22:15:28 alexmiller keys*
2018:01:30 22:23:16 jqmtor great, thanks! missed that when looking through the api
2018:01:30 22:26:47 eriktjacobsen I'm running into the same wall described at https://groups.google.com/d/msg/clojure/i8Rz-AnCoa8/OEg04fbKBwAJ where we'd like to resolve a symbol for a multi-fn in clojurescript but apparently cannot. is this being looked into? that thread died.
2018:01:31 18:31:07 uwo simple enough idea, but I did get it from you 🙂
2018:01:31 20:41:38 drewverlee Has anyone seen any attempts to turn a postgres schema into a set of clojure specs so you can generate data like whats in the database? Or vise versa, fill the database with data?
In the case where your using postgres or a relational db, would that be helpful?
2018:01:31 21:50:03 eggsyntax @drewverlee at one point I did the same thing with a Datomic schema. It was probably simpler because of the natural Datomic/Clojure match, but I'd definitely think you could do the same with a relational DB. You might want to think about what you'd want the specs to capture -- type is the obvious low-hanging fruit, but there might be other stuff. And you could probably capture the set of fields present in each table with s/keys. Possibly other stuff depending on your particular schema, but in my experience worth considering well in advance of taking on the conversion.
2018:01:31 21:53:04 drewverlee @eggsyntax i’m thinking either this sort of thing already exists (maybe in another language) or there are really good reasons why its not useful. But maybe clojure spec opens some new doors.
2018:02:02 16:55:04 stathissideris it doesn’t take care of traversing the schema, but you can run your data through this to get specs out of them: https://github.com/stathissideris/spec-provider
2018:02:02 16:55:20 stathissideris (my project btw, so I’m here for questions)
2018:01:31 22:21:45 alexmiller I’m pretty sure someone has probably done this already for jdbc based on your databasemetadata, and maybe that’s 90% of the work
2018:01:31 22:23:26 alexmiller don’t think I’ve seen it specifically with spec though
2018:02:01 08:23:49 andy.fingerhut "A function provides its return. If you gave me what I required, I will provide to you this result. And of course I would like to broaden this discussion to include services and procedures, and things like that. So if your thing is effectful, one of the things you provide is that effect. If you call this thing with these arguments, the thing will be in the database, or I will send an email for you, or some other thing."
2018:02:01 08:24:13 andy.fingerhut Does anyone have any ideas on how such a thing might be described in a spec-like fashion?
2018:02:01 11:21:08 mattford I'd like to over-ride/extend the body part of the above spec to use spec's I've written that model the JSON body I get.
2018:02:01 11:21:18 mattford Is that a thing?
2018:02:01 11:22:02 mattford How'd people go about that?
2018:02:01 11:55:57 borkdude I’d write a different spec for the parsed body.
2018:02:01 12:23:55 misha @bbrinck quick expound question: is there a function, accepting vanilla spec/explain-data value and returns pretty error message as string?
2018:02:01 15:15:18 bbrinck @misha Yep, that should work. I’ll add that to my notes about what to add to public API for the expound beta
2018:02:01 19:01:57 nwjsmith I'm having trouble getting a useful spec together for this map-entries function I've written:
(defn map-entries
"Returns a map consisting of the result of applying f to the first entry of
the map, followed by applying f to the second entry in the map, until the map
is exhausted."
[f m]
(into {} (map f) m))
(s/fdef map-entries
:args (s/cat :f (s/fspec :args (s/cat :entry (s/tuple any? any?)) :ret (s/tuple any? any?))
:m (s/map-of any? any?))
:ret (s/map-of any? any?)
:fn #(<= (count (-> % :ret)) (count (-> % :args :m))))
2018:02:01 19:03:23 nwjsmith If I turn instrumentation on, and evaluate say (map/map-entries (fn [[k v]] [(name k) (dec v)]) {:a 1 :b 2 :c 3}), I get an instrumentation exception because spec tries to run (fn [[k v]] [(name k) (dec v)]) with a generated (s/tuple any? any?).
2018:02:01 19:06:15 nwjsmith If I loosen the spec on the :f arg to just fn?, then instrumentation is okay, but I'll lose a the "free" property tests. I guess I can just supply my own generator for check...
2018:02:01 19:08:10 misha what would you gain, if you mock your f?
2018:02:01 19:11:00 nwjsmith Well if I had the following spec:
can I have spec conform the strings to keywords and deal with the overloaded "hits" key somehow?
2018:02:01 19:15:04 misha I think, strings to keywords conversion needs to happen separately. but later, you'll be able to use 2 :keys keyword specs with different namespaces
2018:02:01 19:17:25 nwjsmith Maybe I can spec the args differently here. What I'd like is the spec to be is
2018:02:01 19:21:36 misha @nwjsmith try to drop :ret (s/tuple any? any?) for :f
2018:02:01 19:22:03 mattford great ty!
2018:02:01 19:22:19 misha into's spec might catch not-tuples from f
2018:02:01 19:23:53 misha it blows up opieop
2018:02:01 19:28:50 nwjsmith
(defn map-entries
"Returns a map consisting of the result of applying f to the first entry of
the map, followed by applying f to the second entry in the map, until the map
is exhausted."
[f m]
(into {} (map f) m))
(s/fdef map-entries
:args (s/cat :f (s/fspec :args (s/cat :entry (s/tuple any? any?)))
:m (s/map-of any? any?))
:ret (s/map-of any? any?)
:fn #(<= (count (-> % :ret)) (count (-> % :args :m))))
(stest/instrument)
(map-entries (fn [[k v]] [(name k) (dec v)]) {:a 1 :b 2 :c 3})
ExceptionInfo Call to #'user/map-entries did not conform to spec:
In: [0] val: ([nil nil]) fails at: [:args :f] predicate: (apply fn)
clojure.core/ex-info (core.clj:4617)
2018:02:01 19:30:38 nwjsmith How does instrumentation of function arguments work? Does the function get evaluated?
2018:02:01 20:26:41 alexmiller the args spec for the function argument is used to generate inputs. the function argument function is invoked with those examples and the ret spec is validated
2018:02:01 20:29:11 alexmiller also, you should pretty much never use fn? (if you go that route) - ifn? is almost always what you want
2018:02:01 20:30:21 alexmiller generic higher-order functions are inherently difficult to spec in useful ways. specs are great for saying concrete things about your data. The more generic your function, the harder it is to say something specific and meaningful.
2018:02:01 21:59:10 ghadi not sure if this is a bug, but I could use a second set of eyes:
2018:02:01 22:02:21 ghadi I'm asking spec to stub out the function TARGET while overriding the generator for ::bar, which is aliased to ::foo. The gen overriding doesn't work -- if I override ::foo (which ::bar aliases) then it works. Probably a non-minimal example if someone can help me reduce it
2018:02:02 00:07:08 alexmiller I think there is a known issue with swapping gens for aliased specs
2018:02:02 00:07:42 alexmiller On the phone so can’t easily search JIRA
2018:02:02 17:20:28 stathissideris and with aliases it can be shorter
2018:02:02 17:21:05 dominicm @stathissideris But I want to return a namespaced map 😄
2018:02:02 17:21:28 stathissideris cat is for sequences!
2018:02:02 17:21:30 stathissideris not maps
2018:02:02 17:21:55 stathissideris use s/keys instead
2018:02:02 17:22:37 dominicm I have a sequence of things, and I want to conveniently conform it into a namespaced map. I'm being lazy.
2018:02:02 17:23:25 stathissideris oh!
2018:02:02 17:23:29 stathissideris ok makes sense
2018:02:02 17:24:50 dominicm I guess the manual way I go (hi ho!)
2018:02:02 17:31:39 stathissideris would be a pretty trivial macro…
2018:02:02 19:42:49 misha macro all the things!
2018:02:02 21:20:12 gklijs Don't need a macro for that, it's pretty easy with some functions to convert data based on a spec to/from a vector.
2018:02:03 09:51:14 misha @alexmiller is there any estimate on where will this be addressed? It makes generating even slightly complex data structures noticeably painful. https://dev.clojure.org/jira/browse/CLJ-2079 (Generator overrides for spec aliases are not respected)
2018:02:03 12:34:12 benalbrecht @misha @ghadi you could resolve the spec manually before passing it to the generator map:
2018:02:03 13:50:32 misha do you mean "find all aliases manually and explicitly override those"?
2018:02:03 13:50:46 misha that should work, yes
2018:02:04 01:03:21 richiardiandrea Is there a way to not report errors for an argument in a s/cat or spec only the second position of it? The first argument in my case is a JS connection object and I am not interested in seeing it when it does not validate because very verbose
2018:02:04 01:04:01 seancorfield @richiardiandrea If you don't care about it validating, why not spec it as any??
2018:02:04 01:04:37 richiardiandrea Yes that is what I am doing, but it is included as part of the error message
2018:02:04 01:05:10 seancorfield Oh, I see. So you want a custom error reporter instead? Have you looked at Expound?
2018:02:04 01:05:55 richiardiandrea Oh right, yes I am aware of expound. Probably I can have a custom reporter if not too tough too implement
2018:02:04 01:10:15 bbrinck See the example under “You can even provide your own function to display the invalid value.”
2018:02:04 01:13:27 bbrinck Although by default, parts of your data that is valid won’t be shown anyway with Expound
2018:02:04 01:27:24 bbrinck @richiardiandrea Here’s an example that includes a custom printer that replaces the first arg (presumably one that is verbose) with a more succinct representation https://gist.github.com/bhb/5222914641bcdd08d07b7d930e388d89
2018:02:04 04:38:16 richiardiandrea That is great thaaaanks, I am actually using expound
2018:02:04 05:34:45 bbrinck np
2018:02:04 14:08:36 alex-dixon How do you write a spec for a map that has a namespaced keyword like :db/id?
2018:02:04 14:21:41 taylor use a s/keys spec, and use its :req or :opt to specify the individual key specs
2018:02:04 15:01:31 alex-dixon Awesome. Thanks. Thought I had to use ::db/id for some reason
2018:02:04 22:45:10 bbrinck Coming soon in expound: optional human-readable error messages for predicates https://asciinema.org/a/161011
2018:02:04 23:26:17 stathissideris @bbrinck that’s really good, but would it be possible to add an error message to an existing spec? I’m thinking about the case where you’re trying to do this to a third-party spec whose code you don’t control
2018:02:04 23:26:44 stathissideris it may be too much to ask, but I think it would be better to keep the libs off each other’s toes
2018:02:04 23:28:20 bbrinck It’s an interesting idea … certainly you could add a an error message for a spec directly (I can add a function for this).
2018:02:04 23:29:00 bbrinck If you used expound/def on an existing function, you’d probably need to overwrite the actual spec though, so as not to break the way s/def works
2018:02:04 23:29:40 bbrinck in other words, calling expound/def would overwrite existing spec, just like s/def
2018:02:04 23:30:18 bbrinck Another thing to watch out for is that this won’t work unless the original spec author defined a spec for the predicate specifically
2018:02:04 23:39:40 stathissideris @bbrinck yeah, I would expect the parameter in that function to be the name of a spec, not a predicate
2018:02:04 23:40:49 stathissideris I was looking at spec tools recently, and although I liked the functionality, I was a bit sceptical about that fact that I would have to write specs differently (using the “spec record” I think)
2018:02:04 23:41:06 stathissideris so that’s what struck me about your approach as well
2018:02:04 23:41:15 bbrinck Yep, I could add something like (expound/add-message <qualified-keyword> <string>)
2018:02:04 23:42:00 stathissideris I’d be very happy with that 🙂 does that get stored in a separate registry for expound?
2018:02:04 23:42:07 bbrinck Yes, that’s correct
2018:02:04 23:42:46 bbrinck @stathissideris Yes, needing to write specs differently is a fair concern here. I don’t think there’s a way to bolt this functionality onto existing specs, and even if we could, I’m not sure I’d want to
2018:02:04 23:43:11 bbrinck Unless clojure.spec supported this directly, of course … 🙂
2018:02:04 23:43:34 stathissideris there’s been talk about spec metadata for quite a while now
2018:02:04 23:43:37 bbrinck But for now, you can write your specs in either style
2018:02:04 23:43:43 stathissideris …but no official solution yet
2018:02:04 23:44:05 bbrinck and if the consumer is using spec, it works. if they are using expound, they will get the enhanced error messages
2018:02:04 23:44:33 stathissideris yeah, I think it all composes a bit more nicely like that!
2018:02:04 23:45:45 bbrinck (for the record, I’d be happy to see something compatible go into spec, and if it did, i’d use it in expound)
2018:02:04 23:45:57 bbrinck oh wait, I actually implemented this function - it’s register-message
2018:02:04 23:46:12 bbrinck Man, I’ve already forgotten what I wrote yesterday 🙂
2018:02:04 23:46:17 bbrinck I will add it to documentation
2018:02:04 23:46:21 stathissideris looking forward to the new functionality, I have a direct use for it when you release it, so thanks for your work!
2018:02:04 23:46:49 stathissideris haha, that’s great, that’s the style I’m going to use
2018:02:04 23:46:59 bbrinck Cool, I hope to release early this week
2018:02:04 23:47:02 bbrinck until then, it’s in 0.4.1-SNAPSHOT
2018:02:04 23:47:12 bbrinck but, use at your own risk 🙂
2018:02:04 23:47:42 stathissideris not in a hurry, but looking for ways to make the errors in my UI a bit more user-friendly
2018:02:07 04:07:23 bbrinck I’ve released Expound 0.5.0, which includes this feature
2018:02:07 07:55:52 stathissideris great, thank you very much, I’m going to give it a go in the next few days
2018:02:09 13:58:09 stathissideris so, no register-message in the end?
2018:02:05 09:57:07 dominicm Is there a s/cat function which doesn't create a map? I essentially want a regex op which will consume exactly 2 predicates.
2018:02:05 13:15:04 alexmiller Use s/tuple
2018:02:05 13:17:15 dominicm @U064X3EF3 That isn't a regex op, so it doesn't work for this:
2018:02:05 13:24:17 dominicm I didn't really understand s/& when I read the docstring.
2018:02:05 13:49:20 alexmiller it’s like s/and but as a regex op
2018:02:05 14:21:09 dominicm That's a good description. I prefer that
2018:02:05 09:57:36 dominicm Maybe my mistake is using something positional where I should be using a map
2018:02:05 10:23:52 mattford do you have to conform it?
2018:02:05 10:25:50 mattford I've instrumented and stubbed out a function that thanks to the magic of spec can now generate results. The result is generated from a multispec though, is there a way of limiting a multispec to one of the methods [so I can only return certain types of results]?
2018:02:05 10:53:51 acron @mattford add a predicate check for the dispatch type
2018:02:05 10:54:45 acron
(s/cat :foo (s/and ::my/thing is-correct-thing?))
2018:02:05 10:56:03 acron where ::my/thing is your multispec
2018:02:05 10:56:27 acron this has worked relatively successfully for me
2018:02:05 10:57:21 acron What would cause clojure.spec.test.alpha/spec-checking-fn/conform! to be applied to a function that has *not* been instrumented? 🤔
2018:02:05 10:59:45 mattford Feels a bit circular having a mutlispec generate many cases only then to filter it again on the same logic each defmethod has.
2018:02:05 11:01:08 mattford works though...
2018:02:05 11:02:15 acron Well, the real question could be why do we want just one result type generating?
2018:02:05 11:05:28 mattford I instrumented my general get-event handler but I'm only interested in testing "visualisation" events in the next part of the tests..
2018:02:05 16:06:32 tcoupland @mattford check out the :gen option of instrument
2018:02:06 04:01:30 zalky I'm assuming this is a bug?
2018:02:06 05:20:36 alexmiller I would expect it to fail in both, surprised it doesn’t in clojure
2018:02:06 07:51:35 bmaddy Hmm, what am I missing here?
cljs.user> (require '[cljs.spec.alpha :as s])
nil
cljs.user> (require '[cljs.spec.gen.alpha :as gen])
nil
cljs.user> (gen/generate (s/gen int?))
#object[Error Error: Var clojure.test.check.generators/simple-type-printable does not exist, clojure.test.check.generators never required]
Is there somewhere I should be looking for an example of how to do this in cljs? I couldn't find anything on http://clojurescript.org or in the official spec guide.
2018:02:06 08:00:52 bmaddy Oh, and here's part of my :dependencies:
2018:02:06 09:08:08 bronsa it's not throwing in clojure because asserts are disabled by default
2018:02:06 09:08:46 bronsa >>>Can be disabled at either compile time or runtime:
If \compile-asserts\ is false at compile time, compiles to x. Defaults
to value of 'clojure.spec.compile-asserts' system property, or true if
not set.
If (check-asserts?) is false at runtime, always returns x. Defaults to
value of 'clojure.spec.check-asserts' system property, or false if not
set. You can toggle check-asserts? with (check-asserts bool)
2018:02:06 15:04:23 zalky @alexmiller: are you sure? In my example, the ::t spec, (s/keys :req [:ns/y])) does not specify anything about the :ns/x key. But it seems to be applying it anyways, simply by virtue of it being defined globally. This would seem to defy the point of :req. Maybe I'm misunderstanding how keys was meant to be used, but I would have thought the clojure version to be correct.
2018:02:06 15:05:05 alexmiller s/keys is intended to check all registered keys in the map, regardless of whether they are listed in the spec
2018:02:06 15:09:32 zalky ah, thanks for clarifying, but then what is the point of :req? Shouldn't you only need :opt?
2018:02:06 15:12:30 alexmiller req defines the keys that are required (must be present)
2018:02:06 15:12:39 alexmiller req is about membership
2018:02:06 15:13:04 alexmiller so it’s really checking a property of the containing map
2018:02:06 15:13:16 zalky hmm, but if all keys are just going to be globally applied whether they are in the spec or not, wouldn't that mean all keys a implicit req, unless specified by :opt?
2018:02:06 15:13:20 alexmiller validating values according to keys is about checking properties of attributes
2018:02:06 15:13:48 alexmiller no, it does not mean that
2018:02:06 15:13:58 alexmiller they’re not required, but if they are present, they are checked
2018:02:06 15:29:08 zalky Hmm, ok, thanks for the clarification. Is the design assumption that entity attributes should conform the same properties in all contexts? You should never have an entity attribute conform one way in one context, and another way in another context?
2018:02:06 18:08:37 zalky @alexmiller: thanks for the responses, the doc was very helpful. My use case was that I was playing with a simple FSM that brought an entity map through one or more workflows. At first spec seemed to lend itself well to defining the transitions in the FSM. For example, an entity with any number of votes is a valid entity, but once it gets a certain number of votes, it transitions to another state in the FSM. Unfortunately having one valid global definition for an attribute seems to place significant restrictions on defining multiple validity states for an entity. Am I missing an easier way to do this with spec, or is it just that spec is not the right tool for this job?
2018:02:06 18:40:01 alexmiller I find it’s always useful to go back to: what is the truth about this attribute? if I see it in data, what does it mean?
2018:02:06 18:40:36 alexmiller if it has several possible kinds of values, you can use s/or to specify those alternatives - then you’re stating the truth
2018:02:06 18:41:03 alexmiller s/multi-spec can be used to choose different combinations of attributes at different points in the fsm
2018:02:06 18:41:23 alexmiller or perhaps you may want to reconsider whether it’s really the same attribute at every step
2018:02:06 18:41:30 alexmiller and in that case, maybe the attribute should change
2018:02:06 18:45:45 alexmiller to be clearer, maybe there is really more than one kind of attribute
2018:02:06 18:46:06 alexmiller or another option is don’t spec that attribute - you’re expecting it to change over time
2018:02:06 18:46:20 alexmiller you can still spec functions that operate on that attribute more precisely
2018:02:06 19:22:10 zalky Yeah, I was thinking the same thing about the attribute, there are probably times where it indicates that it is really two attributes. However this isn't always the case. The simple example I gave above with the voting is probably a case where splitting would unnecessarily complicating the data model because of an assumptions in the implementation.
I think you're suggestion to not spec the attribute via the entity model, and just spec it separately using custom primitives (at least that is how interpreted your last sentence) is probably the one that makes the most sense . Unfortunately you lose some of the great higher order features of the entity model, like being able to define one transition as a merge of previous transitions and some new criteria.
One final question: do you think that allowing local bindings in keys would break spec's entity model significantly? Something like:
2018:02:06 21:12:22 zalky Thanks alex for you time (and your contributions on spec!)
2018:02:07 18:55:18 Joe Lane If the above snippet is OT and belongs somewhere else, I’ll gladly take it there instead.
2018:02:07 20:36:30 bbrinck @lanejo01 Would it work to just remove the namespace from the keys in args? e.g. (into {} (map (fn [[k v]] [(keyword (name k)) v])) {:foo/bar 1 :foo/baz 2}) ;; => {:bar 1, :baz 2}
2018:02:07 20:37:10 bbrinck If you didn’t want to remove the namespace from all keys, you could pass in a specific prefix to remove in this case, :myapp.entity.user
2018:02:07 20:41:02 Joe Lane Interesting, yeah I could do that to just keep the unqualified keys. Im wondering if i’m missing some design step though since from the jira ticket it was concluded that there wasn’t planned work for namespaced keys in records.
2018:02:07 21:33:28 alexmiller I didn’t read everything above, but have you tried using s/keys with :req-un to spec records
2018:02:07 22:22:33 Joe Lane Ahh, I think thats the answer @alexmiller. I had read through that section of the spec guide probably 15 times last night but it didn’t click. This also pushes me away from desiring namespaced keywords in records. Thank you!
2018:02:07 22:38:05 seancorfield @alexmiller On that subject... given the general advice re: maps / records is usually "use maps, and only switch to records if you need extra performance", how does that sit with using namespaced maps -- since those cannot easily be switched to records?
2018:02:07 22:38:20 alexmiller it complicates it :)
2018:02:07 22:38:37 seancorfield 😆 but not helpful!
2018:02:07 22:39:03 alexmiller I mean if you have maps with namespaced keys, you can switch to records and change specs from :req to :req-un
2018:02:07 22:39:03 seancorfield Fair enough. The spread of namespaced maps does seem to reduce the attraction of using records further.
2018:02:07 22:39:38 alexmiller unfortunately, I agree. I seem to be in the minority, but I like records. :)
2018:02:07 22:39:47 seancorfield @alexmiller Yeah, but all of your code that manipulates those maps would need to change too -- since it will be using :foo/bar now and would need to change to :bar.
2018:02:07 22:39:52 alexmiller true
2018:02:07 22:40:18 alexmiller I’ve had a few conversations with RH about this
2018:02:07 22:40:32 seancorfield I'll be interested to hear what he decides in the end 🙂
2018:02:07 22:40:52 alexmiller records actually do have a namespace context via the package of the defining record type
2018:02:07 22:41:05 alexmiller seems like there should be a good way to leverage that
2018:02:07 22:41:40 alexmiller like field a in record my.R could use spec :my/a if req’ed in a s/keys
2018:02:07 22:42:01 alexmiller but that doesn’t really address the access difference
2018:02:07 22:42:19 alexmiller would also need to allow lookup by ns’ed key in the record, which maybe would be interesting
2018:02:07 22:43:14 seancorfield Perhaps an option on defrecord to produce either plain keys or namespaced keys? I assume the / could be munged to something Java would accept?
2018:02:07 22:43:33 alexmiller meh
2018:02:07 22:43:45 seancorfield Hahaha 🙂 I suspected that might be your response!
2018:02:07 22:43:54 alexmiller I think adding new access mechanisms is greatly preferable to adding options
2018:02:07 22:44:00 noisesmith it’s pretty ugly when munged
user=> (munge "x/y")
"x_SLASH_y"
2018:02:07 22:46:12 alexmiller optional behavior flags either means you’re being lazy or haven’t thought about it long enough :)
2018:02:07 22:48:13 alexmiller probably better to improve how a record spec would link fields to attribute specs in a way that supports more kinds of code evolution
2018:02:08 14:33:10 justinbarclay I’m learning about cljs.spec, but when I try to run the code below in a figwheel repl it times out. Does anyone know why or how I can improve my spec?
2018:02:08 16:28:15 mpenet Expected, merge doesn't flow conformed values. This one comes up quite often
2018:02:08 16:28:52 mpenet There s a (closed) jira issue about this
2018:02:09 21:11:16 justinlee so from the department of This Can’t Be Happening, I speced out a reagent component with fdef and instrumented it, and now it actually calls one of the callback-handlers that is passed to the component as part of the instrumentation process. is this expected/possible? or am i misunderstanding? in the code below, the on-cancel callback gets called 21 times if I leave the instrument call in the code
2018:02:09 21:14:34 alexmiller fspec args are validated by gen’ing from the :args spec and invoking the function, then checking the :ret spec
2018:02:09 21:15:11 alexmiller some people have found this to be surprising :)
2018:02:09 21:16:06 alexmiller probably the easiest “fix” is to replace (spec/fspec :args (spec/cat)) here with ifn?
2018:02:09 21:17:26 justinlee the good news is i’m not crazy 🙂
2018:02:09 21:18:29 justinlee sometimes i suspect i’m using spec improperly. i really just want an assertion library to catch my typos
2018:02:09 21:23:54 justinlee so is it ever possible/safe to use fspec with a side-effecting function? i don’t see how it could ever work unless you passed it a pure function
2018:02:09 21:24:56 alexmiller I would not advise it
2018:02:09 21:25:07 alexmiller passing a side-effecting function that is
2018:02:09 21:25:37 alexmiller there are ways to (for example) swap out an alternate spec when you instrument - those are optional capabilities of instrument
2018:02:09 21:26:26 alexmiller but I’m not sure you’re getting much value than bother out of having the fspec at that point
2018:02:09 21:26:33 misha TIL about stubbing in s/instrument
2018:02:09 21:26:50 alexmiller well I’m talking about replace, not stub, but yeah
2018:02:09 21:27:46 alexmiller stubbing is great for stubbing out a remote call in instrument
2018:02:09 21:28:22 misha if I'd commit fdef + instrument-with-stub instead of actual implementation, I wonder how quickly my teammates will notice in production? kappa
2018:02:09 21:32:22 alexmiller unless you wrote some really good generators
2018:02:09 21:32:56 misha number?troll
2018:02:10 18:16:02 arohner I’m having trouble generating from a recursive spec, I keep hitting StackOverflowError. Are there any docs/guides on how to do that?
2018:02:10 18:19:41 arohner I’ve also tried t.c.gen/recursive-gen
2018:02:10 18:25:11 taylor hard to say because the example is missing several definitions but you could try (binding [s/*recursion-limit* 1] ...)
2018:02:10 18:30:50 arohner That helped, in that gen/sample occasionally finishes, but still stackoverflows some of the time. It surprised me that in the first 10 items, it produces nested c’s 5 deep, which I wouldn’t expect with a recursion limit of 1
2018:02:10 18:34:00 taylor yeah I think there’s been a JIRA ticket w/some discussion about this, can’t find it at the moment
2018:02:12 01:29:38 alexmiller Yes, that last url is the correct one. Earlier, spec was in clojure and the prior one was correct
2018:02:12 01:55:06 Vincent Cantin beginner question: Can we use spec to validate the correlation between values at different places in a structure?
2018:02:12 01:56:30 taylor yes, are you thinking maps or another type of structure?
2018:02:12 01:56:41 Vincent Cantin For example, if I want to write a spec for [a b 1 5 ... 3 a b] where a and b could be anything.
2018:02:12 01:59:10 Vincent Cantin ... supposing that the structure is complex enough so that I would need to use s/conform to match the location of a and b at different places.
2018:02:12 02:00:19 Vincent Cantin I could use a classical function to check if they match at different places, but I would like to know if there is a better spec-ish way to do it. Something like declaratively defining the pattern.
2018:02:12 02:03:11 Vincent Cantin For example, something like: "For any a, b : [a b ... a b] is valid."
how do I check that the args I have conform to that? Thanks!
(I want to use this with expound. I have command functions invoked from a Slack bot and want to verify that the command the user typed has all the correct arguments and show a nice error if not)
2018:02:12 14:34:54 mpenet make a separate spec of the :args and use s/valid? on it ?
2018:02:12 14:35:27 mpenet you can then write (s/fdef myfn :args ::foo-args ... )
2018:02:12 14:55:59 mpenet wondering: why wasn't check-asserts based on a dynamic var like *compile-asserts*? Right now it's a bit all or northing at runtime for assertion checking (for the one that are compiled at least)
2018:02:12 14:57:30 mpenet it's easy enough to work around it anyway
2018:02:12 15:13:07 alexmiller all or nothing is kind of the typical modes for assertions
2018:02:12 15:14:17 alexmiller you can also grab the :args spec of a spec’ed function with
(-> `myfn s/get-spec :args)
2018:02:12 16:51:57 jumar I'm wondering if I can do any better if I want to have two different specs for the same key...
I have a generic auth-provider , e.g. #:auth-provider{:type "ldap" :config {:host "abc" ...}}
This has a generic spec :
which is fine for some generic application code.
However, in some scenarios I want to act differently based on the type of the auth-provider.
E.g. for "ldap" I know that certain keys have to be present inside :auth-provider/config.
I couldn't find a better solution than this:
(s/def ::ldap-provider-spec (s/and
::auth-specs/auth-provider-spec
;; this is less descriptive than using `s/keys` directly
;; but we cannot use `s/keys` because `:auth-provider/config` default
;; spec is already registered in auth-specs namespace
;; and you cannot have two different specs for the same namespaced key
#(s/valid? ::ldap-config (:auth-provider/config %))))
My problem is that since specs are global I cannot register two different specs for the same key (`auth-provider/config`).
I also tried multi-spec but couldn't solved it either.
My solution works but it's quite opaque - the error message just says that data fails ::ldap-config spec (not that e.g. :host key is missing in config map)
2018:02:12 18:04:58 misha look again at multi-spec, @jumar. it would be verbose solution, but sounds like it fits.
2018:02:12 18:27:20 jumar @U051HUZLD well, my problem was that I still couldn't define two different specs for :auth-provider/config key. Basically, based on the value of :type key I'd implement two multimethods - one for the :type value "ldap" and one for :default.
But in both of them I'd need to use something like (s/keys :req [:auth-provider/config]) - and here I'm stuck. I don't know how to define two different specs for :auth-provider/config key.
2018:02:12 18:12:37 misha although, it'd be much easier, if you'd put :auth-provider/type inside config itself
2018:02:12 18:46:46 misha if you put type inside config itself - multispecs will be about config keys, net provider's ones
2018:02:12 18:48:22 misha or you can just s/or the :provider/config, but it will not keep :provider/type and or's branches "in sync"
2018:02:12 21:42:45 roklenarcic how do you guys spec a map where keys are generic and values are constant for a type? To give you an example, clojure.xml will output something like {:tag :first_name, :attrs nil, :content ["John"]}. How would I spec that I want my ::first-name to have this structure? The keys function would require me to make a spec for ::tag but the value is constant for ::first-name and at the same time, different in another spec, I can't globally specify ::tag spec.
2018:02:12 21:55:30 seancorfield @roklenarcic I suspect you'll want a multi-spec for the various types of parsed maps you can get back from clojure.xml
2018:02:12 21:55:47 seancorfield (and the multi-spec would branch on the :tag key's value)
2018:02:12 22:09:21 roklenarcic I'll look into it.
2018:02:12 22:12:42 roklenarcic There's something weird with the large-integer generator. As is, it never generates a number over a 100. Even if I use the large-integer* generator with options, the numbers are very low.
2018:02:12 22:20:47 roklenarcic The test.check generator clojure doc states:
(def ^{:added "0.9.0"} large-integer
"Generates a platform-native integer from the full available range
(in clj, 64-bit Longs, and in cljs, numbers between -(2^53 - 1) and
(2^53 - 1)).
Use large-integer* for more control."
(large-integer* {}))
But in my experience it generates number in about a -100 +100 range instead of 2^64
2018:02:12 22:56:06 taylor try generating a larger sample
2018:02:12 22:56:11 taylor (gen/sample (gen/large-integer* {:min 1 :max 1000000}) 1000)
2018:02:12 22:58:59 roklenarcic hm... I guess I expected that generated numbers were uniformly distributed and so it would be extremely likely that a number over a 1000 was generated in 10 tries
2018:02:12 23:02:55 gfredericks @roklenarcic the large-integer generator exhibits growth just like most of the other generators
2018:02:12 23:03:01 gfredericks meaning it respects the size parameter
2018:02:12 23:03:26 gfredericks gen/sample with 1 arg gives you samples using sizes 0⇒9
2018:02:12 23:08:40 roklenarcic I see, thank you
2018:02:12 23:13:06 roklenarcic The reason I was worried is that when I use integer generator to generate ids, I get a lot of non-unique small numbers (because of the small initial range), which makes it very likely for a sequence of generated entities to have non-unique IDs, which causes such-that to fail occasionally, when IDs are specced to be unique.
2018:02:13 01:35:05 gfredericks there are generators for uniqueness, like gen/vector-distinct-by; and when it has trouble finding different IDs, it increases the size, which should be sufficient here
2018:02:13 09:11:25 holyjak Is there a way to derive a spec from a spec?
We have a checkout flow that in each step adds something to an order and a spec for the final order. I would like to have a spec for each step of the checkout process (ie the final spec with some keys either removed or changed from required to optional).
The problem is that order is not a flat map and may itself contain maps that grow across multiple steps. I want to avoid copy&paste&modify of the end spec. Any ideas? Thanks!
2018:02:13 09:22:23 gklijs Yes, you can do things like
2018:02:13 13:32:20 slipset Maybe I just need to understand the rationale behind why this is intentional behaviour?
2018:02:13 13:53:37 vikeri Any way of overriding the generators so that different specs depend on each other? My use case is testing a function where the arguments have internal relations, i.e. argument 1 contains references to argument 2. Randomly generated arguments will not have this relation naturally.
2018:02:13 14:02:47 alexmiller independently, no. but in a shared context like an args list, yes - you can use combinators like fmap or bind to achieve this
2018:02:14 10:36:56 vikeri I see!
2018:02:14 17:45:56 firstclassfunc hey @alexmiller still a little fuzzy here. Trying to build a generator that produces a common ::id field as in
2018:02:13 21:39:44 misha @slipset if you will flow conformed values through merge's "components" - you will hit false-positive invalidation.
2018:02:13 21:46:11 misha compare:
(s/def :foo/a (s/or :s string? :i int?))
(s/def :m/one (s/keys :req [:foo/a]))
(s/def :m/two (s/keys :req [:foo/a]))
(s/def :m/merged (s/merge :m/one :m/two))
;; not flowing conformed value
(let [m {:foo/a 1}]
(and ;; not accurate, but illustrates the idea
(s/conform :m/one m)
(s/conform :m/two m))) ;; => #:foo{:a [:i 1]}
;; flowing conformed value
(let [m {:foo/a 1}]
(->> m
(s/conform :m/one)
(s/conform :m/two))) ;; => :clojure.spec.alpha/invalid
2018:02:13 21:47:49 misha (I hope @alexmiller will call me on my BS, if it's crazy talk)
2018:02:14 08:27:47 slipset @misha ha I don't quite think this is my use case. I might very well be misunderstanding here, but you're not using :m/merged at all
2018:02:14 08:29:08 slipset And I guess what surprises me here is that the order of how the specs are passed to s/merge is important.
2018:02:14 08:30:25 slipset I guess you could argue that since the order of how maps are passed to clojure.core/merge is important, it follows that the order of the specs passed to s/merge is also important, but I have a hard time accepting that 🙂
2018:02:14 08:40:10 misha @slipset but you do accept clojure.core/merge, don't you? :)
2018:02:14 10:24:21 holyjak If only specs were data and I could use Spectre to transform them...
2018:02:14 10:28:22 mpenet @holyjak there are plans to improve that I think
2018:02:14 10:28:56 mpenet nothing specific was mentioned but they stated the intent a few times
2018:02:14 10:34:44 holyjak Crazy idea: perhaps I could only have a spec for the final order and, when validating for a non-final step of the checkout, I could just filter out exceptions for paths I know this step doesnt require yet.
2018:02:14 10:39:06 mpenet my comment was about spec composition, not about the merge oddities
2018:02:14 10:39:09 mpenet fyi
2018:02:14 10:39:10 vikeri So, if I run a spec with 100 tests and it fails, if I input the seed number it will only also fail if I run 100 tests? The seed doesn’t seem to be for the single test that fails since if I change num-tests to 1 it will usually pass. This decreases the usefulness of seed for functions that take large data structures and take some time to run.
2018:02:14 13:32:48 taylor seeding a RNG just makes it generate a certain sequence of values, it doesn’t mean it’s going to magically find the exact value in the sequence that triggered some behavior and then always generate that number first. I assume the same logic applies to test.check generators?
2018:02:14 13:33:31 taylor anyway, when the check fails it spits out a minimal failing case that you could use in a “manual” test
2018:02:14 14:38:21 alexmiller assuming you don’t use an external source of randomness in your generators , re-using the seed from a test should reproduce the identical generator values and the identical failure
2018:02:14 14:40:50 taylor true, I think the disconnect is expecting the failing case to be re-gen’d first when using the same seed, instead of at whatever position it naturally occurs w/given seed
2018:02:14 14:42:04 alexmiller right
2018:02:14 13:47:53 misha @holyjak specs are sort of data, like everything in clojure opieop
2018:02:14 13:48:30 misha Now it just comes down to how much pain verbosity and macros you can tolerate
2018:02:14 13:48:52 misha s/form
2018:02:14 15:53:10 misha is there a builtin predicate for int in Integer/MIN_VALUE, Integer/MAX_VALUE range?
2018:02:14 16:10:18 madstap @misha I think that's what int? is
2018:02:14 16:17:50 mgrbyte (source int?) - is a check against type (Long/Integer/Short/Byte), not bounds.
2018:02:14 16:17:16 misha @madstap int? includes longs as well
2018:02:14 16:24:05 misha guess I'll use (s/int-in 0 (inc Integer/MAX_VALUE))
2018:02:14 16:26:02 alexmiller isn’t that nat-int? ?
2018:02:14 16:26:14 alexmiller I guess nat-int? goes to Long/MAX_VALUE
2018:02:14 16:34:52 alexmiller if you really want to restrict to java Integer ranges, I think I would make custom predicates and custom specs that use those predicates + appropriate generators
2018:02:14 16:35:15 alexmiller but Clojure is not going to give you that as it does not intend to support them
2018:02:14 16:38:55 misha I am going through kafka's config documentation and writing spec for it.
Sometimes they use int, [0, ...], and sometimes int, [0, ... 2147483647] explicitly
2018:02:14 17:09:32 alexmiller ah
2018:02:14 20:34:09 firstclassfunc is there anyway to force say a large-integer generator to produce unique values?
2018:02:14 20:36:11 taylor @firstclassfunc this might apply
2018:02:14 20:36:53 gfredericks yeah you have to specify it at the higher level, where you know the scope of uniqueness
2018:02:14 20:37:26 gfredericks otherwise you'll have trouble e.g., during shrinking when all your IDs become 0
2018:02:14 20:38:36 firstclassfunc yea i just want to produce a set of integers numbered 1-100 without duplicates
2018:02:14 20:49:46 firstclassfunc I thought I would just narrow the scope of the generator for ::id to do it, but that has been more challenging.
2018:02:14 20:54:17 gfredericks if your IDs are UUIDs you shouldn't have any problems actually
2018:02:14 20:59:32 firstclassfunc the problem is that I need the set of ids to be the same across each record to form a relationship
2018:02:14 21:01:24 gfredericks ah yeah that kind of thing takes more effort
2018:02:16 15:44:52 vikeri Hmm, I can’t generate positive numbers in clojurescript. (s/exercise pos?) doesn’t work. Any pointers?
2018:02:16 16:20:42 alexmiller Pos is not specific to a particular numeric type so does not have a mapped generator
2018:02:16 16:20:58 alexmiller pos-int? does
2018:02:16 16:21:35 alexmiller Or create a composite like (s/and double? pos?) where the initial type will gen
2018:02:17 05:03:50 bmaddy Is there a way to ensure two values are the same in a structure with spec? I didn't see anything in the spec docs (but maybe I missed it). I'd like to define a spec :account/by-id that specs data like this:
2018:02:18 02:39:23 bmaddy Thanks @seancorfield, that's what I was missing!
2018:02:17 12:41:08 gfredericks then you'll have to modify the generator too, to just copy the id from the key spot to the attribute spot
2018:02:17 16:15:15 roklenarcic I'm trying to add clojure.spec.test.alpha/check based testing to my clojure.test tests, but the integration doesn't seem trivial. Do I have to write my own transform of the check map to some sensible assert or is there a lib for that out already?
2018:02:17 16:23:11 gfredericks that's asked a lot; I don't know if anybody's published anything. but in any case doing the integration manually should be just barely nontrivial
2018:02:17 16:23:57 roklenarcic I mean I can whip up something. I have to note that one thing that is very confusing is the :failure key in the return map
2018:02:17 16:24:07 roklenarcic it comes back set to false
2018:02:17 16:24:18 roklenarcic on a failed check
2018:02:17 16:24:32 roklenarcic this will probably confuse a lot of people
2018:02:17 16:30:27 gfredericks yes there's a ticket about that
2018:02:17 16:45:07 roklenarcic There seems to be a curious mix of namespaces in the returned maps as well:
To get predicate that failed you need to get:
2018:02:17 16:46:22 gfredericks yeah, I think spec is currently embedding data in an ExceptionInfo object because test.check didn't used to have a mechanism for adding any info to a failure
2018:02:17 16:46:37 gfredericks I thought it was being unwrapped somehow though
2018:02:17 23:00:13 seancorfield @mathpunk Your test namespace does not require the namespace containing the spec -- how would it know about it?
2018:02:17 23:12:02 mathpunk @seancorfield thanks! Since the name of the spec has the namespace in it, I didn't realize the test namespace still needed to require it
2018:02:17 23:12:55 seancorfield You need to load the namespace for the s/def to be executed, otherwise the spec isn't defined -- the keyword is just a keyword.
2018:02:17 23:13:41 seancorfield Also, spec names (keywords) have no connection to code namespaces (except insofar as the :: alias resolution works).
2018:02:17 23:19:27 mathpunk wow! ok, I thought they were somehow being 'registered' in a way that needed to match the namespaces
2018:02:17 23:23:20 seancorfield s/def and s/fdef are the functions that perform the registration and the spec name just needs to be as globally unique within your application as it needs to be in order to not clash with any other specs.
2018:02:17 23:25:00 seancorfield Namespace-qualified keywords have been around for a long time before spec and have never been tied to code namespaces. For example, we use :ws.web/config as a key in Ring requests in our applications for our application configuration map -- but we don't have a ws.web namespace.
2018:02:17 23:26:52 seancorfield You can also set up aliases without needing code namespaces to match:
The first one is used for specs (`::m/username` for example), the other two are just used as part of unique keys in maps.
2018:02:17 23:29:07 seancorfield (it just so happens we do have code namespaces matching those, but where we use the aliases, we don't need the functions or specs from those namespaces so we don't require the namespaces)
2018:02:17 23:29:44 seancorfield (hope that helps @mathpunk?)
2018:02:17 23:33:12 mathpunk @seancorfield Thank you, it does
2018:02:18 01:30:27 mathpunk I'm writing my first regular expression of specs. Both these tests should pass:
2018:02:18 01:32:36 mathpunk A :sherman.grammar/expanding-term should contain ONE or more :sherman.grammar/expanding-symbols, and ZERO or more :sherman.grammar/terminating-symbols
2018:02:18 01:32:54 mathpunk I thought this might be it:
2018:02:18 01:33:43 mathpunk At least, I thought that would handle "This is #going# #to# #expand# maybe"
2018:02:18 01:33:50 mathpunk which isn't fully what I'm after but would be partway
2018:02:18 01:33:50 taylor have you tried s/alt
2018:02:18 01:33:56 mathpunk I diiiid but not very well
2018:02:18 01:35:38 mathpunk Come to think of it I don't know that my input data is regular
2018:02:18 01:36:24 mathpunk I'm trying to express, "A string that, if you split it at the spaces, one or more of the elements of that collection would be an ::expanding-symbol"
2018:02:18 01:37:47 mathpunk Maybe I should write a function that tests that, and uses it as the predicate
2018:02:18 01:38:16 mathpunk I wanted to check here to see if there is a way to alt this intent
2018:02:18 01:41:30 taylor seems like s/cat works for your case unless I’m missing something
2018:02:18 01:43:38 taylor oh wait, you want your inputs to be actual strings?
2018:02:18 01:44:14 noisesmith s/alt could spec the result of splitting the string at spaces
2018:02:18 01:44:23 taylor in that case, I think I’d tokenize the string before spec’ing it
2018:02:18 01:53:19 mathpunk Getting there. I'm incorrect about what s/+ means, though:
2018:02:18 02:17:10 mathpunk I've changed my terminology a little, but I'm still failing to reject invalid input:
2018:02:18 02:21:48 mathpunk I hypothesize that the right thing to do is to make functions do more work, and specs do less. E.g., much as noisesmith suggested when they suggested tokenizing first, I could have a predicate which detects well-formed tokens, and base a spec off of that
2018:02:18 02:22:09 mathpunk I'm still working out what responsibilities belong in spec and what belong in plain ol' clojure code 🙂
2018:02:18 03:34:58 taylor @mathpunk another option using s/&:
2018:02:18 03:46:37 seancorfield I'll channel Alex Miller and say clojure.spec is not intended for string parsing -- there are much better tools out that for that (e.g., instaparse).
2018:02:18 03:47:27 seancorfield The "regular expression" part of spec is intended for sequence processing.
2018:02:18 03:48:50 seancorfield ^ @mathpunk
2018:02:18 03:58:52 mathpunk I've wondered exactly this!: whether instaparse was obsolete after spec
2018:02:18 03:59:02 aengelberg It is not
2018:02:18 03:59:05 aengelberg Come join us
2018:02:18 03:59:08 aengelberg We have cookies
2018:02:18 03:59:39 aengelberg And somewhat lackluster but existent respond rate to issues
2018:02:18 04:16:35 bmaddy I would expect clojure.spec.alpha/and to be commutative. Is anyone else seeing this behavior?
2018:02:19 12:34:06 igosuki @ikitommi Do you know if there is any way to conform two key aliases into one ?
problematic example :
(s/def ::user
(s/keys [(s/or ::name1 ::name2)]))
Spec verifies for me already that it’s one of the other, subsequently, I don’t want to have to write (or (:name1 m) (:name2 m)) in all of my code
Is there anyway to coerce these keys into another one ?
2018:02:19 12:37:04 mpenet syntax is wrong, I don't think you should use a ns on the or, it should read (or ::foo ::bar)
2018:02:19 12:37:14 mpenet afaik
2018:02:19 12:37:37 mpenet @igosuki I dont think you can no, you need to use eval for that, or a macro
2018:02:19 12:38:03 igosuki yeah sorry I wrote the ns by reflex, but actually am using just a plain or
2018:02:19 12:41:14 igosuki It’s annoying to have a declarative syntax for validation, and then have to write custom code for something that trivial though…
something like (alias ::name1 ::name2) would be nice, equivalent :
import com.fasterxml.jackson.annotation.{JsonIgnoreProperties}
case class User(@JsonAlias(Array("name1", "name2") name))
2018:02:19 15:01:12 mpenet it would really be handy to have specs for specs, I occasionally write (s/def foo ...) or (s/fdef ::foo ...) and it's far from obvious to trace when this happens
2018:02:19 15:02:23 mpenet you get an Illegal arg exception in one case but the error is a bit cryptic:
2018:02:19 15:02:28 mpenet :type java.lang.IllegalArgumentException
:message Invalid match arg: /cdn-cgi/l/email-protection
2018:02:19 17:06:14 l0st3d I'm importing something from some json and would like to use the namespaced map in the rest of the program
2018:02:19 21:26:18 misha you can do it pre- or post- spec, and your further convenience-while-working-with-data-structure is probably irrelevant to spec
2018:02:20 09:31:57 roklenarcic Hm I have a spec that limits string length to be between 0 and 10 characters. As I ask generate to generate a larger sample size, it starts to throw that it failed to satisfy such-that. What's the best way to deal with that?
2018:02:20 13:20:32 gfredericks you have to supply your own generator; (gen/map #(apply str %) (gen/vector gen/char-ascii {:max-elements 10})) or something like that
2018:02:20 13:42:49 roklenarcic test.chuck has a cap-size function that works great for this
2018:02:20 14:03:54 igosuki @misha faire enough, thanks for the heads up
2018:02:20 17:37:17 gfredericks @roklenarcic strictly speaking that's capping the size parameter, not the size of the collection. But they probably happen to coincide in this case.
2018:02:21 01:02:45 richiardiandrea regarding multi-arity functions, can I spec a 4-arity fn this way?
2018:02:21 01:17:43 alexmiller you can also s/alt at the top for the 2, 3, and 4 cases if you want a single map with the same keys conformed out
2018:02:21 17:35:24 richiardiandrea s/alt only works with keywords so does that mean that I need to break the spec in three?
2018:02:21 17:49:53 alexmiller s/alt works with any spec - keywords are used to tag the alternatives
2018:02:21 08:52:09 hawari Can I make a spec definition that depends on another spec like:
(spec/def ::start-time
(spec/and string? valid-datetime?))
(spec/def ::end-time
(spec/and string? valid-datetime?)) ;; need to validate that it's greater than start-time
(spec/def an-entity
(spec/keys :req-un [::start-time ::end-time]))
2018:02:21 09:07:38 moxaj @hawari.rahman17 you should do that validation in the ::an-entity spec
2018:02:21 09:11:47 hawari Can you suggest on how can that be achieved? My apologies, I've just recently started using spec, I don't fully grasp its concept yet.
Do I use spec/and and pair the spec/keys with another predicate which validate that start-time is greater than end-time?
2018:02:21 09:12:48 hawari If so, how can one destructure a map from within a predicate? Does it get treated as an argument to a spec? @moxaj
2018:02:21 09:15:49 moxaj your predicate is just a regular function
2018:02:21 09:16:31 hawari Ah great, I'll try it right away, thank you very much for your suggestion @moxaj!
2018:02:21 13:55:50 yogidevbear Hi everyone. I'm a spec n00b so please have patience with my question here. Can/should I use spec for validating user input passed into my functions? For example, if I have a function that should accept positive integer values only and those value might need to be within a particular range, would spec work for this (and possibly what might the general structure of code look like)?
2018:02:21 14:04:26 yogidevbear Thanks Talyor :+1: I'll check that out now
2018:02:21 14:52:20 alexmiller also check out s/int-in for your range spec
2018:02:21 14:02:06 Olical Considering it makes a very very good parser for all sorts of data structures, that'd probably be a fine use case. Since the result is in itself more data, you could add more constraints in the future then build your error messages from that output.
2018:02:21 14:02:28 Olical Also, by user input do you mean an actual form on a page, or as an API?
2018:02:21 14:03:38 yogidevbear At the moment, user input from a REPL.
2018:02:21 14:04:00 yogidevbear via running lein repl and the calling the functions from the REPL
2018:02:21 14:05:26 Olical Ah, so you're talking runtime, not dev time, which is a big difference. At dev time you can just define your function specs and the user can decide to enable instrumentation (in my opinion). At runtime you'll be able to run your inputs through conform and respond with an informative error if it doesn't fit.
2018:02:21 14:05:26 Olical Ah, so you're talking runtime, not dev time, which is a big difference. At dev time you can just define your function specs and the user can decide to enable instrumentation (in my opinion). At runtime you'll be able to run your inputs through conform and respond with an informative error if it doesn't fit.
2018:02:21 14:07:11 yogidevbear Can you explain what you mean by
>the user can decide to enable instrumentation
2018:02:21 14:08:29 Olical Sure! So, if you define specs for all of your functions with s/fdef, nothing actually happens at runtime. It's just sort of there, like documentation.
2018:02:21 14:08:55 Olical The user, or developer, can decide to instrument a namespace / all namespaces so that these fdef specs are checked.
2018:02:21 14:09:17 Olical In normal spec, this only means the arguments being passed to functions are checked, but you can use something like Orchestra to check the return values too.
2018:02:21 14:21:39 Olical You can use (s/def ::my-num #(s/int-in-range? 5 10 %)) then (s/explain ::my-num 2) gives you a more descriptive error too. Not sure how much spec you know though, so sorry if I'm explaining too much 😅
2018:02:21 14:23:32 yogidevbear At present, I know about 2 hours worth 🙂
2018:02:21 14:23:44 yogidevbear So all the explanation is very welcome
2018:02:21 14:27:08 yogidevbear And if I'm using clojure 1.9.0, do I simply add this to my ns?
2018:02:21 14:29:47 Olical It should be present anyway, it's included as part of Clojure. No need to depend on it afaik
2018:02:21 14:29:52 Olical But yes, that require line is fine.
2018:02:21 14:30:00 Olical And you can use cljs.spec.alpha in cljs
2018:02:21 14:31:01 taylor > Clojure 1.9 depends on this library and provides it to users of Clojure. Thus, the recommended way to use this library is to add a dependency on the latest version of Clojure 1.9, rather than including it directly
2018:02:21 14:31:04 yogidevbear Yeah, it seemed to be throwing some error on my spec function without needing to depend so looks like 1.9.0 includes it by defaut
2018:02:21 14:39:55 yogidevbear I think I've got a spec working without throwing an error when I run lein repl. Excuse the trivial function example:
(defn my-fn
"An example function. Takes two arguments and makes a vector from them"
[a b]
(into [] (list a b)))
(s/fdef my-fn
:args (s/cat :a #(s/int-in-range? 1 11 %)
:b #(< 10 %)))
2018:02:21 14:40:08 yogidevbear Does that seem correct?
2018:02:21 14:41:20 Olical Looks right to me, yep!
2018:02:21 14:41:32 Olical You can put the fdef before the defn if you want to btw
2018:02:21 14:41:50 Olical And to get clojure to check this you have to use spec tools instrument or something like orchestra
2018:02:21 14:42:21 Olical So these are more of a developer tool thing, if you want to check things at runtime that you present to the user, you probably want s/conform and s/explain
2018:02:21 14:42:51 yogidevbear So my next question then... if I run (my-fn 1 2) within the REPL, it works instead of throwing an error
2018:02:21 14:43:09 yogidevbear Is this where the spec tooling you're referring to would come into play?
2018:02:21 14:43:16 Olical Yep, this is where you have to instrument
2018:02:21 14:43:24 Olical You don't want spec checking EVERY function call in prod
2018:02:21 14:43:26 Olical So it's opt in
2018:02:21 14:43:44 yogidevbear Ah, that makes a lot of sense 🙂
2018:02:21 15:52:51 yogidevbear @U38J3881W any ideas why I might be getting Could not locate orchestra/core__init.class or orchestra/core.clj on classpath when running lein repl?
I have [orchestra "0.2.0"] in my dependencies and (:require [orchestra.core :refer [defn-spec]]) in my ns declaration
2018:02:21 15:55:02 Olical Ooh, odd. Maybe it's a doc thing? I suppose it's not found in [orchestra.spec.test :as st]?
2018:02:21 15:55:51 Olical Also, maybe the docs are wrong in another way, try [orchestra "2017.11.12-1"], that one seems to be for newer versions of Clojure.
2018:02:21 15:55:55 yogidevbear Hmm, not sure. They don't seem to include orchestra.spec.test in their defn-spec example on that readme
2018:02:21 15:56:13 yogidevbear Isn't that version the cljs version?
2018:02:21 15:56:41 yogidevbear Let me try the other version and see
2018:02:21 15:57:00 Olical Yeah, I think you want the other one, it's for CLJS and CLJ 🙂
2018:02:21 15:57:50 Olical 0.2.0 seems to be legacy for an older style of spec.
2018:02:21 15:58:17 yogidevbear Thanks, that seems to have fixed it
2018:02:21 16:00:36 Olical Nice! I hope it's all working well. Tried it with expound yet? Definitely makes the errors more palatable.
2018:02:21 16:01:15 yogidevbear Not yet
2018:02:21 16:44:31 viesti hum
2018:02:21 17:40:18 bfabry following on from hawari's example above, what's the best way to attach a generator to the compound spec that generates end-time by adding some random amount to start-time. assume the spec/keys call involves a lot of keys all the things that I'm coming up with have a lot of boilerplate, but I probably just don't understand test.check very well
2018:02:21 17:43:36 gfredericks what do you have so far?
2018:02:21 17:56:36 bfabry well I think I just ran into some unrelated cljs macro problems
2018:02:21 19:13:41 johanatan hi, does spec have support for heterogeneous maps? in particular i want to validate a map like this: {:a-special-kwd [:some :elements] "string" ::some-structure "string2" ::some-structure ... }
2018:02:21 19:16:21 johanatan i suppose i could do: #(s/and (valid-special? %1) (s/valid? (s/map-of string? ::some-structure) (dissoc %1 :a-special-kwd))) ?
2018:02:21 19:16:37 johanatan is there a better way?
2018:02:21 19:23:24 johanatan i suppose i was trying to avoid having lambdas which explicitly call valid? though (if there is a way)
2018:02:21 19:35:27 aaron51 Hello! How do we check return values using fdef and stest/instrument? I see you can add the :ret key, but it doesn't seem to fail when it should.
Also can stest/instrument check all fdef'd functions (instead of having to list them all)?
2018:02:21 19:36:04 taylor stest/instrument doesn’t check :ret specs, only :args AFAIK
2018:02:21 19:36:29 johanatan stest/check does however
2018:02:21 19:36:35 taylor and calling (stest/instrument) with no args should instrument everything that’s been loaded
2018:02:21 19:40:29 aaron51 Any way to check ret without auto generating tests with check?
We're unit testing and trying to use specs on the return value
2018:02:21 19:41:04 taylor I think someone has written some utility code for doing that, can’t remember which lib it’s in
2018:02:21 19:41:40 taylor pretty sure that’s it ☝️
2018:02:21 19:42:34 aaron51 We were looking for a way to continue using defn, instead of defn-spec, because defn-spec confuses Cursive
2018:02:21 19:44:17 taylor I don’t think you’re required to use defn-spec with orchestra
2018:02:21 19:45:01 taylor i.e. just using orchestra’s instrument seems like it will do :ret checking on plain’ ol defns and s/fdefs (hopefully?)
2018:02:21 19:47:10 aaron51 oh wow, that's exactly the answer I needed. We were fighting with Orchestra and Cursive for a while
2018:02:21 19:47:13 aaron51 thanks!
2018:02:21 19:48:22 taylor FWIW (and it probably wouldn’t help you here) but Cursive has a feature where you can tell it to treat custom macros like the core/known macros, assuming the custom macro follows one of those known patterns
2018:02:21 23:11:15 aaron51 oh, we tried that. It partially worked, but then jumping between tests and implementation broke. Seems like the feature we want but it isn't quite fully baked yet
2018:02:21 23:11:21 aaron51 thanks again
2018:02:21 19:39:35 gfredericks @bfabry ignoring the unrelated issues with using rand-int, I think the boilerplate you have is necessary given the current API
maybe a helpful thing would be a variant of s/and that lets you modify the default generator of the first argument
2018:02:21 19:40:22 bfabry @johanatan I think you're looking for s/keys
2018:02:21 19:40:46 johanatan @bfabry no, that won't work. the "strings" (string1, string2) are dynamic/unknown
2018:02:21 19:40:52 bfabry @gfredericks what are the issues with using rant-int and what should I be doing instead?
2018:02:21 19:41:18 bfabry oh sorry I didn't realise you meant heterogeneous keys as well
2018:02:21 19:41:26 johanatan yes, heterogeneous
2018:02:21 19:48:30 gfredericks using your own randomness instead of a generator means you lose growth, reproducibility and shrinking; one way to address that is to add a temporary key to ::call' that maps to something that generates the sort of nonnegative integer you want, and then remove that key in ::call
2018:02:21 19:53:59 bfabry can I just use duration (s/gen (s/int-in 1 3600)) in the gen/let and then use that?
2018:02:21 19:55:03 gfredericks oh yeah that's a good idea
2018:02:21 19:56:24 bfabry ok cool, slowly understanding
2018:02:23 12:01:29 otfrom dumb question of the day: can you add a fsepc to a defmethod?
(I remember reading somewhere you couldn't but I think I'm misremembering and google has been failing me)
2018:02:23 12:47:55 mpenet You can't. you can spec the dispatch fn or/and create separate defns that will be called by the multimethod impl
2018:02:23 13:27:22 otfrom @mpenet thx!
2018:02:23 13:28:32 otfrom I hadn't thought about specing the dispatch function but it is obvious now that you point it out
2018:02:23 13:29:19 mpenet @alexmiller pointed that out the other day, didn't think of it either before then 🙂
2018:02:23 13:29:52 alexmiller That covers the args side at least
2018:02:23 13:29:56 otfrom man, he should write a book
2018:02:23 13:30:09 otfrom oh, sorry didn't see you there @alexmiller;-)
2018:02:23 13:31:49 otfrom @alexmiller I was a bit surprised yesterday and today w/instrumenting an fspec. It looks like it calls gen/check on the way in which I wasn't expecting (21 times if I'm reading correctly)
2018:02:23 13:32:10 otfrom we got caught out b/c we had a spec that was wider than the code that was being called
2018:02:23 13:32:15 alexmiller Yeah, you are not the first
2018:02:23 13:32:19 otfrom 😄
2018:02:23 19:16:50 jjttjj Is leaning on spec's conform/unform to do translations to and from some outside api an intended and/or acceptable use case? IE if the api gives dates and integers as strings and i need to convert them to java dates and real ints when the come in, and convert them back to strings when they go back out to the api, and there are a lot of varied translations like this in a nested structure. I'm thinking I spec all the incoming string data with conformers, then additionally spec out all my own representations as normal. Is there a reason I'm missing not to do this?
2018:02:23 20:08:45 taylor agree that cautioning against “abusing” conformers should be documented
2018:02:23 20:31:06 seancorfield I would have sent a PR for the site repo -- but conformer isn't even mentioned anywhere there. So I think it would have to be a docstring update for the function itself? @alexmiller thoughts?
2018:02:23 21:03:41 alexmiller It’s intentionally not mentioned as anti documentation :)
2018:02:23 21:04:06 alexmiller but maybe it should be more explicitly unmentioned :)
2018:02:23 20:02:20 jjttjj thank you
2018:02:24 08:54:32 Peter Wilkins Hi - I'm learning spec. I've googled and doc'ed but can't find a clue how to spec the sum of all numbers in a collection (a map) in this case
2018:02:24 10:04:11 yogidevbear Does anyone here have an open source repo where they're using spec? I feel like I'm missing something fundamental about spec and I'm hoping that looking at a few good examples of it being used within a project might help solidify some of the core concepts for me.
2018:02:24 10:24:10 jannis @yogidevbear We wrote https://github.com/functionalfoundry/entitydb some time ago. Starting from workflo.entitydb.core/empty-db, most functions and input/output data have specs to allow random testing and integrity checking.
2018:02:24 12:52:10 luskwater Actually, @poppetew, your (s/and ,,, #(= 100 (apply + (vals %)))) would work, if you were using integers. That’s why we needed the approx= fn up there.
2018:02:24 12:53:10 luskwater Your e may vary, of course. 🙂
2018:02:24 17:03:09 robert-stuttaford @poppetew i apologise if my rapidly typed response wasn’t suitably welcoming for you - i figured something would be better than nothing. i recognise now that it could have come across as condescending - not my intent! looks like @luskwater provided a far more considered response, which has taught me something!
2018:02:24 17:03:39 robert-stuttaford @poppetew i am incredibly supportive of beginners 🙂
2018:02:24 17:38:02 luskwater Wouldn't have had an answer for @poppetew so quickly, had it not been for @robert-stuttaford and his initial suggestion. Collaboration across timezones takes a little time itself.
2018:02:25 14:39:26 Petrus Theron Having a hard time with Spec performance on runaway regular expressions for a simple lexer/parser to lex a string into tokens before I parse it. I'm trying to parse a string into tokens so I can put it through a second "compiler" pass:
I'm not using any optional s/? regexes, so I don't understand why this would cause any kind of exponential search
2018:02:25 18:59:05 seancorfield @petrus clojure.spec is not intended for string parsing -- you know that, right?
2018:02:25 18:59:45 seancorfield The regexes are for "parsing" sequences, not strings. You probably want to look at Instaparse which is designed for parsing strings.
2018:02:26 07:46:43 Petrus Theron Thanks - a string is a sequence, no? 😉 I can switch to instaparse for tokenising, but I'm worried about the length of data sequences I'll be able to handle with Spec
2018:02:26 17:54:07 seancorfield A string can be coerced to a sequence 🙂 I saw @U0516PHE3 mention that your spec is an ambiguous grammar and that might be causing the exponential slow down. As for data sequences, I would expect most folks would use coll-of or every and the elements would be self-contained specs (so, no ambiguity).
2018:02:26 19:54:49 Petrus Theron How can I make my spec unambiguous? If I split my string into a word tokens, can I still use spec and coll-of/every to parse the substrings? Or do I need an instaparse step before I send it to spec?
2018:02:26 20:04:24 seancorfield Right now "aa" can parse as "atom (a)" "atom (a)" or "atom (aa)" so you could specify that the overall sequence is "atom" then a sequence of "whitespace" and "atom" perhaps? (assuming you trim leading/trailing whitespace before parsing?)
2018:02:26 20:05:02 seancorfield If you require whitespace separating the atoms, then "aa" can only parse as "atom (aa)"
2018:02:26 20:07:10 seancorfield You'll want to make sure whitespace and line endings don't overlap too (so there's no ambiguity there) and if you care about the difference have your overall spec clearly indicate line / sep / line structure and line as a spec indicate atom / whitespace / atom structure. Make sense?
2018:02:25 19:02:43 gfredericks it's still a surprising slowdown, I would say
2018:02:25 19:02:54 gfredericks (and I did say it, in fact)
2018:02:25 19:08:34 seancorfield I'm kinda surprised even the 15-element string is handled "fast". That would be long for an argument list so I'd say it was outside the intended design space?
2018:02:25 19:11:19 gfredericks the spec regexes are only meant for arglists?
2018:02:25 19:11:26 seancorfield Processing "drive" takes 3ms, "drive home" takes 72ms, "drive home fast" takes almost 1 second. That doesn't feel "fast" to me.
2018:02:25 19:12:41 gfredericks I coincidentally just wrote code that uses spec to parse the lines of a 17277-line file; it took 0.6s
2018:02:25 19:13:09 seancorfield @gfredericks Not just arglists but I don't think folks are spec'ing long sequences in data...? And data is more likely to get spec'd with coll-of or every I think?
2018:02:25 19:13:19 ghadi I wonder why that example appears to show exponential behavior. What are the bounds on parsing with derivatives?
2018:02:25 19:13:30 gfredericks I'm using regexes because I actually want the parsing capabilities, and I think this would be too hard with instaparse
2018:02:25 19:13:40 ghadi There was an updated paper Matt Might released that improved performance.
2018:02:25 19:14:22 ghadi Even if that example above isn't a bug -- it's a valuable example for spec's test suite.
2018:02:25 19:14:45 ghadi (... but it's probably a bug)
2018:02:25 19:16:04 gfredericks yeah I can't imagine any good reason for something that simple to take a long time
2018:02:25 19:16:32 seancorfield I got an OOM error trying to parse that long string 👀
2018:02:25 19:16:58 ghadi Is the performance still bad if you feed it the seq directly and remove the s/and & the conformer?
2018:02:25 19:22:43 seancorfield Adding each new word seems to double the time it takes. I think there's a clue in how it conforms -- for one word you get a sequence of one element which is a sequence for the atom: (time (s/conform ::plexpr "one")) => [[:atom ["o" "n" "e"]]] (in just a ms or two), but (time (s/conform ::plexpr "one two")) => [[:atom ["o" "n" "e"]] [[:whitespace [" "]] [:atom ["t" "w" "o"]]]] -- note: two elements, with the second containing the whitespace and the next word. Adding more words still produces a two-element sequence with the second one containing all the extra elements.
2018:02:25 19:34:18 aengelberg You could change the set predicates to functions that print some debug information. That would give you an idea of what things the spec engine is unnecessarily repeating.
2018:02:25 19:38:13 aengelberg I think that parser is ambiguous, because a string of "aa" could match [:atom ["a"]] [:atom ["a"]] or [:atom ["a" "a"]]. That could explain the exponential growth.
2018:02:25 19:57:07 gfredericks ooh good point
2018:02:25 23:59:06 devn How does one access a spec within the registry's req/req-un/opt/opt-un info?
2018:02:25 23:59:27 devn I'd like to peek at a spec's list of req/req-un/opt/opt-un
2018:02:26 00:00:44 devn are s/describe or s/form what I want to use here, or am I missing something obvious?
2018:02:26 00:01:54 devn I'd like to peek at req and opt to pass them into a pull query
2018:02:26 00:51:52 devn @seancorfield that'll get me the spec objects by name, but not in a way that's easy to pluck out the req/opt
2018:02:26 00:52:50 devn i have a feeling im just missing something obvious, maybe in how i define the :req to begin with. Is it possible for me to use a spec as the arg to :req I wonder
2018:02:26 00:53:02 seancorfield Yeah, right now you'd have to get the form for each spec and then walk them.
2018:02:26 00:53:21 seancorfield Since the keys part could be embedded deep inside a spec...
2018:02:26 00:53:48 gfredericks doesn't somebody mention a yet-to-exist spec spec every time this comes up?
2018:02:26 00:55:00 seancorfield Alex says they're looking at ways to make it easier to build specs programmatically but it's not there yet.
2018:02:26 00:55:47 devn err, well, when it's a set, I can at least do (s/form ::the-keys) and get back #{::a ::b}, it's just that I can't use it in the s/keys form
2018:02:26 00:56:22 seancorfield clojure.spec is a wonderful concrete example of how macros don't compose 👀
2018:02:26 01:02:51 devn im not complaining. i think there's probably a solution im missing here, let me try to be a bit more clear on what i want to do.
I have a spec which says that ::a and ::b are required.
I need some value holder that can be used both in the spec definition: (s/keys :req ...) and a pull query (d/q '{:find [(pull ?blah ?the-qualified-keys) :in [$ ?the-qualified-keys]] ...} db the-qualified-keys)
2018:02:26 01:05:35 devn err, i could relax that second part and say I'm not sure if that's possible. if it isn't, what kind of clever solutions are there? I suppose I could generate the spec and a corresponding def, for instance, so i don't have to manually keep modifications to either place consistent.
2018:02:26 01:11:02 seancorfield What we've done at work is to define a spec with keys, build any more complex specs on top of that, then get the spec form itself, parse out the :req [...] part and pull the keys from that to use elsewhere. So the base spec was the "system of record" and we derived other stuff from it.
2018:02:26 01:11:26 seancorfield It's kind of icky but it works, given what we have available right now. It seems like the path of least resistance.
2018:02:26 01:13:17 devn maybe (s/describe) is good enough for what i want to do here, the structure of it just felt a little wonky to me: (keys :req [:foo/a :foo/b]), so i need to do something like (apply hash-map (rest (s/describe ::the-map))) or whatever
2018:02:26 01:19:10 seancorfield Hmm, we use s/form... what is s/describe?
2018:02:26 01:19:18 devn roughly the same deal
2018:02:26 01:19:40 seancorfield Ah, an "abbreviated" form as data...
2018:02:26 01:21:32 devn i'm surprised it's not a map, but perhaps there are Reasons™ why that I don't understand
2018:02:26 01:21:59 gfredericks getting a map isn't hard though: (apply hash-map (rest thing))
2018:02:26 01:31:16 devn fair point
2018:02:26 01:31:51 devn it gave me that "is this a hint that i shouldn't be using it like this?" feeling
2018:02:26 01:37:47 gfredericks probably just a hint that it's not done yet
2018:02:26 13:26:20 mpenet @alexmiller any details/insight about how could spec meta data look like in the future?
2018:02:26 13:39:30 alexmiller No
2018:02:26 13:26:59 mpenet another question while I am at it: was it ever considered to implement (s/def ::foo ::bar) via clojure.core/derive? I would personally find it quite interesting with that form, you can leverage ancestors/isa? etc
2018:02:26 13:43:11 alexmiller Not sure if Rich ever considered that.
2018:02:26 13:44:18 mpenet does that sound like something desirable in your opinion?
2018:02:26 13:46:20 alexmiller Don’t know. The example above I would consider aliasing, not a hierarchy. The notion of derivation with respect to s/keys (containership) is interesting though for additive aggregations.
2018:02:26 13:46:41 mpenet we use that on a small lib to add metadata to specs, we then can just say spec ::x derives from another and add useful little utils on top of that that can optionally fetch all the metadata from the ancestors and merge them for instance.
2018:02:26 13:46:55 mpenet yes exactly why I am asking
2018:02:26 20:09:19 alexmiller I think probably replacing the big set with a char int numeric check would probably help a lot
2018:02:26 20:12:46 ghadi I don't think so @alexmiller -- it's OOMing which means some extra retention
2018:02:26 20:13:08 ghadi I think we could remove the chars entirely and have a smaller example
2018:02:26 20:23:32 alexmiller if you replace the :atom branch with something like (defn atom-char? [c] (let [i (int c)] (or (<= 48 i 57) (<= 65 i 90) (<= 97 i 122)))), I think you’ll find it doesn’t have the issue
2018:02:26 20:24:45 ghadi trying it...
2018:02:26 20:26:50 ghadi it does not help at all @alexmiller
2018:02:26 20:27:02 alexmiller helped for me
2018:02:26 20:27:14 ghadi did you (s/+ atom-char?)
2018:02:26 20:27:20 ghadi mine is still running
2018:02:26 20:27:40 alexmiller oh, maybe I just did atom-char?
2018:02:26 20:27:59 ghadi pretty sure it's the s/alt and s/+ combo
2018:02:26 20:28:00 alexmiller not (s/+ atom-char?)
2018:02:26 20:28:05 alexmiller yeah, prob
2018:02:27 12:33:33 borkdude I have written a macro to get rid of some boilerplate. I’m using a spec to validate arguments passed to a generated function.
Is there a good solution for this, e.g. key destructuring using spec?
2018:02:27 12:41:21 gfredericks I'd probably use plumbing/safe-get
2018:02:27 12:42:22 borkdude yeah, I thought about writing that before I wrote the specs… might be just as effective
2018:02:27 12:42:54 borkdude but nice to have it in a lib
2018:02:27 15:51:23 misha well, it has nothing to do with spec opieop
2018:02:27 16:33:13 borkdude but it does what I want. 😄
2018:02:27 16:37:25 gfredericks theoretically it overlaps with spec because the spec could tell you whether you're accessing a key that can be ex-spec-ted to be there
2018:02:27 16:40:58 stathissideris it has get, get-in, assoc, assoc-in, update, update-in
2018:02:27 16:41:21 stathissideris you can also compose the lenses
2018:02:27 16:45:26 misha @gfredericks should it, say, throw, if you are asking for key not in spec?
2018:02:27 16:45:45 gfredericks @misha that sort of thing, yeah
2018:02:27 16:46:27 misha then it'd be nice app-level let-speced macro
2018:02:27 16:46:41 stathissideris that’s what spectacles does! (more or less)
2018:02:27 16:47:04 stathissideris if someone wants to extend it with let-speced pull requests are very welcome
2018:02:27 16:47:45 misha I think you are missing binding part of destructuring, @stathissideris
2018:02:27 16:48:17 misha this is where the money is
2018:02:27 16:48:35 stathissideris definitely, but there is quite a bit of code there to introspect the specs etc
2018:02:27 16:49:07 stathissideris so it’s a good base to build on, that’s what I’m saying 🙂
2018:02:27 16:49:45 misha I think the most common and useful case would be specced-destructuring for maps (s/keys)
2018:02:27 16:50:28 misha because most of conformed things end up being maps anyway opieop
2018:02:27 16:51:03 stathissideris the main reason I didn’t do it was the syntax, couldn’t come up with a nice way to express it
2018:02:27 16:51:06 misha and seq-destructuring does not need any names from input value anyway
2018:02:27 16:51:21 stathissideris because the spec name has to be mentioned
2018:02:27 16:51:50 misha true
2018:02:27 16:52:09 stathissideris it’s already a bit awkward with get-in
2018:02:27 16:52:49 misha I'd love it to have let-like form
2018:02:27 16:54:04 stathissideris @misha can you come with an example of what it would look like using it?
2018:02:27 16:58:10 misha not yet, thinking of options.
what I know, is:
- binding has to be a core feature, to avoid repetition
- should look "familiar", so plugins, like #cursive could understand it (most probably like let)
- should be natural to use, because following does not really looks good:
2018:02:27 16:59:19 misha but that is second best of my ideas (but I dont have top1 yet :) )
2018:02:27 16:59:39 stathissideris instead of :keys you give the name of the spec
2018:02:27 16:59:54 stathissideris (haven’t worked out the implications of the syntax yet)
2018:02:27 17:00:46 misha this is better, although preserving all destructuring features would be neat
2018:02:27 17:03:03 misha (not saying you can't have both with this design, just need to think it all through)
2018:02:27 17:03:58 stathissideris yeah, the ideal situation would be for spectacles to handle the “special” parts and pass on anything not related to spec to clojure.core/let
2018:02:27 17:04:40 misha up next:
how to bind keys from the same map using different specs?
2 lines? or 2 specs in destructuring form?
So the input map should conform to the spec, which I can validate with e.g. s/assert, s/valid? etc.
2018:02:27 17:10:15 borkdude but you can also get the keyword wrong of course, this is what spectacles checks
2018:02:27 17:10:46 stathissideris @borkdude are you saying that the last call should throw instead of returning nil?
2018:02:27 17:10:55 misha I think the value proposition here would be, "(say) throw, if trying to bind value under key not in spec"
2018:02:27 17:11:26 borkdude both things can go wrong: your input map or your selection in the map
2018:02:27 17:12:51 misha if we are still on let-like macro topic: I'd just check if the key is supposed to be in map according to spec,
because validation can be expensive, and I'd like to be able to separate the two
2018:02:27 17:13:17 misha although it starts to sound like static types opieop
2018:02:27 17:13:56 borkdude safe-get is the most effective in my case here: I just get an exception and prints out the keys that were in the map and the key I used. I can figure out which one was wrong
2018:02:27 17:14:35 borkdude I guess you can mess up specs themselves too, nothing protects you from typos… typos all the way down
2018:02:27 17:15:10 misha from a broader perspective, destructuring and select-keys - are a bit orthogonal to each other, and both would need to be addressed individually
2018:02:27 17:15:42 stathissideris @misha select-keys would be a good (and easy) addition to spectacles
2018:02:27 17:16:07 borkdude I just discussed select-keys with someone else. After you use select keys you can still mess up selecting from the result of select keys
2018:02:27 17:16:29 stathissideris not if you’re using lens/get! 🙂
2018:02:27 17:16:54 misha @borkdude this is why I want let :)
2018:02:27 17:17:35 misha I much more often (destructure) bind keys, than select-keys and then bind
2018:02:27 17:18:33 stathissideris would be good to work out a good syntax for this
2018:02:27 17:19:52 misha if collision with :my-ns/keys can be solved - it is the best syntax there could be, already
2018:02:27 17:20:28 mpenet Personally I would just add a :spec key in the map destructuring form and limit it at that: {:as foo :keys [..] : spec ::foo}
2018:02:27 17:20:30 stathissideris just don’t ever call your specs ::keys😂
2018:02:27 17:21:00 misha another data point, i usually don't use keys in :keys, but symbols
(let [{:my.ns/keys [foo bar]} m])
2018:02:27 17:21:01 stathissideris @mpenet that does sound a bit less collision prone, thank you!
2018:02:27 17:21:26 mpenet Would work for seq destructuring too
2018:02:27 17:21:57 misha @mpenet you then miss on opportunity to destructure by multiple specs in the same map :)
2018:02:27 17:22:17 mpenet Yes true, trade-offs
2018:02:27 17:22:27 stathissideris @mpenet I’ve added your idea to the issue
2018:02:27 17:22:32 stathissideris with attribution!
2018:02:27 17:22:35 misha you probably don't need spec-destructuring for seqs
2018:02:27 17:23:01 mpenet Prolly not
2018:02:27 17:23:06 misha unless you are willing to support spec form walking for nested specs like s/or
2018:02:27 17:23:34 mpenet Spec walking is gross
2018:02:27 17:23:35 misha then OMG you'd solve "damn, I forgot which keys I used in s/or for this"
2018:02:27 17:23:50 mpenet Too alpha to play this game imho
2018:02:27 17:25:07 mpenet Necessary evil for now
2018:02:27 17:25:10 misha 2 years strong alpha kappa
2018:02:27 17:25:57 mpenet I am hoping we ll get something better for introspection/composition
2018:02:27 17:26:39 stathissideris spectacles relies on looking at forms to figure out the keys
2018:02:27 17:26:51 stathissideris but yeah it feels risky
2018:02:27 17:27:13 misha at this point I'd like to have compositional spec-walking
2018:02:27 17:27:51 misha it seems like everyone does it over and over again, for spectacles, for expound, etc.
2018:02:27 17:29:41 mpenet Yup, spec-tools, speculate, etc been there done that too and trying to avoid doing it from now on;)
2018:02:27 17:31:12 stathissideris spec-based seq destructuring could check that
(1) the spec is in fact a coll-of, a cat or a tuple
(2) for finite length specs, you’re not attempting to destructure more elements than what is defined in the spec
2018:02:27 17:31:22 stathissideris not sure how useful…
2018:02:27 17:39:18 stathissideris sorry I’m not sure what that emoticon means!
2018:02:27 17:39:26 stathissideris @misha thanks for the comment
2018:02:27 17:40:23 misha I went for "not really useful"
2018:02:27 17:41:24 stathissideris yeah maybe not really worth it
2018:02:27 17:56:33 misha
2018:02:27 17:58:28 misha oh wait, is this fixed in spec already, sweet
2018:02:28 04:27:00 bbrinck Is there a way to provide a seed to s/exercise or gen/sample? I have some code that uses these functions and I’d like to test the result, so forcing a specific seed would keep my tests reliable.
2018:02:28 04:27:54 bbrinck (or, if there is another test.check function that can generate values based on a fix seed, I could use this)
2018:02:28 10:27:25 taylor @bbrinck looking at test.check there seems to be a way to do this by tweaking the sample-seq function that gen/sample wraps:
the only practical difference being the sample-seq in test.check doesn’t take seeds. sample is just calling this function with a default sample size. consider this answer extremely non-authoritative, but it does always generate the same samples for a given seed 🙂
2018:02:28 11:49:37 gfredericks also gen/generate has a seed arg in the master version I believe
2018:02:28 11:50:41 gfredericks it wasn't there originally because sample is meant to be a dev-only function
which raises the question of what you're doing with it that it needs to be tested 🙂
2018:02:28 12:40:55 misha @gfredericks eyeballing data structure generated with the same seed is easier while developing spec for it. Presumably, only subset of data structure, related with spec change, would change on subsequent generate calls
2018:02:28 12:42:31 gfredericks yeah that makes sense; I think I misread what @bbrink said originally also, I had thought he must be calling sample in non-test code
2018:02:28 12:43:25 misha I think you were not wrong :)
2018:02:28 12:44:22 misha but exposing seed argument would benefit dev-time use case.
2018:02:28 12:46:10 gfredericks yep; for now gen/generate is pretty similar
2018:02:28 13:15:14 taylor nice, I was only looking into the 0.9.0 code
2018:02:28 15:11:56 gfredericks I'm like two years late at making a new release
2018:02:28 16:55:49 misha and a book!
2018:02:28 16:59:43 gfredericks and a guide
2018:02:28 17:00:09 misha
2018:02:28 17:00:20 gfredericks second edition of the book
2018:02:28 17:00:35 misha lambo license plate
2018:02:28 17:00:54 gfredericks company offering enterprise support
2018:02:28 14:33:38 bbrinck @taylor @gfredericks thanks for the ideas. Yes, my original question about was using s/exercise (with random seed) in non-test code, but then controlling the seed for tests (to make them reliable). Although, on further consideration, I guess it would be fine to control the seed in non-test code as well. The use case is, given a spec, to print out example data.
2018:02:28 14:34:42 bbrinck In this case, the randomness of examples isn’t desirable
2018:02:28 14:36:40 taylor I also did this to auto-gen some API docs, but I was too lazy to try using a fixed seed
2018:02:28 14:43:33 gfredericks Yeah, test.check generators are awkward in a few ways for the example use case
2018:02:28 19:10:55 jjttjj what if I want to spec something as being an ordered list of "params" or a map of the same params with names attached, like this:
I have many different things that need to be spec'd like this. Is this worth trying to write some hacky function to make easier to type out? Or is this probably just a bad idea and I should stick to being explicit about it?
2018:02:28 19:15:43 gfredericks the :arglist version is unfortunate because it dictates the order of the keys
2018:02:28 19:21:54 jjttjj not sure I get what you mean. the idea is i can figure out what i'm getting either via the order of the values, or by names being associated with the values
2018:02:28 19:22:16 gfredericks oh I'm sorry
2018:02:28 19:22:47 gfredericks I misread the first half as being like (f :a1 x :a2 y ...)
2018:02:28 19:23:08 gfredericks disregard
2018:02:28 19:23:14 jjttjj no worries 🙂
2018:02:28 19:37:08 alexmiller s/keys* can be used for your arglist I think
2018:02:28 19:37:30 alexmiller it’s like keys, but in any order, in a sequential structure
2018:02:28 19:45:28 gfredericks I don't think that kind of signature is even being used, that was my misunderstanding
2018:02:28 20:00:45 jjttjj yeah. it's more like "if x is a map, validate it with s/keys. if it's sequential, validate the conformed map after calling s/cat on it"
2018:02:28 20:01:38 misha is s/keys* considered being public api?
2018:02:28 20:02:04 jjttjj I believe it is, it's how you're supposed to validate optional keyword arguments i think
2018:02:28 20:03:13 jjttjj like (json->clj x :keywordize true) or something. it's mentioned in the spec guide
2018:02:28 20:12:27 alexmiller yes, the * here is supposed to be evocative of s/*, not as a marker for something implementation-ish
2018:02:28 20:13:19 alexmiller you can still use s/keys* here in with a wrapper s/spec so you get a layer of nesting
2018:02:28 20:15:25 alexmiller
(def s (s/or :arglist (s/spec (s/keys* ...stuff...))
:argmap (s/keys ...stuff...)))
2018:02:28 20:16:31 alexmiller or maybe I’m still not understanding the intent
2018:02:28 20:17:45 alexmiller I guess looking at your examples above, I’m not
2018:02:28 20:19:09 alexmiller for the arglist, I think you can just (s/and (s/cat ...) (s/keys ...)) ? where the s/keys bit is the same spec as :argmap, presuming so, I’d pull out the :argmap spec into it’s own thing to reuse it
2018:03:01 01:41:06 clumsyjedi I must be missing something very simple about composing specs
2018:03:01 01:44:01 clumsyjedi so the ::nums spec uses s/conform and changes the value being validated. So ::int is trying to validate [:pos 5] and failing.
2018:03:01 01:44:46 clumsyjedi How do I avoid this though? I just want to chain validations together, each validating the original value I pass in
2018:03:01 01:45:50 gfredericks I feel like I ran into this recently but can't find any evidence of that
2018:03:01 01:46:30 gfredericks I think there's an s/unform or something like that that could be a workaround
2018:03:01 01:46:35 gfredericks but that seems like it shouldn't be necessary
2018:03:01 01:46:50 gfredericks I know this has been talked about a lot, just don't remember any conclusions
2018:03:01 02:02:56 clumsyjedi thanks @gfredericks - It seems like for my actual work scenario I can use s/merge instead of s/and.
2018:03:01 02:03:33 clumsyjedi I'm curious what the recommended approach is for the example I pasted though
2018:03:01 02:03:49 clumsyjedi I googled around for unform, didn't find much
2018:03:01 03:17:11 alexmiller It’s best to start s/and with the pred that conforms to itself, particularly a type predicate like int? that will also generate
2018:03:01 03:17:59 alexmiller You’re probably thinking of the undocumented s/nonconforming wrapper
2018:03:01 03:18:13 alexmiller Still on the fence about that one
2018:03:01 03:19:13 alexmiller However I’d just do (s/and ::int ::nums) here
2018:03:01 03:21:15 alexmiller And s/or is better than s/alt here (although you won’t see any difference in behavior until you combine ::nums with another regex spec
2018:03:01 14:39:41 misha I use s/nonconforming quite often (not sure how to feel about this though)
2018:03:01 15:57:46 stathissideris here’s a tricky situation: I have a very large spec to describe my very large config. The config contains values in different nested maps that need to be consistent with each other (some have to co-occur, some refer to each other so IDs need to be consistent etc), so in order to validate the config I have extra predicates on the top-level spec to check for consistency. That was great, until I tried generating a config, and of course it wasn’t valid (because it’s very unlikely to get consistency by chance). So I started writing overwriter functions that would be used via fmap in the generator in order to re-align the different parts of the randomly generated config. But it does feel a bit like an uphill struggle. Am I missing something?
2018:03:01 16:15:44 alexmiller generating large structures with semantic constraints is (inherently) hard
2018:03:01 16:16:06 alexmiller an alternate approach is to supply a custom generator at the point of use (with stest/check for example) that picks from example configs
2018:03:01 16:16:20 alexmiller by using a generator on a set of enumerated examples
2018:03:01 16:34:09 stathissideris @alexmiller thanks for the insight. I don’t like the idea of giving up the variability that I get from using an actual generator instead of example configs (which I assume you’re implying would he “hand”-written). You’re right, it’s inherently hard… I’ll see if I can refactor a bit to decomplect the constraints between distant parts of the structure, and also see if I can express the ones that cannot be avoided in a more fluent way.
2018:03:01 16:45:28 alexmiller the general approach if you are using custom generators is to first build a generator for a simpler model, then use gen/bind to create a generator that produces the appropriate structure
2018:03:01 16:49:55 stathissideris I’ve done this before, but I think it’s not so applicable in this case, this one has cases where you get a bunch of IDs generated for part of the structure (say metrics collection), and then in another part (for example filter-metric) you need to refer to one of the existing metrics, so the overwriter just replaces the generated filter-metric with one from the IDs in the metrics collection
2018:03:01 16:51:44 stathissideris you’d think that a flag on one of the metrics (something like :filter?) would be better… I’m beginning to think the same 🙂
2018:03:01 17:11:02 cap10morgan I have a string I want to write a spec for, but it is composed of previously-spec'd string A, followed by a /, and then previously-spec'd string B. They are GitHub "user/repo" strings. If I already have a :github/user spec and a :github/repo spec, how would I write a spec for the "user/repo" strings that composes those first two specs?
2018:03:01 17:13:12 alexmiller spec is not great for doing composition of string specs like this - I suspect composing the patterns and creating a spec from that will probably end up better
2018:03:01 17:13:54 cap10morgan 👋 Hey Alex! OK, thanks. I figured it was either something like that or I was missing something really simple. I'll go down the route you suggest. Thanks!
2018:03:01 17:14:41 alexmiller it’s possible, I just don’t know that you’d be happy with the spec, the generated values, or the conformed values once you were done
2018:03:01 17:14:51 cap10morgan I see
2018:03:01 18:49:46 misha @stathissideris generator is "just a function", so you can generate a set of ids upfront, and inject those in sub-generators. it will be verbose and might not be pretty, but sometimes you got to do what you got to do...
2018:03:01 18:50:13 misha (or change config format, if you control it)
2018:03:01 18:50:46 ghadi that's what I do ^
2018:03:01 18:51:02 ghadi generate the stuff that you want to "align" first -- then pass them down
2018:03:01 18:51:57 misha another option, is to generate sub-structure and explicitly overwrite id. this saves you some code, and sub-generators customizations
2018:03:01 18:53:58 misha ofc, "thicker" dependency is – tighter your generators would be coupled, and "just finish it with assoc-in's" would not be enough
2018:03:02 17:08:24 jimbob I don’t really understand why Cloujre spec is any better than simple pre conditions for simple validation. example:
2018:03:02 17:13:16 jimbob i suppose so, but is not these pre conditions and or conditions documentation enough?
2018:03:02 17:13:23 jimbob and much fewer lines of code
2018:03:02 17:13:43 bbrinck @ben.borders what’s the error message if you pass in something incorrect? What if one of the args was not just a string, but, say, a vector of strings?
2018:03:02 17:13:47 jimbob (assuming we are not doing generative testing)
2018:03:02 17:18:07 jimbob its not terrible
2018:03:02 17:18:14 jimbob ex:
Exception in thread "async-dispatch-12" java.lang.AssertionError: Assert failed: (integer? event-uri)
2018:03:02 17:18:24 jimbob but yes not as good as spec errors.
2018:03:02 17:19:06 bbrinck I think the different may be starker if you pass in nested data e.g. a vector of maps that must contain keys
2018:03:02 17:19:55 bbrinck but for simpler predicates like string? (as in your example), the differences in error messages won’t be as significant
2018:03:02 17:20:40 jimbob sure, but then couldnt i just use clojure.test/is ?
2018:03:02 17:38:09 nha I don’t recall what exactly but this ended up causing trouble for me down the line (maybe it was nested is? I don’t remember now)
2018:03:02 17:20:51 jimbob and get things like
;FAIL in
2018:03:02 17:22:18 bbrinck Yes, although it becomes a little less clear for nested data e.g. a vector of strings
2018:03:02 17:22:28 bbrinck or vector of maps of specific keys
2018:03:02 17:27:48 jimbob i see
2018:03:02 17:28:06 jimbob thanks for your responses
2018:03:02 17:29:38 bbrinck I copied and pasted from my REPL session and omitted my mistakes, so hopefully I didn’t miss anything important 😉
2018:03:02 17:32:19 jimbob so then to get the same desired runtime validation, would it be advised to place s/valid? in the :pre block of the function? or are there better ways to do this?
2018:03:02 17:32:48 jimbob i’ll probably end up exploring this.. it does indeed seem to be much better self documenting
2018:03:02 17:33:39 bbrinck there are a few options. You can use fdef, or you could use a s/assert
2018:03:02 17:34:08 jimbob mm alright
2018:03:02 17:34:42 bbrinck for s/assert, keep in mind you’ll need to run (s/check-asserts true)
2018:03:02 17:35:14 bbrinck you could also use s/explain-str and raise an exception
2018:03:02 17:35:34 bbrinck (if the result is not literally "Success!\n")
2018:03:02 17:43:05 jimbob gotcha. and for speccing the example map above, would i have to create an s/def for every key in the map? for example some of the keys in the map should contain the same type and structure of data, so it would be nice to reuse something like
2018:03:02 17:49:21 alexmiller yes, you will need one for every key
2018:03:02 17:49:44 alexmiller that puts it in the spec registry which is used by s/keys
2018:03:02 17:49:56 jimbob ok, gotcha thanks alex
2018:03:02 17:50:02 alexmiller if you have many repetitive such things, macros can help
2018:03:02 17:50:08 jimbob right, cool
2018:03:02 17:50:22 alexmiller maybe eventually we’ll provide something to def multiple things at a time
2018:03:03 11:55:40 pfeodrippe Woww, I saw the problem, s/and enters at the land of predicate instead of staying at the regex as said by the great @alexmiller at https://groups.google.com/forum/#!topic/clojure/S3UaS_4FRqY. Gonna use s/&
2018:03:05 02:50:28 seancorfield @mpcarolin You can use back tick you expand a symbol's name:
2018:03:05 02:52:52 mpcarolin @seancorfield Hey, thanks! I did not know that existed. It will save me a lot of redundant typing.
2018:03:05 06:15:37 tianshu how to write a spec allows only the specified keys？ is there a convenient way?
2018:03:05 06:17:08 tianshu I want to use spec to constraint the schema of edn which used as a configuration file. I want useless keys are not allowed.
2018:03:05 09:29:50 stathissideris @doglooksgood there is no official way, you use s/and with s/keys and another predicate that constrains the keys to be in a specified set, but then you are listing the keys twice. Some people have written macros for this sort of thing
2018:03:05 09:30:27 stathissideris but spec is like that by design, probably the decision that has caused the most controversy about it
2018:03:05 09:31:41 tianshu so I should write something like s/keys, okay
2018:03:05 14:26:12 Andreas Liljeqvist Asking because it is like 3 months since the last commit
2018:03:05 14:31:17 bronsa yes
2018:03:05 14:31:50 bronsa clojure repos usually see activity in bursts not continuous
2018:03:05 14:32:10 bronsa apart from initial development
2018:03:05 14:37:02 Andreas Liljeqvist ok, thanks. Looking to fix a bug
2018:03:05 14:44:18 Andreas Liljeqvist Now I just have to figure out how it is built, I expected a .deps.edn,project.clj, build.boot, makefile....
2018:03:05 14:51:16 alexmiller It’s maven - you can test with mvn clean test etc
2018:03:05 14:53:11 Andreas Liljeqvist got it to run with mvn clojure:compile
2018:03:05 14:53:49 Andreas Liljeqvist going with mvn clean test since I want to fix a problem
2018:03:05 14:53:50 Andreas Liljeqvist thansk
2018:03:05 16:55:20 nnbosko Is there a way to make a function return an error if parameters don't conform to their given spec?
2018:03:05 17:37:32 mpcarolin I think what you’re looking for is spec.test/instrument. Once you execute it, it will tell you if any :args specs are violated for a given function.
2018:03:05 17:59:21 mpcarolin But if you want it at runtime without instrument, you can also use :pre condition for a function with s/valid?
2018:03:05 16:57:37 nnbosko The function's given spec, anyways
2018:03:05 17:17:53 gfredericks does clojure 1.9 come with specs for clojure core functions somehow?
2018:03:05 17:20:15 taylor there must be some coverage in there, here’s a spec error for invalid let:
CompilerException clojure.lang.ExceptionInfo: Call to clojure.core/let did not conform to spec:
In: [0] val: () fails spec: :clojure.core.specs.alpha/bindings at: [:args :bindings :init-expr] predicate: any?, Insufficient input
#:clojure.spec.alpha{:problems [{:path [:args :bindings :init-expr], :reason "Insufficient input", :pred clojure.core/any?, :val (), :via [:clojure.core.specs.alpha/bindings :clojure.core.specs.alpha/bindings], :in [0]}], :spec #object[clojure.spec.alpha$regex_spec_impl$reify__2436 0x482c71e3 "
2018:03:05 17:23:22 gfredericks well that's a macro, which is a pretty different use case (in particular, it's checked at compilation; function specs would only get involved if a user intentionally instrumented them)
2018:03:05 17:18:34 gfredericks (or are there official specs somewhere else?)
2018:03:05 17:29:51 gfredericks I guess they'd probably be in core.specs.alpha if anywhere, and I see those are basically just a handful of macros
2018:03:05 17:59:45 bronsa that’s all there is available
2018:03:05 19:22:30 gfredericks surely somebody somewhere is assembling a non-official collection of such things?
2018:03:05 20:13:32 bbrinck But they might be sufficient for some perf testing
2018:03:05 20:13:51 gfredericks :+1: thanks
2018:03:05 20:15:39 alexmiller They’re not good
2018:03:05 20:16:36 bbrinck If we had a large enough test suite across enough projects, could we infer “good enough” specs for core functions with https://github.com/stathissideris/spec-provider ? 🤔
2018:03:05 20:17:12 bbrinck I understand that for general usage, “good enough” is not really a concept, or we’ll get confusing false positives and false negatives 🙂
2018:03:05 20:18:50 bbrinck But in my case, “good enough” specs are useful so I can see if error messages can be improved
2018:03:06 08:13:08 stathissideris @U08EKSQMS spec inference for functions is still work in progress in spec-provider (but not too far off). I suspect it wouldn’t be able to give you anything very useful if you ran it on map though (which I think is very tricky to write a spec for anyway)
2018:03:05 19:22:59 gfredericks I wonder how much it would slow down a test to instrument all of clojure.core 😄
2018:03:05 20:16:16 alexmiller A lot :)
2018:03:05 20:16:59 gfredericks do you think some aspect of this will be useful for some things at some point?
2018:03:05 20:17:27 alexmiller I hope so
2018:03:05 20:18:02 alexmiller I have worked with portions of core speced and it’s really useful at the repl
2018:03:05 20:18:41 gfredericks do you think there's a reasonable place to draw the line that maximizes utility and speed?
2018:03:05 20:18:43 alexmiller I think it’s mostly not useful in a test suite (if the suite already runs)
2018:03:05 20:19:01 alexmiller I don’t feel ready to answer that yet
2018:03:05 20:20:22 gfredericks :+1:
2018:03:05 19:25:42 cap10morgan I have a utility fn that can operate on any map, so its :args spec is just (s/cat :v (s/keys)). But when I do generative testing with that, I get a bunch of empty maps (which makes some sense I suppose). What's a good way to make that generate a bunch of random maps instead (ideally w/ an emphasis on namespaced keyword keys)?
2018:03:05 19:32:52 cap10morgan Should I just use map? instead of s/keys?
2018:03:05 19:34:12 gfredericks sounds plausible
I'm of the opinion that virtually all uses of maps fall into two nonoverlapping categories, roughly corresponding to s/keys and s/map-of
2018:03:05 19:34:25 gfredericks map? seems to not commit to either
2018:03:05 20:22:21 bbrinck @alexmiller Are specs for the core library something that contributors could help with? In other words, are they blocked on dev/reviewer time, or are they more fundamentally blocked on any upcoming changes in spec?
2018:03:05 20:24:23 alexmiller They are really blocked on Rich review time. have written some of them but don’t have them public anywhere
2018:03:05 20:24:59 alexmiller They tend to raise many questions with issues of taste and judgement
2018:03:05 20:25:42 bbrinck I see, that indeed seems like it would require quite a bit of time to review
2018:03:05 20:25:52 bbrinck and change accordingly
2018:03:05 20:26:06 alexmiller Eric Normand actually was working on some of this at one point and I gave him some ideas of things to do with it
2018:03:05 20:26:32 bbrinck He was working on official specs? Or on his own set?
2018:03:05 20:28:14 bbrinck I understand the issues of taste will take time to resolve, but to the extent that the official specs need testing, I’m sure the community could provide feedback there
2018:03:05 20:28:24 alexmiller He wanted to help and so I told him what would be helpful
2018:03:05 20:28:28 bbrinck Gotcha
2018:03:05 20:29:24 bbrinck Perhaps correctness is much easier than making idiomatic specs that conform/generate cleanly 🙂
2018:03:05 20:29:42 gfredericks I doubt correctness is easy either
2018:03:05 20:29:47 seancorfield And there's also the issue that some functions are extremely hard to spec correctly...
2018:03:05 20:30:09 bbrinck Correct, although that’s one place where the community could help - install specs, run tests, see what breaks
2018:03:05 20:30:40 alexmiller I don’t remember now what that was but I think I asked Eric to help in categorizing what was out there. I think he did but it was a while ago
2018:03:05 20:31:42 alexmiller My experience in testing core specs is that it’s extremely difficult to test more than one at a time
2018:03:05 20:35:40 bbrinck Hm, that’s a good point. Since we can see the list of isntrumentable symbols, a project could loop over them, instrument each one in isolation, and run tests
2018:03:05 20:35:56 bbrinck that way, if there was a failure, it would be isolated from other specs.
2018:03:05 20:36:40 bbrinck again, if it was useful to get broader validation of specs by running across a large variety of project
2018:03:05 22:54:19 Scot Hi, is there a good way to spec anonymous functions? I am trying to use fspec, but I keep getting Cannot read property for_all_STAR_ of undefined
2018:03:05 23:39:33 seancorfield @scot-brown Do you have org.clojure/test.check as a dependency?
2018:03:05 23:41:19 seancorfield In order to check whether the function conforms to the spec, it is tested used a generative approach, so you need test.check in your project for that (`clojure.spec` will try to load it dynamically and you'll get some odd-looking failures if it isn't available)
2018:03:06 11:20:13 roklenarcic How would I create a BigDecimal generator?
2018:03:06 12:05:05 taylor you could use fmap to coerce some other number generator to BigDecimal but you wouldn’t get the full range of values (and you might not want that?)
2018:03:06 14:18:28 gfredericks yeah it's hard to give a good answer without knowing what kind of distribution you want; but (gen/let [factor gen/large-integer, log gen/int] (* factor (... 10 to the logth power ...))) wouldn't be bad
2018:03:06 14:23:39 gfredericks there's probably a method on BigDecimal that would do the scaling a bit more naturally
2018:03:07 00:07:58 roklenarcic thanks
2018:03:07 01:14:15 taylor Haha I just read that the largest possible big decimal value can consume about 8GB RAM
2018:03:07 01:33:52 gfredericks I'm disappointed that there's a largest
2018:03:06 19:02:28 cap10morgan Is there a good workaround for keeping NaN out of your spec-generated test data? Filtering at the top level doesn't keep them out of nested data structures, and they break all kinds of equality tests when they show up.
2018:03:06 19:04:18 seancorfield @cap10morgan I thought the double-in generator had options to suppress that?
2018:03:06 19:04:32 cap10morgan I'm working with any?
2018:03:06 19:05:02 seancorfield Ah, then you'll get "anything" 🙂
2018:03:06 19:07:11 cap10morgan yep. there just doesn't seem to be a way to define any-but-nan. my code handles NaNs just fine, it's the specs that barf on them (since the equality tests in my predicates choke on them).
2018:03:06 19:07:57 cap10morgan makes spec'ing fns that can handle any value tricky
2018:03:06 20:24:59 cap10morgan I have tested it locally and it seems to solve the problem.
2018:03:06 21:09:40 gfredericks this is a sticky issue because merely removing NaN from everywhere isn't obviously the best thing, since NaN is an important edge case to test in a lot of situations
2018:03:06 21:09:58 gfredericks it's obviously hard to opt-out of at the moment
2018:03:06 21:17:55 cap10morgan Agreed but the concept of removing it from the any? generator seemed at least tacitly endorsed by the core team.
2018:03:06 21:18:28 cap10morgan If they instead requested a way to toggle it, I’d be happy to work on that too.
2018:03:06 21:19:37 noisesmith pedantically, if you accept any value, you better do the right thing if the value is NaN right?
2018:03:06 21:21:14 cap10morgan Yes. And the code does. Only the spec has a problem. And it is not easy to work around.
2018:03:06 21:21:35 cap10morgan (In my use case)
2018:03:06 21:22:41 cap10morgan A NaN at any level of nesting blows up equality checks in predicates.
2018:03:06 21:23:16 dpsutton and you said your code gracefully handles that, right?
2018:03:06 21:23:26 dpsutton > do the right thing if the value is NaN right?
2018:03:06 21:25:29 cap10morgan Yes, the code handles it just fine.
2018:03:06 21:25:41 cap10morgan It’s not doing any equality checking.
2018:03:06 21:30:52 cap10morgan I think my ideal solution might be a spec-specific equality function that handled NaN, but I don't know where the core team stands on that.
2018:03:06 21:32:40 alexmiller Just driving by here - would much rather prevent the NaN from occurring in the first place. Custom equality has never gone well imho :)
2018:03:06 21:33:18 cap10morgan @alexmiller That's fair. 🙂 Does that patch ^^^ seem OK to attach to the JIRA ticket?
2018:03:06 21:36:07 alexmiller No time to look atm but it’s ok either way :)
2018:03:06 21:36:21 cap10morgan OK, attaching it then.
2018:03:06 23:06:09 gfredericks the downside to that is that the list of generators that gen/any is composed of is now in two places
2018:03:06 23:08:09 cap10morgan @gfredericks Not sure I follow. Where are the 2 places?
2018:03:06 23:10:08 gfredericks Didn't you paste & tweak that code from test.check?
2018:03:06 23:13:07 cap10morgan yes, but it is the one and only code for the any? generator now. and test.check itself largely repeats that one-of form between the any and any-printable generators.
2018:03:06 23:14:22 gfredericks Yes, I was pointing out that if test.check added anything to its gen/any, that wouldn't propogate to spec anymore
2018:03:06 23:14:36 cap10morgan If test.check defined the vector separately I definitely would have just re-used it though. Not sure there's a good way to re-use it as is.
2018:03:06 23:14:52 gfredericks Probably not
2018:03:06 23:15:47 gfredericks Even if test.check changed you couldn't take advantage of that without introducing a stricter version requirement between the two. I'm not sure if that's acceptable or not
2018:03:06 23:18:17 alexmiller Not
2018:03:07 00:06:03 gfredericks that was my guess
2018:03:07 15:36:07 Andreas Liljeqvist is there any function like (complete ::somespec {::provided-value 2}) that will generate keys that aren't provided?
2018:03:07 15:37:57 Andreas Liljeqvist I could provide overrides like {::provided-value #(gen/return 2)}
2018:03:07 16:33:18 Andreas Liljeqvist I have written such a function, just interested to know if it exists somewhere in spec
2018:03:07 17:11:19 jimbob is there any way to have fdef functions be runtime validated?
2018:03:07 17:11:37 jimbob Or are these function specs only for documentation / testing and generation
2018:03:07 17:11:48 taylor you can instrument the functions to check the :args specs
2018:03:07 17:12:24 taylor and there’s another lib called Orchestra that gives you a version of instrument that checks :args and :ret (and maybe :fn?)
2018:03:07 22:00:18 aengelberg I would expect only one message to get spit out, not two. But since s/merge calls each of its sub-maps' validators, it gets the error message twice.
2018:03:07 22:35:45 misha @aengelberg each s/keys validates keys for presence, and every key of a known spec (even not mentioned ones) for conformance
2018:03:07 22:38:13 misha and s/merge does not reduce issues from each subspec
2018:03:08 11:29:13 Andreas Liljeqvist If I found a lot of problems with indentation in spec source, what is the best chance of doing a successful pull-request?
2018:03:08 14:03:27 Andreas Liljeqvist Thanks, actually I am registered as a contributor.
More like if the changes would be accepted in any form.
Perhaps they don't consider it as a problem
2018:03:08 14:43:06 tbaldridge I doubt it, it’s really hard to make a good argument about indentation
2018:03:08 14:43:18 tbaldridge Got an example though?
2018:03:09 13:00:36 tbaldridge on the second one I don't see a problem at all
2018:03:09 13:01:08 tbaldridge to be frank, indentation doesn't matter much in Clojure, so I doubt these sort of patches would ever make it through.
2018:03:09 13:01:36 tbaldridge The style guides that do exist for Clojure are more opinions than actual rules, so it's fairly common to see changes in indent styles across projects.
2018:03:09 13:43:04 Andreas Liljeqvist Disclaimer, I know I cannot possible "win" this
2018:03:09 13:43:30 Andreas Liljeqvist The problem is that a lot of tools force indentation, parinfer for example. So diffs becomes problematic
2018:03:09 13:44:45 Andreas Liljeqvist This is kind of a problem with the tools, but I figure that Clojure would be better of with a standard like pep8
2018:03:09 13:46:38 Andreas Liljeqvist Mostly because indentation really doesn't matter that much. But a standard that enables linters and automatic formatters is valuable according to me
2018:03:09 12:47:43 guy Hi people, whats the best way to generate a not blank string?
2018:03:09 12:48:16 gfredericks you can wrap the generator in gen/not-empty
unless you meant something different by "blank"
2018:03:09 12:48:46 guy ok thanks!
2018:03:09 15:40:55 Andreas Liljeqvist I have made a fix for CLJ-2311 - an issue that was preventing me from providing an override for the dispatch-fn(key) in a multimethod
2018:03:09 22:22:46 gfredericks that works at one level, yes
2018:03:09 22:22:53 gfredericks schema does it recursively
2018:03:10 02:33:21 mpcarolin Hi everybody. I’ve been learning spec lately, and I think I have a decent grasp of its mechanics. What I don’t fully grasp is the real life effective use of spec. When should we be using spec? Should I be speccing as many of my functions and as much of my data as I can? Or just the borders? Or just what is publicly accessible?
2018:03:10 03:32:29 alexmiller there is no one right answer to that question
2018:03:10 03:32:44 alexmiller I don’t think you should or need to instrument everything
2018:03:10 03:33:10 alexmiller spec’ing at the borders is particularly helpful, but any function with sufficiently complex inputs or outputs is worth considering
2018:03:10 03:33:58 alexmiller instrumentation is worth using at dev time, check is worth doing at test time, and validation is (maybe) worth doing at runtime
2018:03:10 04:49:11 mpcarolin Thanks for the response. I’ll keep that in mind!
2018:03:10 21:14:47 borkdude we’re considering putting some (s/assert ...) at the beginnings of some functions, which can be compiled away with the right system property
2018:03:11 11:22:50 borkdude We have a poor man’s graphQL in which we can express *: {:a {:filter/default {:b true}}} applied to {:a {:some-key {:b 1 :c 2}}} gives you {:a {:some-key {:b 1}}}
2018:03:11 20:44:58 borkdude As of now it’s kind of hard to relate function arguments to the specs for them. Maybe this is what the above tweet was also about: creating a different naming scheme
2018:03:12 14:08:53 misha @borkdude what do you mean by “relate” exactly?
2018:03:12 14:10:21 misha The only coupling/relation you should have is order.
2018:03:12 14:36:56 oconn I’m working on writing a spec for a model that uses multi-spec. Is it possible to use unqualified keywords in a multi-spec? I found this thread https://www.mail-archive.com/clojure@googlegroups.com/msg99107.html and tried altering the example in the docs with no success.
2018:03:12 14:43:29 bbrinck > Is it possible to use unqualified keywords in a multi-spec?
@oconn Can you explain this a little bit more? I’m not sure I understand the question.
2018:03:12 14:44:40 taylor @oconn it’s possible, and the example in the multi-spec doc string uses an unqualified keyword
2018:03:12 14:46:10 oconn Oh nice, yeah not sure what I’m doing wrong here.. So here’s the spec
2018:03:12 14:49:36 taylor that explain returns “Success!” for me
2018:03:12 14:50:35 taylor having trouble inferring what the variant-type "Text" spec is intending to do though
2018:03:12 14:51:40 oconn I stripped out a lot of other keys, there are other types as well.
2018:03:12 14:52:41 bbrinck @oconn Can you expand a bit on what is happening when you call explain and what you expect to happen?
2018:03:12 14:55:23 bbrinck I also see "Success!" printed, but I’m guessing that’s not what you expect (?)
2018:03:12 14:56:15 bbrinck (minor thing, but your defmethod appears to have a missing paren at the end)
2018:03:12 14:56:23 oconn Yeah, not seeing that but I think it’s because I stripped out some keys before pasting here.
2018:03:12 14:57:21 oconn Ok, now I’m getting Success!. Sorry for the trouble
2018:03:12 14:57:28 bbrinck np, good luck!
2018:03:12 19:23:24 eggsyntax I'm currently using gen/generate to make some sample data for myself, including fairly nested structures which use s/* at multiple levels. It'd be nice during dev to have the produced data be fairly short so it doesn't take up a whole repl screen. Wasn't there some way to tell it to keep those short? I vaguely thought there was. From the documentation it seems like it would be s/*recursion-limit*, but if I do
I still get large numbers of items on the s/* specs.
2018:03:12 19:26:44 eggsyntax Oh, I see why it's not *recursion-limit*, I think; it's not actually recurring many layers deep, just creating a long sequence at each layer.
2018:03:12 19:28:02 eggsyntax But it'd be really nice to have an equivalent way to force shorter sequences. Am I missing something?
2018:03:12 19:30:49 misha I'm afraid you are not missing anything, @eggsyntax
2018:03:12 19:31:18 eggsyntax Ah well. I guess I can walk it and take 3 or something.
2018:03:12 19:31:23 misha you can provide overrides, which wrap original seq specs with just items count upper limit
2018:03:12 19:31:33 misha yeah ~
2018:03:12 19:32:20 eggsyntax Oh, so something like (s/with-gen (s/* ...) (take 3 ...))
2018:03:12 19:32:39 misha I'd love to be proven wrong on this one, though : )
2018:03:12 19:33:28 eggsyntax It'd make a terrific addition to a future version of spec, to have s/*sequence-limit* alongside s/*recursion-limit*.
2018:03:12 19:34:42 misha I doubt it is a good idea to have a single global count limit. I'd rather had a way to override :min-count/:max-count opts for maps and seqs specs
2018:03:12 19:35:57 eggsyntax From my POV, that'd be nice too -- but for the case of just quickly generating a piece of data in the REPL, it'd be lovely to just bind a global limit.
2018:03:12 19:37:03 eggsyntax Hmm, test.check does have :min-elements and :max-elements on a number of its generators, I see....
2018:03:12 19:37:06 misha like an extra count args here:
(with-gen* [_ gfn] (every-impl form pred opts gfn))
2018:03:12 20:59:16 triss why are all the facilities for defining specs macros? I’m tempted to use spec for pattern matching but the syntax is too heavy…
2018:03:12 21:29:15 alexmiller they’re macros to capture the form for explanations
2018:03:12 21:29:28 alexmiller that said, some of that is likely to change in the next rev
2018:03:12 21:57:35 pavani @alexmiller How can I validate a schema like this using Spec?
I have the following spec but if ::billing_contact has keys other than the optional keys, it considers the map to be valid. I just want to check if the map has either those specified keys in opt or no keys.
2018:03:12 22:06:56 alexmiller You can s/and with an every? predicate that checks every key is a known key set
2018:03:12 22:16:46 pavani @alexmiller Thank you for that. I am not sure how exactly I could use that here. Could you please exhibit that with my example?
2018:03:13 03:30:20 bbrinck Whoops, I forgot the call to keys. Would this work: (s/def ::billing_contact (s/and (s/keys :opt [::street_address ::city ::region ::postal_code]) #(every? #{::city ::street_address ::region ::postal_code} (keys %))))
2018:03:13 22:10:42 pavani Thanks a lot @U08EKSQMS
2018:03:13 03:36:03 rahulr92 Hi, can someone help me understand the difference between spec/and and spec/merge?
The example of merge given in the Clojure Spec guide https://clojure.org/guides/spec (using :animal/common and :animal/dog) seems to work even if we use and instead of merge.
2018:03:13 04:17:02 seancorfield @rahulr92 s/merge will merge two s/keys specs (and, if I recall correctly, does not flow the conformed value); s/and is for combining a general spec with a general predicate (or spec) -- and it does flow the conformed value.
2018:03:13 04:25:22 seancorfield In the merge docs, you'll see another difference: it generates maps that conform to all the merged specs. With and, the first spec is used as the generator, and then the other specs are used as predicates to filter the generated values (which may not satisfy those predicates and so overall generation of values can fail).
2018:03:13 04:25:33 seancorfield Hope that helps @rahulr92?
2018:03:13 04:40:50 rahulr92 @seancorfield Thanks for the help. I understood the difference in generated value. However I still do not fully understand the difference while using these for validation.
Could you please elaborated on what you mean by the ‘flow’ of conformed value?
> ’Unlike ‘and’, merge can generate maps satisfying the union of the predicates.
Doesn’t and also perform union i.e. it passes validation only if all the specs in it pass?
2018:03:13 04:52:36 seancorfield @rahulr92 Here's an example showing a difference
2018:03:13 04:55:33 seancorfield If you look at the result of conforming that map against (s/keys :req [::a]) you'll see why it fails with s/and but works with s/merge:
2018:03:13 04:58:41 seancorfield (which is no surprise because the ::anded spec won't accept anything)
2018:03:13 04:58:51 seancorfield Does that help @rahulr92?
2018:03:13 05:09:28 rahulr92 @seancorfield Great explanation. This really helps. Thanks a lot!
2018:03:13 16:50:35 tbaldridge I'm not sure if anyone else has had this issue before, but I have a really strange spec problem that has shown up twice: I have a spec that defines a or:
2018:03:13 16:51:37 tbaldridge And then I put that inside something else:
(s/def ::coll-of-stuff (s/coll-of ::val))
2018:03:13 16:53:07 tbaldridge But when I try to validate some data, explain fails and tells me:
(s/explain ::coll-of-stuff '[x 1 2])
"value [:int 1] failed to conform to integer? or symbol?"
2018:03:13 16:54:23 tbaldridge The problem is that something is trying to validate the conformed value? What's strange is I can't seem to reproduce this in a smaller situation. But so far it's always been with s/or and every time it tries to validate the conformed value
2018:03:13 16:57:52 guy how are you doing the symbols out of interest?
2018:03:13 16:58:07 tbaldridge doing the symbols?
2018:03:13 16:58:15 guy (s/explain-data ::vals ['s 1])
2018:03:13 16:58:18 guy like that?
2018:03:13 16:58:32 tbaldridge that was part of the example here, I'm actually using keywords
2018:03:13 16:58:33 guy The symbols in your data you are trying i mean sorry
2018:03:13 16:59:01 guy Sorry i’m a little confused
2018:03:13 16:59:46 tbaldridge So here's the actual error message:
2018:03:13 17:02:33 tbaldridge So what I don't understand is why it's saying [:attr :bar] is failing to conform to simple-keyword?
2018:03:13 17:02:52 tbaldridge It's like or is trying to validate after it conforms the value or something.
2018:03:13 17:03:55 jgdavey Wouldn’t [:attr :bar] be a (s/coll-of simple-keyword?), rather than a simple-keyword??
2018:03:13 17:05:12 tbaldridge this is interesting, at the top level, there's a multi-spec. If I remove that it seems to work fine.
2018:03:13 17:06:16 jgdavey Oh, I see, you’re saying that [:attr :bar] is what you’d get after conforming (s/conform ::select-attr :bar)
2018:03:13 17:06:21 tbaldridge @jgdavey that's what's strange, the data I'm passing in looks like this [:foo :bar]. Hence the path in the error being [:ast/select-attrs 0] so it gets into the select-attrs path, and then the first item doesn't validate because [:attr :bar] is not a simple-keyword.
2018:03:13 17:06:29 tbaldridge right
2018:03:13 17:07:21 jgdavey Does using s/every instead of s/coll-of yield the same results?
2018:03:13 17:08:21 tbaldridge wow, yep, I changed my ::select-op spec to this:
2018:03:13 17:09:45 tbaldridge That's the top level spec. If I run it through multi-spec the values it tries to conform are [:attr :bar]. If I run it without multi-spec, the values are :bar
2018:03:13 17:09:53 tbaldridge So multi-spec is doing something
2018:03:13 17:15:23 tbaldridge let's see if I can get a minimal case now
2018:03:13 17:38:59 tbaldridge guess so: " Returns a spec that returns the conformed value. Successive
conformed values propagate through rest of predicates."
2018:03:13 17:42:03 tbaldridge boom that's it, @jgdavey I used s/and and that flows conformed values through the predicates (rather mis-leading name). Replaced it with s/merge and it works fine
2018:03:13 17:43:40 jgdavey Interesting. That’s probably a subtly worth calling out in docs or something.
2018:03:13 18:02:46 tbaldridge yeah, funny thing is that the docs are fairly clear, both s/and and s/merge mention it. I wonder if the behavior there changed somewhat recently
2018:03:13 18:05:15 tbaldridge lol, nope, it's been like that since the start
2018:03:13 18:24:29 mpenet I guess both could have options to control how/if the flow of conformed values is done
2018:03:13 18:26:09 mpenet I tripped on the behavior of s/merge when I started with spec, so the other way around. I am not alone either
2018:03:13 19:00:59 tbaldridge Yeah, I think part of it is that names are a tad overloaded in English. Merge makes sense to me since I'd expect (s/merge ...) to and things together. But it seems like a better name for s/and would be s/flow or something like that.
2018:03:13 19:01:10 tbaldridge Maybe we should rename s/merge to s/smoosh😛
2018:03:13 19:01:38 alexmiller we considered having s/and (non-flowing) and s/and-> (flowing) at one point
2018:03:13 19:01:54 tbaldridge s/& works the same way
2018:03:13 19:01:55 alexmiller unfortunately the current s/and does the latter :(
2018:03:13 19:02:10 tbaldridge but in the end, I didn't read the docs, so I'll slap my own hand this time
2018:03:13 19:02:11 alexmiller yes, s/and and s/& flow. s/merge does not
2018:03:13 19:02:26 alexmiller s/merge is conceptually pushing the value through all specs “in parallel”
2018:03:13 19:03:17 alexmiller so flowing doesn’t make sense (except conforming returns the conformed result of the last spec which is hard to reconcile with that view)
2018:03:13 19:06:11 tbaldridge So what's the common use-case for flowing specs? I think I've only ever used them with s/conformer, which I tend to avoid these days
2018:03:13 19:12:23 alexmiller if you flow through a regex you’ll get structure which you can then easily apply predicates to
2018:03:13 20:02:03 bronsa naming vars & is not the best idea
2018:03:13 20:15:28 alexmiller it was originally called andre :)
2018:03:13 20:19:39 misha flowing is handy when you have s/keys spec checking map's keys and vals independently, and then you apply predicate checking inter-kv-dependencies.
2018:03:13 20:20:46 misha although, thinking about it a moment longer – this use case is sort of orthogonal to flowing :D
2018:03:13 20:28:12 tbaldridge 'Is it pronounced "an-dray" or "and-re"?'
2018:03:13 20:28:51 tbaldridge Would be the new "a-sosh" or "a-sock". 😄
2018:03:13 20:49:10 alexmiller the former
2018:03:14 01:32:56 gfredericks if it was big you could call it andre the giant macro
2018:03:14 01:33:24 gfredericks @bronsa what's wrong with &?
2018:03:14 08:31:07 bronsa syntax quote doesn't qualify it
2018:03:14 11:56:40 gfredericks why on earth would that be true?
2018:03:14 12:12:02 bronsa because otherwise you’d have to
`(fn [a# ~'& b])
2018:03:14 12:24:29 gfredericks aw dang
I figured it somehow related to &env and &form; forgot about destructuring
2018:03:14 12:25:15 gfredericks so this only affects users who are refering s/&
2018:03:14 12:25:34 bronsa yeah
2018:03:14 12:25:42 bronsa or macros defined in the same ns
2018:03:14 12:25:52 bronsa but tbh the same happens when naming a var def
2018:03:14 12:25:56 bronsa which I also don’t like
2018:03:14 12:26:14 gfredericks that's a different underlying cause, right?
2018:03:14 12:26:40 bronsa no same reason
2018:03:14 12:27:33 gfredericks I guess I figured special formedness was checked in one place and &ness somewhere else; are there any other exceptions?
2018:03:14 12:28:26 bronsa they’re all in one place
2018:03:14 12:28:27 bronsa
user=> (keys Compiler/specials)
(& monitor-exit case* try reify* finally loop* do letfn* if clojure.core/import* new deftype* let* fn* recur set! . var quote catch throw monitor-enter def)
2018:03:14 12:28:38 bronsa & is considered a special form by the reader
2018:03:14 12:33:14 gfredericks welp.
2018:03:14 13:47:11 mikerod Insightful
2018:03:14 22:28:18 dpsutton can you spec cljs.core/+?
2018:03:14 22:31:53 dpsutton and i realize that's a seemingly silly question but we've got js/big floating around which allows for arbitrary precision decimal arithmetic but the built in operators do not behave well. So i'd like to spec it and use it to help verify that no js/Big's are leaking into any arithmetic anywhere
2018:03:15 00:45:55 gfredericks I don't think you could instrument it (which I assume is the point) at runtime, since it's normally inlined
2018:03:15 00:46:33 gfredericks I'm curious how easily compilation could be modified to avoid the inlining
does cljs spec do runtime instrumentation at all? I don't even know that
2018:03:15 05:11:57 didibus How does one go about adding functionality to spec? I tried to extend s/keys so that it also did a contains check, basically making it a closed-key spec. I did so by wrapping s/keys in a separate macro which takes the same input then s/keys and returns an anded spec of keys and one which will validate that no extra keys are present. But composing macros like that isn't great. And so now I'm trying to extend it even more, so that my closed-keys macro can be used in a s/merge style. Anyways, bottom line is, the experiment has been tedious. So I was thinking, maybe instead of building on top of s/keys, I can just build a whole new spec, but how do you do that?
2018:03:15 05:13:04 didibus I also found it frustrated all the specs were macros, which some of them were plain old functions that took clojure data. Like s/keys could have taken a map, or a vector of vectors, instead of a custom macro DSL.
2018:03:15 05:57:27 ikitommi @didibus current version of spec doesn’t really support extensions. But you can reify clojure.spec.alpha/Spec and copy-paste current spec internals to make something similar.
2018:03:15 06:01:49 ikitommi .. actually the closed keys would be easy to implement on top of that via extra :closed? key or similar. hmm.
2018:03:15 13:44:00 guy If you were writing an fdef for a simple function that takes a collection of strings (at least one) could you use (s/+ string?) instead of (s/coll-of string?) for the :args.
2018:03:15 16:24:25 guy ahhhhh right got it thanks
2018:03:15 14:07:12 alexmiller absolutely
2018:03:15 14:07:49 alexmiller I would even say that’s preferred (although you won’t really see much practical difference)
2018:03:15 14:14:37 guy ok great thanks
2018:03:15 14:26:06 bbrinck @guy keep in mind coll-of also has :min-count arg which might be useful here
2018:03:15 14:26:12 bbrinck (although more verbose than +)
2018:03:15 14:26:34 guy ah yes thats right thank you!
2018:03:15 14:33:12 alexmiller generally when spec’ing function signatures, you’re spec’ing syntax, and spec regex ops are the best match so I usually try to stick to those in fdef args
2018:03:15 14:33:58 guy ok so in general favour spec regex ops then?
2018:03:15 14:40:50 alexmiller when spec’ing function args, favor regex op specs
2018:03:15 14:41:09 alexmiller when spec’ing other kinds of data, I favor collection specs
2018:03:15 14:42:20 guy Can you give me an example of the latter sorry
2018:03:15 14:42:44 guy Do you mean if you were creating a spec for some sample data perhaps?
Now I want to have a warning for myself when someone passes something that has more keys than I expected. Can I use my spec for this or do I have to duplicate the keys?
2018:03:15 15:40:26 borkdude I’m refactoring this component, so other arguments are suspicious
2018:03:15 15:44:49 seancorfield @borkdude You can extract the list of keys from the spec form itself, and write a predicate that checks those are the only keys. Stu Halloway posted an example on the mailing list a while back I think... Probably in a Gist somewhere...
2018:03:15 15:50:24 borkdude @taylor Oh that’s great. I think I will adopt Schpec
2018:03:15 16:05:15 borkdude I want to use schpec in clojurescript, but now I also have to bring in test.check.. ok
2018:03:15 16:07:32 borkdude Hmm, I already have that… maybe I should upgrade. I get errors about clojure/spec/gen/alpha
2018:03:15 16:09:56 gfredericks What sort of errors?
2018:03:15 16:10:00 borkdude
• public/js/app.js
WARNING: Use of undeclared Var com.gfredericks.schpec/limit-keys at line 53 src-cljs/dre/components/widget.cljs <-- this is my namespace
WARNING: No such namespace: clojure.spec.gen.alpha, could not locate clojure/spec/gen/alpha.cljs, clojure/spec/gen/alpha.cljc, or JavaScript source providing "clojure.spec.gen.alpha" at line 53 src-cljs/dre/components/widget.cljs
WARNING: Use of undeclared Var clojure.spec.gen.alpha/fmap at line 53 src-cljs/dre/components/widget.cljs
2018:03:15 16:11:55 borkdude I first tried the one with #_ but then I get:
No such namespace: com.gfredericks.schpec, could not locate com/gfredericks/schpec.cljs, com/gfredericks/schpec.cljc, or JavaScript source providing “com.gfredericks.schpec” in file src-cljs/dre/components/widget.cljs
2018:03:15 16:19:13 gfredericks Does clojure.spec.gen.alpha exist in cljs?
2018:03:15 16:19:58 borkdude hm, it appears not no
2018:03:15 16:20:08 gfredericks Interesting.
2018:03:15 16:20:51 borkdude ah, cljs.spec.gen.alpha does exist
2018:03:15 16:21:56 borkdude have you tried this in clojurescript?
2018:03:15 16:23:10 borkdude probably limit-keys should be in a cljc file?
2018:03:15 16:23:57 borkdude and some other adaptations
2018:03:15 16:24:15 alexmiller cljs should rewrite the clojure to cljs one automatically, but you would need it in a cljc file
2018:03:15 16:24:18 alexmiller iirc
2018:03:15 16:25:54 borkdude @gfredericks you could also put everything in a .cljc file with the help of macrovich… just wrap every macro in macros/deftime
2018:03:15 16:26:31 gfredericks @borkdude I have not tried it in cljs; schpec hasn't gotten a lot of attention. I had the silly idea that I could just make a generic bucket to put things in and then people would put things in it and use it, but I think people prefer to make libraries with more specific purposes
2018:03:15 16:26:48 gfredericks but I'm happy to incorporate any improvements you have, like making sure it works in cljs
2018:03:15 16:28:19 borkdude I’ll make an attempt
2018:03:15 16:30:54 mgrbyte hi, I'm wanting to make a model in spec of my data where the values for 2 keys in my map are dependant on another key's value. I've watched Stu's blog post about using gen/bind and gen/fmap; are there any other examples someone could point me to?
2018:03:15 16:51:32 anmonteiro @alexmiller we’re very confused about this behavior, wondering if it has come up before:
WARNING: No such namespace: clojure.spec.gen.alpha, could not locate clojure/spec/gen/alpha.cljs, clojure/spec/gen/alpha.cljc, or JavaScript source providing "clojure.spec.gen.alpha" at line 54 src-cljs/dre/components/widget.cljs
WARNING: Use of undeclared Var clojure.spec.gen.alpha/fmap at line 54 src-cljs/dre/components/widget.cljs
when I call the macro excl-keys
2018:03:15 17:05:11 borkdude I’ll try to change the require into:
2018:03:15 20:26:58 pablore I have a spec like this: (s/def (s/map-of ::id ::person) where ::id must be a property of ::person. Could anyone help me making the spec with s/with-gen so I can generate this map for testing?
2018:03:15 20:29:14 guy (s/def :entity.person/id string?)(s/def :entity/person (s/keys :req [:entity.person.id])
2018:03:15 20:29:20 guy like that do you mean?
2018:03:15 20:29:39 guy what do you mean by property of person?
2018:03:15 20:30:03 pablore yes
2018:03:15 20:30:14 pablore person is defined in terms of ::id
2018:03:15 20:31:24 pablore so when I generate a map of ids and persons, the map of id-persons must have sense
2018:03:15 20:41:28 pablore Indexed collections are structures I've come to make a lot of times doing clojure.
Ie, I rather do (get-in state [:people id :name]) than have to find the vector index every time.
2018:03:15 20:34:44 bbrinck @seancorfield @borkdude keep in mind that any attempt to analyze specs won’t work for multi-specs in CLJS, at least AFAICT
2018:03:15 20:35:38 bbrinck (which limits our ability to do things like look for undefined specs, or to automatically add checkers for missing values)
2018:03:15 20:42:10 alexmiller @pablore I don’t have time to give you a full answer, but I would start with a generator that creates a collection of ::person (which will have ::id’s in it). then gen/fmap over that generator to construct the map from the ids in the generated person entities
2018:03:15 21:01:49 pablore this would be a more generic version
2018:03:15 21:59:18 dottedmag How do I write a spec for [[1]]?
2018:03:15 22:00:23 dottedmag Obviously my data is more complicated than that — it's an number of vectors enclosed one in other. I'm trying to use spec for validation, so it's an external data type, not something I can change.
2018:03:15 22:00:23 alexmiller #{[[1]]}
2018:03:15 22:00:27 alexmiller :)
2018:03:15 22:01:28 alexmiller seriously though, do you know how many levels of vectors?
2018:03:15 22:01:54 dottedmag Two levels, it's a parsed CSV.
2018:03:15 22:02:38 dottedmag Not a "nice" uniform CSV, a complicated one. It's got a multiline header and a multiline footer, and empty strings delimiting data sections.
2018:03:15 22:03:45 alexmiller well, spec the inner levels, then spec the outer levels out of those :)
2018:03:15 22:04:53 dottedmag That's what I tried. s/cat on outer level, then s/cat on inner level, and first predicate in inner level tried to match the whole line, not the first element of an array.
2018:03:15 22:06:32 dottedmag Let me shorten it a bit
2018:03:15 22:10:05 dottedmag This ought to match [["Account Number"] ["1234123412341234"] [""]], but apparently the specs for sequences are flattened.
2018:03:15 22:13:39 dottedmag Ah, if I add (s/and (s/coll-of string? :kind vector) ...) to the ::header-info then in succeeds.
2018:03:15 22:13:44 dottedmag @alexmiller Thanks
2018:03:16 05:14:20 ikitommi Toyed more with spec coercion, with idea that coercion functions could be defined in spec meta-data so that the specs could remain self-contained . Currently requires spec-tools to work.
This didn't seems spec-specific at first but this test shows I'm probably misunderstanding something about s/def
2018:03:16 18:47:06 taylor @jjfine I think the problem is assert is returning its input if it’s valid, and bar is nil, and :pre is interpreting nil as an assertion failure
2018:03:16 18:47:31 taylor try {:pre [(s/valid? ::bar bar)]}
2018:03:16 18:47:31 jjfine oof
2018:03:16 18:48:24 jjfine thanks
2018:03:17 18:36:22 lgessler hi all, I have an xml schema describing a data format I want to translate into a clojure spec. i should be able to translate it by hand, but i was wondering if anyone knew of an easier way? the closest thing I found takes the XSD and then turns it into some kind of clojure schema, but it's not a spec and it didn't seem to produce the right output for my XSD https://github.com/kolov/xelery
2018:03:19 14:39:50 mgrbyte I have 3 specs describing a possible "identity" for an entity. I then have an "identity" spec along the lines of (s/def ::identity (s/or :domain/id1 ::id1 :domain/id2 ::id2 :domain/id3 ::id3). one of the specs can't use the built-in generator, how would I go about providing a custom generator that works for s/or? I've managed to write some custom generators before for maps, s/and specs, but can't get my head around how to start writing one for this s/or case, and google hasn't yielded anything so far. Does anyone have any pointers pls?
2018:03:19 14:48:14 mgrbyte I'm guessing I need to figure out how to use a combination of gen/tuple and gen/one-of 🤔
2018:03:19 15:04:11 bbrinck @mgrbyte Can you call with-gen inside the or?
2018:03:19 15:04:48 bbrinck Would that work for your case?
2018:03:19 16:16:49 mgrbyte @bbrinck Trying to keep generators in tests only, since I'm using minter.strgen/test.check stuff to help make the gen for "::id1" .
I've managed to get somewhere near what I need with (gen/one-of [gen1 gen2 gen3]), but as mentioned in the spec guide, need it's a good idea to generate s/conform like example for any s/or like specs: "For r specs that have a conformed value different than the original value (anything using s/or, s/cat, s/alt, etc) it can be useful to see a set of generated samples plus the result of conforming that sample data."
2018:03:19 16:17:37 mgrbyte So I'll need to interleave the specs and generators. but think that will do it
2018:03:19 16:31:44 bbrinck @mgrbyte I haven’t tried it out, but I believe you can use (require '[clojure.spec.gen.alpha :as gen]) and that will be OK in dev and test
2018:03:19 16:32:20 bbrinck (it lazy loads test.check when generator is called, not defined, but presumably you won’t be invoking generators outside of tests)
2018:03:19 16:33:31 bbrinck Might simplify your implementation, although I don’t know if you want to keep generators in test for dependency reasons, or for other reasons like attaching different generators at different tiems
2018:03:19 16:37:34 ikitommi Hi. Did someone make a stab at spec bijections? Any lessons learned? Bumped into a actual need: need to uncoerce path-parameters in reverse-routing into string based either on the value type or on the defined spec.
2018:03:19 16:38:03 ikitommi With spec, I guess we could put both the encode & decode functions into spec metadata, something like:
2018:03:19 16:57:42 gfredericks st/spec? does that exist?
2018:03:19 16:58:13 gfredericks I've been talking about bijections a lot; I was thinking of setting up a generic library for composing bijections, that's not spec-specific
2018:03:19 16:58:50 gfredericks if you have ideas about how to tie them with spec, that would be interesting
2018:03:19 16:59:11 ikitommi st/spec is from spec-tools, until spec meta-data appears.
2018:03:19 16:59:24 gfredericks that's a 3rd-party library?
2018:03:19 17:00:45 ikitommi does already the encode/coercion, could do the other way too.
2018:03:19 17:01:26 gfredericks my starting point is assuming that the most common use for bijections would be where you have some internal canonical representation of something, and you want to specify how to transform it to/from other "contexts", which could be things like json, jdbc, cvs, etc.
2018:03:19 17:01:31 ikitommi Generic library/foundation would be nice, as this is needed with Schema too.
2018:03:19 17:02:32 gfredericks so when deciding how to integrate that with spec, one question that would come up would be whether it's worth automating the process of transforming the specs themselves so that you get context-specific versons of the specs, or merely to automate the process of writing the functions that take values in each direction
2018:03:19 17:02:34 ikitommi yes, the name of the (`string` in ::encode/string) is the context name.
2018:03:19 17:03:49 gfredericks the downside of actually generating specs for each context is that it's more complicated and probably more work for the programmer
the upside is I think you can get more features, like validating closer to the edge, and giving validation errors based on that representation rather than the internal one
2018:03:19 17:04:35 ikitommi I tried the context-versions of spec, but coudn’t get it working.
2018:03:19 17:04:56 ikitommi failed with the qualified keys with s/keys.
2018:03:19 17:05:30 gfredericks s/keys is a big barrier; especially if you want to convert the keys to strings like in a json context
2018:03:19 17:05:55 ikitommi
(s/def ::id keyword?)
(s/keys :req [::id])
2018:03:19 17:06:30 ikitommi if you have that, you can’t have a different ::id as the full key names are exposed outwards.
2018:03:19 17:07:31 ikitommi why would the validation be closer to the edge?
2018:03:19 17:07:35 gfredericks yeah you'd need a new namespace for each context probably
2018:03:19 17:08:19 ikitommi … or a different registry.
2018:03:19 17:08:21 gfredericks e.g., if I write a public JSON API and biject my internal spec to a JSON version of that spec, I can theoretically validate the user input using the json spec, before trying to transform/coerce the data to the internal representation
2018:03:19 17:08:49 gfredericks you can also then provide validation errors to the user in a way that's specific to the json representation, rather than the less helpful internal representation
2018:03:19 17:30:04 gfredericks I've also wondered about separating the bijective parts from surjective parts (the times when you want to drop information, e.g., the order of keys in a query string)
2018:03:19 17:30:31 gfredericks if the surjections are formalized, you can use them to create generators of all the variant representations of something
2018:03:19 18:22:27 stathissideris super-interesting stuff, I think I should do some reading on this
2018:03:19 18:30:32 ikitommi thanks for the pointer! In my test, the transforming functions take two arguments: the spec and the value. This allows the transformation to extract information from the spec to do the transformation.
2018:03:19 18:31:19 ikitommi for example, stripping out non-defined keys from keys-spec needs the set of keys defined:
2018:03:19 18:43:05 gfredericks I used that for real code at my old job, and it worked pretty good but made the code difficult to read
2018:03:19 19:09:07 gfredericks Validating before converting isn't any extra walks compared to validating after converting I don't think
2018:03:19 16:59:59 drewverlee Knowing full well this doesn’t make sense. How would you write a spec for a collection of maps where each maps keys were
1) unique
2) represents a value themselves
e.g
[data, data, data, …] where data is something like {5 6}
2018:03:19 17:04:42 alexmiller well you would start with (s/coll-of (s/map-of int? int?))
2018:03:19 17:05:22 alexmiller each individual maps keys are (by definition) unique, but I assume you mean all of the keys over all the maps are unique
2018:03:19 17:05:37 alexmiller not sure what #2 means
2018:03:19 17:06:49 alexmiller for checking the uniqueness constraint, s/and another predicate that checks whatever property you need - something like #(apply distinct? (mapcat keys %))
2018:03:19 17:22:36 drewverlee > each individual maps keys are (by definition) unique, but I assume you mean all of the keys over all the maps are unique
correct
2018:03:19 18:38:16 ghadi you can't have duplicate keys in a map @drewverlee
2018:03:19 18:39:02 ghadi oh I didn't see the collection of maps requirement. yeah, s/and is your friend here
2018:03:19 18:52:22 roklenarcic I'm trying to write my own Spec protocol implementation. Is there somewhere where Spec and Specize protocols' contracts are explained?
2018:03:19 19:05:17 alexmiller no, because they are likely to change
2018:03:19 19:08:51 misha @mgrbyte you can override generator just for ::id1 during exercise time
2018:03:19 19:14:16 borkdude hmm yeah… for me too when I isolate this
2018:03:19 19:16:23 borkdude works now!
2018:03:19 19:16:30 borkdude I guess some reloading glitch
2018:03:19 21:18:58 cch1 Is there an idiomatic way of expressing a spec for “anything” other than (constantly true)?
2018:03:19 21:19:35 cch1 I’m trying to define a spec for a value that must ultimately be usable as a key in an associative structure.
2018:03:19 21:19:57 cch1 In Clojure, I think that is pretty much everything.
2018:03:19 21:20:54 bronsa any? is what you want
2018:03:19 23:18:02 camachom I'm trying to pass a vector to s/keys, but it keeps failing. Works great if I just write it out though. Something like this:
2018:03:19 23:18:23 camachom any ideas as to what i'm doing wrong?
2018:03:19 23:28:30 alexisvincent Hi guys, got a bit of a modelling problem. How do you build up specs with context dependent information.
E.g. Say you’re building a space for card details, say a map of the keys :card/number:card/expiry:card/name:card/cvv.
Now we want this to be a nice composable spec called :card/details. But in some contexts :card/cvv is required and in others it isnt. So The :card/details spec is really a partial spec with requiredness of :card/cvv dependent of context.
We might then say make two spec’s :card/details and :card/full-details. But this isn’t really scalable. It means whenever we have branching up the composition tree we need to rename things for all cases, giving an exponential explosion. Is there any way to build ‘partial’ specs where specs higher up the composition tree provide more or overriding data for specs they depend on.
Similar to dependent types.
2018:03:20 00:12:39 bbrinck However, this adds a dependency on test.check to use expound at dev time. Would you prefer to have this be a hard dependency OR would you prefer expound to disable this feature if you don’t have test.check available?
2018:03:20 00:24:46 seancorfield @mcama200 spec/keys is a macro so it takes literal code forms, not variables.
2018:03:20 00:25:55 seancorfield What you can do instead is to use the literal vector in s/keys and then call s/form (I think?) on the spec itself and walk that to get the list of required keys back out of a spec definition.
2018:03:20 00:35:50 camachom awesome, thanks for the help!
2018:03:20 00:29:25 seancorfield @alexisvincent The short answer is: specs are currently pretty static so you have to have multiple specs -- or specify that :card/cvv is :opt in the base spec and then wrap it in another spec that uses s/and and a predicate that "requires" that key be present.
2018:03:20 09:12:11 borkdude I guess spec could eat its own dog food?
boot.user=> (s/fdef foo (s/cat :a1 string?))
java.lang.IllegalArgumentException: No value supplied for key: (s/cat :a1 string?)
clojure.lang.Compiler$CompilerException: java.lang.IllegalArgumentException: No value supplied for key: (s/cat :a1 string?), compiling:(null:1:1)
2018:03:20 13:12:22 alexmiller Actually it can’t, unless you enjoy infinite recursion
2018:03:20 10:47:31 mpenet been there done that: my personal favorite is (s/def foo any?) or (s/fdef ::foo :args (s/cat))
2018:03:20 10:48:34 mpenet both silently fail, which makes it even better
2018:03:20 10:57:54 alexisvincent @seancorfield Thanks for the answer, the s/and is neat, and will help reduce branching factor, but I don’t think it’s a long term behaviour. I can use functions that return specs, but loose out on spec’s repository. Do you know if dynamism will be added to named specs in the future, or if it even needs to be?
2018:03:20 11:06:30 borkdude I have the current spec (simplified):
2018:03:20 14:44:23 acron Most depressing error ever.
2018:03:20 14:50:25 alexmiller agreed :)
2018:03:20 14:51:44 alexmiller unfortunately, the current linkage between spec and test.check prevents good error reporting on where this is happening. there are plans to make this better, but it requires changes in both projects.
2018:03:20 15:26:03 gfredericks @alexmiller I'm curious how what you just described relates to an earlier comment that spec can't enscricten its version requirements of test.check
2018:03:20 15:26:42 alexmiller enscricten = ?
2018:03:20 15:28:44 danielneal make more scrict
2018:03:20 15:29:19 alexmiller and scrict = ?
2018:03:20 15:33:36 danielneal opposite of pecmissive
2018:03:20 17:59:13 misha @alexisvincent define all your fields’ specs, and then have several s/keys specs for different combinations of those fields. Then you might want to have the card spec, which is s/or around those s/keys
2018:03:20 18:05:12 misha Remember, you can have keys with different namespaces in the same map/object, so don’t paint yourself into corner with unified namespace “:card” when some attributes might be better named, like :amex.card/number, because it’ll have different spec and generator.
2018:03:21 00:45:17 gfredericks @alexmiller sorry, meaning that spec would, e.g., require test.check >= 0.10.0
2018:03:21 00:45:37 gfredericks you made a comment a few weeks back that implied that wouldn't be acceptable
2018:03:21 00:45:45 alexisvincent @misha 🙂 Thanks. Im just concerned about the name explosion when composing up the tree
2018:03:21 00:53:15 alexmiller @gfredericks I wouldn’t make a range version but we could certainly bump the requirement
2018:03:21 00:53:46 gfredericks the difference between those two things is whether a max version is specified?
2018:03:21 00:53:48 alexmiller I just don’t like version ranges
2018:03:21 00:54:07 seancorfield No one likes version ranges 🙂
2018:03:21 00:54:31 alexmiller The only people that like version ranges are people that haven’t used them :)
2018:03:21 00:58:31 gfredericks :+1: thanks
2018:03:21 01:00:06 kahunamoore I'm looking to do some visualization of specs and by default would think using the reader would be the most straight forward way to consume/read the specs. Is there a better way? A library perhaps that produces a higher level representation? Thoughts?
2018:03:21 02:13:32 andy.fingerhut @kahunamoore I have not used this tool, and not sure if it is what you are looking for, but it sounds at least related: https://github.com/jebberjeb/specviz
2018:03:21 14:00:12 hkjels the above obviously does not work
2018:03:21 14:00:58 hkjels I could use this and digg into the meta, but I’d prefer not to
2018:03:21 14:05:45 alexmiller what problem are you trying to solve?
2018:03:21 14:14:24 hkjels I output form-controllers based on the type of a specification. So for simple spec definitions like (spec/def ::placeholder string?) I can output a textfield because I know it’s a string with (type the-def), but for definitions that use with-gen; type returns nil
2018:03:21 14:24:30 bbrinck @hkjels Can you talk more about how you determine the type in the simple case? Are you inspecting the spec itself, or an example generated value?
2018:03:21 14:26:17 bbrinck Also, that generator above doesn’t generate valid values: (fn? nil) is false
2018:03:21 14:27:17 bbrinck If you want a “maybe-fn”, perhaps s/nilable would help?
2018:03:21 14:27:30 alexmiller he’s returning (constantly nil), which is a function
2018:03:21 14:27:37 bbrinck ah
2018:03:21 14:27:40 bbrinck doh
2018:03:21 14:28:09 alexmiller I feel like you should still be driving from the spec, not the generator
2018:03:21 14:29:08 hkjels yeah, not sure how though
2018:03:21 14:29:15 alexmiller (you should also be using ifn?, not fn? btw)
2018:03:21 14:29:24 bbrinck Agreed, @hkjels if I were trying to generate forms based on the spec, I’d look at introspecting the specs using s/form. Obviously it’s be hard with all possible specs
2018:03:21 14:29:50 bbrinck but if you have a subset of forms in your app, you could reasonably inspect them and generate the cases that interest you
2018:03:21 14:30:10 alexmiller well in this case, s/form doesn’t currently work on predicates
2018:03:21 14:30:20 alexmiller (pending patch to fix that)
2018:03:21 14:30:48 hkjels ahh, nice
2018:03:21 14:30:52 bbrinck But s/form returns something sensible for ::placeholder, yes?
2018:03:21 14:30:58 bbrinck And ::maybe-fn
2018:03:21 14:31:29 bbrinck You could parse those (probably using specs!!) and then build cases based off that
2018:03:21 14:31:54 alexmiller you could use spec specs for that :) (CLJ-2112)
2018:03:21 14:31:59 bbrinck ftw
2018:03:21 14:32:00 bbrinck 🙂
2018:03:21 14:34:00 hkjels 🙂
2018:03:21 14:34:33 hkjels here you see what I’m trying to achieve for all kinds of elements
2018:03:21 14:35:03 hkjels it currently works for buttons, since there’s no atoms etc in use
2018:03:21 14:36:39 hkjels Ohh, shoot. I got to run
2018:03:21 17:30:44 kahunamoore @andy.fingerhut Thanks! I remember reading/hearing about it but Google failed me - too many spec tutorials, docs and other misses.
2018:03:21 19:37:56 hkjels ohh, that nonconforming is just some cruft left there from trial and error
2018:03:21 23:43:27 gfredericks @hkjels would (spec/* (s/alt string? ::hiccup)) work?
2018:03:21 23:52:38 ghadi Need to label the branches @gfredericks ^
2018:03:21 23:52:58 gfredericks yeah that sounds truthful
2018:03:22 07:42:19 andy.fingerhut So Stuart Halloway gave a talk on Clojure spec at Strange Loop 2016 that I am transcribing now (mostly done), where he said this: 'So one of the things that is fun about developing with spec is: as soon as you have made a spec, and it can even be a very rudimentary one, you can then say "give me example data" or "give me example functions that do this".'
2018:03:22 07:43:40 andy.fingerhut The part that I am wondering how to do is the part where he says "give me example functions that do this". Does anyone know if that is possible, and if so, has examples of doing that?
2018:03:22 12:44:15 alexmiller He was probably referring to the stub capabilities of instrument
2018:03:22 11:44:12 gfredericks fspecs can be generated I think
2018:03:22 11:49:33 taylor is it just making a function that ignores its inputs and using the :ret spec to generate return values?
2018:03:22 11:50:34 taylor > fspecs can generate functions that validate the arguments and fabricate a return value compliant with the :ret spec, ignoring the :fn spec if present.
2018:03:22 11:51:26 guy Well you could try doing (my-fn "string") and see if it complains
2018:03:22 11:51:48 guy Because you are giving it an int right?
2018:03:22 11:51:54 taylor it does validate the inputs, I asked before I read the docstring 🙂
2018:03:22 11:51:58 guy ah
2018:03:22 11:52:13 guy 👍
2018:03:22 11:59:09 gfredericks yes the return is unrelated to the input I think
2018:03:22 11:59:28 gfredericks I made a ticket many decades ago to discuss whether it should at least be a pure function
2018:03:22 11:59:38 gfredericks it's also currently independent of the test.check seed I think
2018:03:22 12:00:59 guy You can use orchestra to make it related with the :fn i think?
2018:03:22 12:03:06 guy
2018:03:22 12:21:18 taylor I think that only applies to instrumentation
2018:03:22 12:55:55 danielcompton I think this is a really dumb question, but how can I detect which branches were taken/captured in a s/cat?
2018:03:22 12:56:34 gfredericks doesn't the conformed value tell you that?
2018:03:22 12:57:36 danielcompton yeah I can see it in the data, I was just wondering if there was a function you could call to get back which parts matched
2018:03:22 12:58:12 danielcompton the context is that I'm trying to parse defn forms and reassemble them with tracing wrapped around parts of the function
2018:03:22 12:58:55 danielcompton and the function specs have quite nested data that is a little bit annoying to pull out
2018:03:22 13:23:57 danielcompton So I end up with this disgusting code:
2018:03:22 13:24:26 danielcompton was wondering if there's a pattern for doing this in a cleaner way
2018:03:22 13:26:22 mpenet small critique, seems like the if name bits could be done inside the (fn ... part to avoid repeating the whole fn expression
2018:03:22 13:26:32 mpenet both times
2018:03:22 13:26:45 mpenet or I might be missing something
2018:03:22 13:27:01 danielcompton but if there is no name then I'll end up with (fn nil [] ...)
2018:03:22 13:27:48 mpenet oh right, I didn't pay attention
2018:03:22 13:28:12 danielcompton Am I possibly approaching this whole thing in the wrong direction?
2018:03:23 19:23:07 bbrinck The trick is to modify the conformed value and then call unform to go back to valid clojure code
2018:03:23 19:42:04 bbrinck Even if I deleted the docstrings, examples, and workaround for the spec bug, I think my code would longer than your solution, but here’s an example of the approach described in that link https://gist.github.com/bhb/128bf97619e83541a8adda7094bc370d
2018:03:23 20:56:55 danielcompton Thanks, that’s a very clever way to do it
2018:03:22 15:56:39 borkdude By mistake, but it surprised me:
(s/def :dropdowns/options (s/coll-of :dropdowns/options))
(s/valid? :dropdowns/options []) ;; true, ok, I can get that
(s/valid? :dropdowns/options [[]]) ;; true, uuh...
2018:03:22 15:58:18 guy 🤔
2018:03:22 15:58:22 alexmiller coll-of doesn’t limit cardinality by default
2018:03:22 15:58:30 alexmiller 0 size is valid
2018:03:22 15:58:38 borkdude yes, I get that
2018:03:22 15:58:49 alexmiller Use :count, :min-count etc if needed
2018:03:22 15:59:03 guy what is your spec for :dropdown/option ? suprised it allowed [[]]
2018:03:22 15:59:17 borkdude no, you see, it’s recursive
2018:03:22 15:59:24 guy oh right
2018:03:22 15:59:46 guy Sorry missed that
2018:03:22 16:00:18 borkdude I missed that too, but then I was surprised about this behavior, because spec will allow arbitrary nesting of empty colls this way
2018:03:22 16:00:43 borkdude I don’t know if set theory is happy with this
2018:03:22 16:01:47 borkdude the way to justify: an empty coll is a valid :foo, a collection of empy collections is a collection of valid :foo, hence an arbitrarily nested empty coll is a valid :foo
2018:03:22 16:02:29 borkdude so actually this is a spec for arbitrarily nested empty colls
2018:03:22 16:05:21 mpenet if you re thinking about monoid instances, it's perfectly valid imho
2018:03:22 16:06:03 borkdude well if you think about types, a [a] isn’t the same as [[a]]
2018:03:22 16:06:18 mpenet yeah but the mempty of [a] is []
2018:03:22 16:06:49 mpenet but yes, no type annotation to validate that [] is indeed potentially a coll of foo
2018:03:22 16:06:57 mpenet but other than that it's ok
2018:03:22 16:07:40 borkdude I guess
2018:03:22 16:08:48 borkdude it’s an unexpected way to describe all possible sets of empty sets
2018:03:22 16:08:54 borkdude (s/valid? ::empty-nested [[] []]) is also true of course
2018:03:22 16:08:57 mpenet spec cannot go that far. same problem with (s/def ::first-name string?) (s/def ::last-name string?)
2018:03:22 16:09:34 mpenet using ::first-name instead of ::last-name in an fdef args signature will be silent, in haskell it would scream at you
2018:03:22 16:09:50 mpenet same shape, not necessary same meaning
2018:03:22 16:10:27 borkdude yes, I guess that makes sense. it’s enabled things too
2018:03:22 16:10:54 mpenet no such thing as newtype or some kind of wrapper for validation
2018:03:22 16:11:07 mpenet yes, it's all trade-offs I guess
2018:03:22 16:13:32 borkdude How would you write this:
dropdown-option/id or dropdown/option-id . You can only namespace one level
2018:03:22 16:14:45 guy >dropdown-option/id
Says to me, thats the id of the dropdown option.
>dropdown/option-id
Says to me, thats the option-id of the dropdown
2018:03:22 16:14:53 mpenet I use a little helper and create a new namespace for it
2018:03:22 16:15:22 mpenet then ::dropdown.option/id
2018:03:22 16:15:44 mpenet That's what I found to be the least horrible so far.
2018:03:22 16:15:49 borkdude I could just use the dot anyway right? without creating a namespace?
2018:03:22 16:16:07 guy Yeah
2018:03:22 16:16:19 mpenet you can use a keyword with a dot yes
2018:03:22 16:16:25 borkdude ok, I’ll do that then
2018:03:22 21:50:38 roman01la Is it possible that both s/conform and s/valid are failing, but s/explain returns Success!?
2018:03:22 21:51:01 roman01la Not doing anything fancy. Clojure 1.9.0
2018:03:22 21:53:25 roman01la ok, nevermind, forgot to reload ns
2018:03:22 23:52:15 alexmiller any case of that is a bug
2018:03:23 01:22:56 pablore Are there any differences in using spec in clojure and clojurescript? Theres a cljs.spec.alpha namespace
2018:03:23 01:23:48 pablore Can I define specs in a .cljc file and use them in both clojure and clojurescript?
2018:03:23 01:30:08 seancorfield I believe that ClojureScript is supposed to do an automatic mapping from clojure.* namespaces to cljs.* namespaces so I would expect you could just write .cljc files with a require of clojure.spec.alpha and it should just work...
2018:03:23 01:30:27 seancorfield (caveat: I don't do anything with cljs so I might be talking nonsense)
2018:03:23 01:34:16 benzap I believe that is correct, I just pull in clojure.spec.alpha, and it works in clj, cljs, and cljc
2018:03:23 02:36:21 alexmiller that’s the idea
2018:03:23 16:29:03 borkdude Is there a way to not have this conform in the two options but just pass through as it is?
was what I was suggesting but not sure if that’s a terrible idea or not
2018:03:23 16:37:51 alexmiller You can wrap s/nonconforming around the or
2018:03:23 16:37:58 borkdude awesome
2018:03:23 16:38:13 alexmiller Currently that’s undocumented
2018:03:23 16:38:31 alexmiller I think we will ultimately either doc it or make a nonconforming or
2018:03:23 16:38:36 borkdude hmm, not in clojurescript yet probably?
2018:03:23 16:38:50 borkdude oh it is
2018:03:23 16:40:06 borkdude works great, thanks
2018:03:26 11:37:46 borkdude is it possible to instrument an anonymous function?
2018:03:26 11:41:25 mpenet fspec?
2018:03:26 11:42:45 borkdude how? say I have a re-frame event:
(reg-event-fx ::foo
(fn [...] ...))
2018:03:26 11:43:00 borkdude I could pull the fn out and name it, but that gets tedious
2018:03:26 11:44:02 mpenet I think the approach would be to spec reg-event-fx and have the second arg speced via fspec
2018:03:26 11:44:39 mpenet otherwise you can/need to add s/asserts if you want to have it "contained" at the anonymous function level
2018:03:26 11:46:07 borkdude you can’t do that because the function reg-event-fx receives is a different one every time. I want to be specific about the concrete function, not the function argument of reg-event-fx
2018:03:26 11:46:14 borkdude I think for now it’s easiest to name it then
2018:03:26 12:03:42 borkdude hmm:
cljs.user=> (def param-keys [:query/query
#_=> :query/source
#_=> :vocab/filter
#_=> :dict/guid
#_=> :dict/labels?])
#'cljs.user/param-keys
cljs.user=> (s/def ::params
#_=> (s/keys :opt
#_=> param-keys))
^
param-keys is not ISeqable at line 2
2018:03:26 12:04:30 borkdude it only expects me to pass in literals? that’s disappointing
2018:03:26 12:04:33 mpenet you need to use eval to pass params-keys, s/keys is a macro
2018:03:26 12:04:36 mpenet yeah
2018:03:26 12:05:18 mpenet Hopefully it something improved in the next spec iteration, it's a common problem
2018:03:26 12:06:12 mpenet s/merge (or s/and) can sometimes help in these case too
2018:03:26 12:07:17 borkdude how?
2018:03:26 12:08:31 mpenet to compose multiple sets of s/keys
2018:03:26 12:09:02 mpenet hence the "sometimes" it does not always work
2018:03:26 12:15:00 borkdude Is this a valid way of saying I want either :dict/guid or :query/query + these optional keys?
2018:03:26 12:33:12 alexmiller if you implement xor as a function and use there, it will work. but I make no guarantees that that will continue to work in the future.
2018:03:26 12:41:52 borkdude this is undocumented right
2018:03:26 12:53:22 alexmiller yes and not guaranteed to work in the future
2018:03:26 14:15:12 borkdude Are there parts that we can rely on, since spec is still in alpha?
2018:03:26 14:17:07 borkdude Don’t mind breakage btw if it’s for the better
2018:03:26 14:19:58 alexmiller generally, it’s better to rely on the documented usage than the undocumented usage
2018:03:26 14:32:53 borkdude I’m emitting a warning in a s/and conformer function.
I see this message twice. Is this because spec does something like “if not valid run it again for the explanation”?
2018:03:26 14:35:12 borkdude The error that spec returns itself is a bit unsatisfying here, that’s why I want to print extra info
2018:03:26 14:38:38 alexmiller there are no guarantees on the number of times a conformer function might be run (in a regex with branches, it may be run many times). I’m not sure why it’s run more than once in this particular case, but it’s possible that your explanation is correct.
2018:03:26 14:38:53 borkdude ok no problem at all, just wanted to know
2018:03:26 14:39:11 borkdude maybe I could write this spec a little better, so spec provides what keys are unexpected though
2018:03:26 14:39:31 borkdude but this goes against the default of spec where extra keys are just allowed
2018:03:27 07:31:47 borkdude I think it doesn’t know how to generate a ratom from a predicate.
2018:03:27 08:20:14 hkjels That springs another question though. How do I express that?
2018:03:27 07:33:12 borkdude Maybe you should try making ::ratom with it’s own with-gen so that this with-gen also determines the contents of the ratom.
2018:03:27 08:01:06 hkjels ahh, ofcourse. Thank you @borkdude
2018:03:27 08:20:36 hkjels fails with the same message
2018:03:27 08:32:26 borkdude @hkjels I made a small example in JVM Clojure which should be straightforward to port to cljs:
I’m not sure if there’s a better way as I’ve not used this part of spec much yet.
2018:03:27 09:11:08 hkjels hey, that works
2018:03:27 09:11:39 hkjels hmm, weird that (s/coll-of ::item :kind? set) does not return a set
2018:03:27 09:11:47 hkjels that must be a bug right?
2018:03:27 09:12:13 borkdude Don’t know. I’m sure Alex will reply later today
2018:03:27 09:13:22 hkjels Thank you!
2018:03:27 09:59:35 minimal @hkjels use a predicate like everything else: (s/coll-of ::item :kind set?)
2018:03:27 10:00:23 hkjels ahh, ok
2018:03:27 10:00:26 minimal and it’s :kind, not :kind?:
2018:03:27 10:00:49 hkjels ahh, that explains why it didn’t barf
2018:03:27 10:00:52 hkjels 🙂
2018:03:27 10:01:16 minimal yeah, would be nice if it told you
2018:03:27 10:10:26 borkdude ah I wasn’t awake enough to spot those typos 🙂
2018:03:27 14:39:18 borkdude I like it how spec lets you specify things gradually or as fine-grained as you want
2018:03:27 19:34:15 manutter51 My google-fu is failing me: I want to write a spec for a function that takes an atom (actually a Reagent atom). Is there a way to do that?
2018:03:27 20:24:07 manutter51 Thanks much, I’ll see if I can get that to work. The tricky part is this all has to work in CLJS, but I want to match both regular atoms and Reagent atoms so I can pass in test dummies
2018:03:27 20:24:37 borkdude then IAtom seems the right abstraction
2018:03:27 20:24:44 bbrinck Arguably, testing anything more than the class isn’t very useful, but YMMV
2018:03:27 20:25:08 bbrinck (I mean, testing the interface may be useful too, but the contents, maybe not so much)
2018:03:27 20:25:15 borkdude yes, you can just as well create ratom dummies, so it’s a good question why you want to do that
2018:03:27 20:26:27 bbrinck The advice I remember is that spec is mostly useful for testing values. Testing references is likely less useful
2018:03:27 20:27:10 bbrinck since testing the reference at any point in time doesn’t say much about what it will become at any point in the future
2018:03:27 20:28:06 bbrinck it might be more useful to spec the functions that mutate that reference, or, say, use spec in a validator
2018:03:27 20:28:20 manutter51 Yeah, that’s a good point, I could just use ratoms everywhere
2018:03:27 20:28:36 manutter51 That’s probably the best solution — tks much
2018:03:28 02:56:39 leontalbot Hello! I am looking for a way to give free pass empty string values when provided as optional
2018:03:28 02:58:18 leontalbot I could check for url, but if empty string, since it is in opt-un, let it be valid
2018:03:28 02:58:55 leontalbot (but if was in :req-un, then should apply full ::url spec)
2018:03:28 02:59:03 leontalbot is there a way to achieve that?
2018:03:28 02:59:50 leontalbot It does work for nil but I’d like it to work for “” too
2018:03:28 02:59:53 leontalbot thanks!
2018:03:28 03:14:06 seancorfield @leontalbot I think what I'd do there is use :opt/url as the :opt-un key and define it as a spec like
(s/def :opt/url (s/or :url ::url :empty empty?))
which will allow it to be the full ::url string spec or nil or ""
2018:03:28 03:15:00 seancorfield (or however you want empty values to be spec'd... maybe #{nil ""})
2018:03:28 03:19:13 leontalbot cool! Thanks @seancorfield. Is this good too to you:
2018:03:28 03:19:22 leontalbot
(defn valid-map?
“dissoc keys with empty vals to let optional keys pass, then validate...”
[spec m]
(s/valid? spec (apply dissoc m (for [[k v] m :when (#{“” nil} v)] k))))
2018:03:28 03:21:25 leontalbot probably less general
2018:03:28 03:34:57 seancorfield @leontalbot I wouldn't use that approach because now you have a custom function, not just a spec. You can't conform or explain with that approach.
2018:03:28 03:35:28 seancorfield It also wouldn't work for specs that have keys whose values can be nil or "" but are still required.
2018:03:28 03:37:16 seancorfield FWIW, we have exactly this situation at work and the approach I suggested is basically what we do.
2018:03:28 03:38:36 leontalbot Thank you so much @seancorfield!
2018:03:28 03:40:09 seancorfield Spec is pretty flexible and powerful -- and the ability to use multiple qualified keywords for "similar" unqualified keys specs lets you deal with a lot of context.
2018:03:28 03:41:17 seancorfield Another aspect to consider is having different versions of specs at different "layers" in your application. For example, we have API-level specs (which deal with strings mostly) and we have domain-level specs (and we have a few DB-level specs as well).
2018:03:28 03:41:53 leontalbot makes sense
2018:03:28 03:41:59 leontalbot and one last question:
2018:03:28 03:42:38 leontalbot why :opt/url and not :url/opt?
2018:03:28 03:45:01 leontalbot @seancorfield ^
2018:03:28 03:45:04 leontalbot thanks!
2018:03:28 03:46:01 leontalbot Yeah I guess this is to map this hierarchy:
2018:03:28 03:46:11 leontalbot
(s/keys :req-un [::id] :opt-un [::url]))
2018:03:28 03:46:24 leontalbot yeah…
2018:03:28 04:45:46 seancorfield Yeah, if the unqualified key is :url then the options are :<something>/url
2018:03:28 13:46:36 leontalbot Wanted to get end-user error message from spec. Useful for webform field validation.
2018:03:28 13:46:48 leontalbot Found Phrase library.
2018:03:28 13:47:34 leontalbot Seemed a bit overkill though. As I just wanted is attach an error text to a spec.
2018:03:28 14:37:39 bbrinck @leontalbot Would this work?
(expound/def :user/name string? "should be a valid name")
(defn msg [spec val]
(if (s/valid? spec val)
nil
(expound/error-message spec)))
(msg :user/name "John") ; => nil
(msg :user/name :John) ; => "should be a valid name"
2018:03:28 14:42:29 leontalbot Hey! Thanks for answering @bbrinck! Fantastic! Thanks!
2018:03:28 14:42:52 bbrinck np, let me know if you have any other questions about expound
2018:03:28 14:43:10 leontalbot Ok! Nice lib btw!
2018:03:28 14:43:18 bbrinck thanks! 🙂
2018:03:28 17:43:08 borkdude @bbrinck Sometimes I get an error message from expound when it complains about not being able to render an error (exact error message I don’t have handy). Wouldn’t it be better in that case to print the vanilla spec error instead of only the expound error?
2018:03:28 17:55:46 bbrinck @borkdude Is this a bug in expound? Or a case where your spec has conformers, perhaps?
2018:03:28 17:55:55 borkdude the latter
2018:03:28 17:56:37 bbrinck Gotcha. Yeah, in that case, it might very well make sense to just print the default error. Let me think about that. Thanks for the idea!
2018:03:28 17:57:24 bbrinck Although if you’re using conformers, the vanilla spec error may not be very helpful either 🙂
2018:03:28 17:57:32 bbrinck But still better than nothing, for sure
2018:03:28 17:58:03 borkdude Usually when I run into this, I turn off expound, re-run the code, inspect the error, fix it, and enable expound again.
2018:03:28 17:58:25 bbrinck Agreed, that’s a pain. I’ll fix it.
2018:03:28 18:08:33 seancorfield @bbrinck FWIW, that was why we tried and then stopped using Expound -- we have several conformers in our specs.
2018:03:28 18:18:50 bbrinck @seancorfield Would it work in your case to just fallback to s/explain?
2018:03:28 18:19:13 bbrinck Depending on how many conformers you have, I guess at some point you would rarely see an expound error
2018:03:28 18:20:01 bbrinck So the fix @borkdude suggested is a good idea, but it’s mostly useful for projects that have a relatively small number of conformers compared to the total usage of spec
2018:03:28 18:20:20 bbrinck @seancorfield FWIW, apparently pinpointer works with conformed values
2018:03:28 18:25:28 seancorfield @bbrinck Well, the main cases where we wanted Expound's better messages were all conformer-specs, so we just fell back to explain ourselves 🙂
2018:03:28 18:25:51 seancorfield I'll take a look at pinpointer -- I hadn't heard of that.
2018:03:28 20:40:26 lilactown has anyone used spec-tools to generate swagger objects?
2018:03:28 20:40:45 lilactown I'm having trouble with getting the response schema's to generate correctly
2018:03:29 03:47:33 ikitommi @lilactown one of the authors here and just integrated that to reitit routing library and worked ok. Porting Schema=>Swagger into the same model, which will cause small changes to spec-side to get unified tranformation model for both. How can I help?
2018:03:29 03:52:08 ikitommi Still need to backport support for json schema refs for recursive references, also will change the apis a bit. e.g. tranformations return both the transformed swagger schemas and the named schema definitions they refer to.
2018:03:29 14:17:04 lilactown I was trying to use the swagger-spec fn with an endpoint that had a response map like in the README:
2018:03:29 14:18:54 lilactown however, this was getting converted to something like :x-spec-tools.swagger.core/spec :user/user
2018:03:29 14:19:18 lilactown instead of expanding the ::user spec into the :schema key
2018:03:29 18:23:21 ikitommi @lilactown sorry, thread got buried. You should raise the ::swagger/responses one level up. So, ::swagger/responses {200 {:schema ::user}}
2018:03:29 18:24:02 ikitommi should add specs to the input so it would tell what happened.
2018:03:29 18:55:35 lilactown which AFAICT does not work
2018:03:29 20:39:25 ikitommi Oh, good catch! I was too blind to see that it's not correct. Could you write an issue, need to fix either the code or the sample in README.
2018:03:29 16:38:40 jimbob how do you guys organize your specs? Are all of your specs in a top level spec directory? or do you also mix and match spec assertions and definitions in core namespaces
2018:03:29 17:45:43 taylor I usually fdef next to the function, but put other specs in their own namespaces
2018:03:29 16:38:48 jimbob any tips?
2018:03:29 17:44:34 borkdude When I write an fdef I place it above the function
2018:03:29 18:56:17 seancorfield @ben.borders It depends. If I need code to work with pre-1.9 Clojure, I put all the specs in a separate namespace. Otherwise, if I'm spec'ing data I tend to put the specs in their own namespace (along with predicates and closely related helper functions), else if I'm spec'ing functions I tend to put those directly above the function they are for.
2018:03:29 19:15:45 borkdude @ben.borders Just moved the fdef + related specs to a widget/spec.cljs namespace because it became too large 😉
2018:03:30 06:41:32 robert-stuttaford likely a common question, and also likely not something spec is suited for, but i’m curious even so - i have a map with start and end dates. is there a spec pattern for declaring a relationship between those two values, i.e. one must be larger than the other? one structure that occurs is spec/fdef’s :ret, but i’m wondering if perhaps the spec api has something else like this?
2018:03:30 09:55:08 borkdude @robert-stuttaford you can use s/and + a conformer function
where < is the comparison function of your choice.
2018:03:30 10:08:00 robert-stuttaford wonderful, thank you! that seems ridiculously simple, in hindsight. like, ‘how did i not see this’ simple
2018:03:30 10:10:56 robert-stuttaford @borkdude i guess i’d have to write my own generator too then
2018:03:30 10:11:54 borkdude Don’t know if the default generator generates enough samples where the conformer can strip away the invalid ones. Not as efficient as writing your own, but it could work
2018:03:30 10:15:11 borkdude it’s about 50% chance for each sample
2018:03:30 10:17:20 robert-stuttaford right
2018:03:30 10:22:41 borkdude when you write an fdef in another namespace than the function, require the namespace where the function lives? or don’t and just fully qualify the symbol in fdef? hmm
2018:03:30 10:23:25 borkdude Right now I have an init namespace that just requires them all, so no cyclic dependencies and I can just fully qualify in fdef
2018:03:30 10:24:03 robert-stuttaford what’s causing you to want to keep the fdef separate, @borkdude?
2018:03:30 10:24:21 borkdude because it’s more than 100 lines
2018:03:30 10:26:16 dominicm I feel like I remember there being some kind of way to add conformers to specs "locally"? Is that a correct memory?
2018:03:30 10:26:32 robert-stuttaford ah 🙂Qualifies fn-sym with resolve, or using *ns* if no resolution found. seems to suggest that fully-qualified is fine, and probably better for discoverability
2018:03:30 10:28:27 borkdude I have an interesting problem:
(s/def ::selection
(s/nilable
(s/and
(s/keys :req-un [:selection/options
:selection/id]
:opt-un [:selection/default-option
:selection/type])
;; default option must be one of the option ids
(fn [dropdown]
(if-let [opt (:default-option dropdown)]
(contains?
(set (map :id (:options dropdown)))
opt)
true)))))
(s/def ::dropdown ::selection)
I want the id in dropdown to be optional… Maybe I should make an extra spec without the required id and then make the id in selection required with s/and?
2018:03:30 10:31:11 borkdude Like this:
(s/def ::selection
(s/nilable
(s/and ::selection*
(s/keys :req-un [:selection/id]))))
(s/def ::selection*
(s/nilable
(s/and
(s/keys :req-un [:selection/options]
:opt-un [:selection/default-option
:selection/type])
;; default option must be one of the option ids
(fn [dropdown]
(if-let [opt (:default-option dropdown)]
(contains?
(set (map :id (:options dropdown)))
opt)
true)))))
(s/def ::dropdown ::selection*)
2018:03:30 10:34:51 borkdude Seems to work
2018:03:30 11:51:45 robert-stuttaford what’s the blessed method for checking whether a spec is registered? s/spec? seems to be for something else
2018:03:30 11:52:54 robert-stuttaford aha s/get-spec
2018:03:30 11:53:08 borkdude @robert-stuttaford brute force method: use a println in a conformer 😛
2018:03:30 11:53:25 borkdude or just make an obvious mistake and if no exception, then no 😉
2018:03:30 11:54:11 robert-stuttaford -grin- its for datomic attrs. the code using the spec can’t assume a spec is registered; it has to check first before it attempts to use it to validate.
2018:03:30 11:54:17 robert-stuttaford get-spec works
2018:03:30 11:59:30 robert-stuttaford grr, i’m having to wrap my defmulti with a normal defn so that i can instrument it. otherwise defmethods defined after instrumentation fail
2018:03:30 11:59:52 borkdude Funny that s/spec? doesn’t return a boolean
2018:03:30 12:00:36 borkdude don’t all new fdefs after instrumentation fail?
2018:03:30 12:00:48 borkdude in the sense that they aren’t instrumented yet
2018:03:30 13:01:31 alexmiller Yes, although I wouldn’t call that a fail
2018:03:30 13:05:34 borkdude more appropriate: not yet in effect
2018:03:30 14:33:19 gfredericks argument validation in libraries: going forward, should it be done entirely with s/fdef, meaning no validation happens unless the user instruments the functions?
2018:03:30 14:35:23 borkdude s/assert is also an option I guess which can be compiled away
2018:03:30 14:35:35 borkdude but for arguments s/fdef is nicer
2018:03:30 14:37:19 gfredericks the downside is that things are a lot more GIGO for users who don't think to instrument
e.g., as a user, can I easily instrument all the spec'd functions in all my libraries without having to know which ones have specs?
is that too much? wouldn't it be more efficient to instrument only the functions I'm calling directly?
2018:03:30 14:38:00 borkdude you can instrument all fdef-ed functions with stest/instrument?
2018:03:30 14:38:31 borkdude but it has to be after you load their namespaces
2018:03:30 14:38:53 borkdude maybe adding it to reloaded.repl/reset will be a common thing
2018:03:30 14:39:45 borkdude but I get what you mean now. so you want to go only one level deep
2018:03:30 14:39:57 borkdude no transitive fdef checking
2018:03:30 14:40:21 gfredericks that'd be nice, since you probably have a large tree of libraries and don't want to slow down your dev by testing all the interactions between them
2018:03:30 14:41:11 borkdude What overhead are we talking about? I don’t mind a couple of milliseconds more during dev
2018:03:30 14:41:44 gfredericks totally depends on the libraries and what they're doing
2018:03:30 14:42:36 gfredericks in the extreme case, if specs get added to most of the clojure.core functions, instrumenting those will result in milliyears instead of milliseconds
2018:03:30 20:21:54 alexmiller that’s not a question :)
2018:03:30 20:22:07 hlship I'm working on it ...
2018:03:30 20:22:15 alexmiller :)
2018:03:30 20:21:58 hlship I've been down this path before, and what I've found is that the next term in the s/and gets the conformed value from the s/or, a tuple of (say), [:keyword :frob].
2018:03:30 20:23:08 hlship That's been fine so far EXCEPT as I'm switching to using Expound, the use of a conformer here is a problem:
(s/explain ::schema/enum-value "this-and-that")
clojure.lang.ExceptionInfo: Cannot convert path. This can be caused by using conformers to transform values, which is not supported in Expound
clojure.lang.Compiler$CompilerException: clojure.lang.ExceptionInfo: Cannot convert path. This can be caused by using conformers to transform values, which is not supported in Expound {:form "this-and-that", :val [:string "this-and-that"], :in [], :in' []}, compiling:(/Users/hlship/workspaces/github/lacinia/src/com/walmartlabs/lacinia/expound.clj:50:3)
2018:03:30 20:23:50 alexmiller this issue is actually discussed in the backchat
2018:03:30 20:24:19 hlship Recently? Got a link?
2018:03:30 20:24:38 alexmiller 2 days ago in this room - just scroll up till you see the expound stuff
2018:03:30 20:29:09 hlship That discussion wasn't helpful, if its the right one. I think they're hitting the same problem and want Expound to print it differently. I want to modify my spec to not trip over this scenario. s/nonconforming may work!
2018:03:30 20:24:06 hlship So my question is, how can I achieve the kind of spec I want in a way that avoids the use of a conformer in the middle.
2018:03:30 20:24:55 alexmiller another option is to wrap s/nonconforming around s/or
2018:03:30 20:25:06 alexmiller then you get just the value without the tag
2018:03:30 20:25:12 alexmiller when you conform that is
2018:03:30 20:25:40 alexmiller currently that’s an undocumented function but I think it’s likely we will either keep it or add a nonconforming variant of s/or
2018:03:30 20:27:25 seancorfield And in the backchat, one suggestion was to look at pinpointer instead of Expound.
2018:03:30 20:28:50 bbrinck @hlship I will likely be adding a “fallback” feature to expound soonish where you will see the vanilla spec error in this case
2018:03:30 20:29:18 bbrinck Note that will help if you only occasionally use conformers, but not if you have lots of them. Definitely check out pinpointer 🙂
2018:03:30 20:29:39 hlship Again, I'm quite willing to modify my spec to bypass this problem.
2018:03:30 20:29:58 bbrinck Ah, sorry, I missed that in the thread. Yes, that’d be the best approach! 🙂
2018:03:30 20:33:21 bbrinck FWIW, I tried to support conformers in expound but it’s really tricky if you’re just looking at the explain-data
2018:03:30 20:34:09 hlship s/nonconforming looks to be just what I want:
(s/explain ::schema/enum-value "this-and-that")
-- Spec failed --------------------
"this-and-that"
must be a valid GraphQL identifier: contain only letters, numbers, and underscores
BTW why are explicit messages not indented by Expound? Should I file an issue?
2018:03:30 20:35:23 bbrinck Can you modify that to show what you’d prefer?
2018:03:30 20:36:00 bbrinck (I’m always happy to get bug reports too 🙂 if that’s easier to discuss the options)
2018:03:30 20:37:47 bbrinck I’ll say this - off the top of my head, I think it’s working as I intended, but I’m always interested in improving the layout of error messages if it’s not clear
2018:03:30 20:40:47 hlship Here's a better example:
(s/explain ::schema/resolve {})
-- Spec failed --------------------
{}
should satisfy
fn?
or
implement the com.walmartlabs.lacina.resolve/FieldResolver protocol
The final line should be indented the same as the fn? line, don't you think?
;; is passed and should return.
(s/def ::resolve (s/or :function ::resolver-fn
:protocol ::resolver-type))
(s/def ::resolver-fn fn?)
(s/def ::resolver-type #(satisfies? resolve/FieldResolver %))
2018:03:30 20:52:24 bbrinck Yeah, when it’s part of the “or” it looks weird. I guess I was thinking that since “should” starts on left, custom messages would be the same.
2018:03:30 21:05:22 bbrinck The reason it’s on the left is that it should be in the same spot as “should” like so:
(s/def :example/temp #{:hot :cold})
(expound/expound :example/temp 1)
;;-- Spec failed --------------------
;;
;; 1
;;
;;should be one of: :cold, :hot
(expound/def :example/name string? "should be a string")
(expound/expound :example/name 1)
;;-- Spec failed --------------------
;;
;; 1
;;
;;should be a string
2018:03:30 21:05:54 bbrinck but I agree that when it’s part of a long “satisfy..or” block, it looks bad. I’ll make a bug
this works fine when I use set!, but not when I use alter-var-root!. Any ideas?
2018:03:30 21:50:00 bbrinck I’ve always used set! myself, so I’m not sure. Can you talk a little bit more about your use case and neither set! nor binding are a good fit here?
2018:03:30 21:50:55 hlship Well, at application startup, might want to alter-var-root, so that any later threads will use the Expound printer, not the default one.
2018:03:30 21:58:25 bbrinck Thanks for that. The short answer is I don’t know unfortunately - this seems to affect any printer, not just expound. Try: (alter-var-root #'s/*explain-out* (constantly (fn [ed] "Hello!"))). It may be a result of the way the REPL is set up - does it reproduce if you put the alter-var-root in the main of an application (as opposed to doing it in the REPL context)?
2018:03:30 22:03:44 bbrinck Ah, gotcha. I certainly understand your use case and I suspect others may run into the same thing. If you figure it out, let me know and I’ll update the docs. I have a note about using alter-var-root in non-REPL context, which IIRC, works fine, but if your scenario is at the REPL, then no dice
2018:03:30 22:07:18 hlship binding should work, as that will carry over into most started threads, including core.async threads.
2018:03:30 23:32:55 alexmiller I think the difference is that the repl binds explain-out so alter-var-root changes the root but the repl binding is the one being seen
2018:03:30 23:33:30 alexmiller So you have to set! at the repl
2018:03:31 15:34:53 gabriele how can i write a spec to validate a map like this: {[:type1 "args"] {:key-type1 value} [:type2 "args"] {:key-type2 value} ...(type-n)}
2018:03:31 15:35:18 gabriele where the value is dependent on the first element of the list that composes the key
2018:03:31 15:39:54 borkdude @gabriele.carrettoni First of all I would try to get the source of that data in a more workable format
2018:03:31 15:40:35 borkdude I usually try to go from dynamic keys to something more regular
2018:03:31 15:41:45 gabriele @borkdude i see, i wanted to use specs to do that
2018:04:02 20:07:27 avi 👋 hi all!
I’m having some trouble with a function spec with a :fn predicate… and my Web-searching has turned up nothing.
I’m using stest/check like so:
(-> (stest/check 'fc4c.core/shrink) first stest/abbrev-result)
wherein fc4c.core/shrink is my function that has an attached spec.
I always get a failure, but… if I grab the value of :val from the result, and then manually pass it to my predicate, like so: (pred val) then I always get true.
So am I missing something? I thought that if the predicate returns true, then the value conforms… I’m confused!
Thanks!
2018:04:02 20:13:47 seancorfield @aviflax We'll need to see a bit more code in order to help you...
2018:04:02 20:15:01 avi yeah? I’m happy to share more code, but I think I’m trying to just confirm an invariant about stest/check and function specs… that if the :fn predicate returns true then the “test case” will be considered to have succeeded… does that make sense? Or am I thinking about this wrong?
2018:04:02 20:16:32 taylor are you sure it’s failing due to the :fn spec? could it be failing on the :ret spec?
2018:04:02 20:17:48 avi I’m fairly sure but the output of check failures is somewhat overwhelming, even when abbreviated with abbrev-result … I’ll double-check right now
2018:04:02 20:19:12 avi Here’s a chunk of the output:
2018:04:02 20:27:16 avi Thanks for the help!
I’m a little confused… I don’t see that. If I dig the value of :ret out of the value of :val — I see a map.
2018:04:02 20:27:42 avi So it looks like shrink returned a map when check called it?
2018:04:02 20:28:02 taylor actually disregard, I copy/pasted your code poorly
2018:04:02 20:28:13 avi ah ok no worries! happens to us all!
2018:04:02 20:40:08 taylor I was just able to successfully check the function when I replaced the :args and :ret spec:
:args (s/cat :in map?)
:ret (s/nilable map?)
2018:04:02 20:40:36 taylor since the shrink function is more general/doesn’t need to know about your :diagram spec
2018:04:02 20:41:05 taylor but… I did try running your :fn spec function against that sample and it does return true, so not sure what’s up with that
2018:04:02 20:45:18 avi that makes sense… thanks for the help!
2018:04:02 20:48:18 seancorfield @aviflax I notice you're post-walking the tree -- are you aware of the bug in postwalk where MapEntry elements are not preserved?
2018:04:02 20:48:37 avi no, I wasn’t aware of that
2018:04:02 20:48:48 avi I guess I can search the Web to find more info on that
2018:04:02 20:49:14 taylor that doesn’t seem to be the big issue here
2018:04:02 20:49:38 avi But I’m not sure how that helps me understand the “contract” of fn predicates… my predicate is returning true so I just don’t understand why a failure is being reported
2018:04:02 20:50:11 taylor maybe dumb question, but have you tried restarting your REPL if you’ve been working on this for a while?
2018:04:02 20:50:30 avi not dumb!
2018:04:02 20:50:34 avi yes, more than a few times
2018:04:02 20:50:13 avi 🤔 maybe there’s a problem with the name shrink … maybe … that’s somehow confusing check since it internally does shrinking (seems like a long shot)
2018:04:02 20:51:15 seancorfield Why not test it with a much simpler :fn predicate that is (constantly true) and see if it still fails?
2018:04:02 20:51:29 avi hm ok good idea I’ll try that!
2018:04:02 20:52:01 seancorfield If that works (passes) then build the predicate up, piece by piece until you get a failure.
2018:04:02 20:52:46 taylor his :fn spec predicate does return true if you manually call it with the map from the check problem output, which is odd
2018:04:02 20:53:29 avi yeah that’s precisely what I’m confused by
2018:04:02 20:53:59 taylor but I agree w/trying to (re)build up from the simplest working case until it breaks
2018:04:02 20:54:30 avi but yeah, since (as I just learned) replacing the predicate with (constantly true) does lead to the check call succeeding… I’ll try rebuilding the predicate to try to debug at what point find a small case wherein I can trigger the confusing behavior
2018:04:02 20:54:45 taylor another thing to keep in mind is that it checks out fine if you don’t use his custom keys spec for in/out
2018:04:02 20:55:43 avi yeah… my input and output specs (and thus the generated args) are deeply nested colls… not sure why that’d matter but maybe it does
2018:04:02 21:15:06 avi anyone know whether/how to use stest/check and have it skip shrinking?
2018:04:02 21:22:28 avi ah never mind, I can work with it 😉
2018:04:02 21:45:45 avi hmm well I’ve been able to get the predicate to always work as expected by changing (= in-vals ret-vals) to (= (set in-vals) (set ret-vals)) — it appears that the two colls had the same values but in different orders. I’m confused as to why I wasn’t seeing this earlier. I’m very unclear on what I’m doing differently now 😞 … PEBKAC … sorry to have bothered everyone
2018:04:02 21:57:38 seancorfield That's cool that you've figured out the problem!
2018:04:02 21:58:20 avi thanks!
2018:04:02 21:58:22 avi yes!
2018:04:02 21:59:07 avi but also a bit disorienting that I suddenly can’t reproduce my confusing tangent…
2018:04:02 22:00:10 avi my only idea is that earlier I was copy-and-pasting the values as strings and then pasting them into the repl prompt, versus using e.g. (-> *1 :key :key) to get the in-memory values instead… I don’t know why that would matter though.
2018:04:04 23:30:35 flyboarder hello everyone, is there a version of multi-spec that uses an outside piece of data to determine the shape of the data? Instead of the current multi-spec which uses a piece of the data to figure out the shape of the rest of it?
2018:04:05 01:52:19 alexmiller No
2018:04:05 01:54:37 alexmiller And generally that implies you are prob doing something you shouldn’t as you then have a spec that is impure.
2018:04:06 14:39:18 flyboarder Could you explain an impure spec? I am trying to generate a spec from some nested data
2018:04:05 16:07:48 Charles Fourdrignier If I want to spec’ data from a CSV file, what would be the best strategy ?
Should I describe it as a tuple ? Or should I convert it to a map then spec the map itself ?
(I don’t find any article or blog post on this subject.)
2018:04:05 16:32:18 alexmiller a csv file is inherently a collection of collections of strings
2018:04:05 16:32:36 alexmiller and thus very uninteresting to spec and unlikely to tell you much
2018:04:05 16:33:14 alexmiller I suspect from an app perspective, it is far more interesting to convert to maps (coercing data if needed) and to spec the result, which is the data your app actually uses
2018:04:05 16:41:54 Charles Fourdrignier Thanks !
Tuples was my first intuition, but some reading changes my mind and your response finish to convince me.
2018:04:05 20:53:53 drewverlee I have some unqualified keywords that conceptually map to some specs i have created. Whats the suggested way to convert a unqualified keyword to a qualified one so i can exercise it (generate data) or conversely, use a generator on a unqualified keyword.
I feel i could do this my pulling things apart and putting them back together with strings but im guessing im missing an obvious exisiting way to handle this:
(str *ns* "/" :unqalified)
2018:04:05 21:25:30 manderson @drewverlee any reason that you couldn't just add namespaces to your unqualified keywords?
If you do want to go back and forth, the best way I can think of is as follows (perhaps someone knows of something better):
2018:04:06 13:01:01 kardan Anyone who can point on where to read on how structure specs? I started to spec in com.example.user but then it was odd define the user spec using namespaced keys. Would really want to to type (s/valid? :: #::{:email “<mailto:/cdn-cgi/l/email-protection|/cdn-cgi/l/email-protection>” :first-name “Bill” :last-name “Karlson”}) . So I’m guessing I should read up 🙂
2018:04:07 12:56:02 leongrapenthin @kardan it has become commonplace to use unsegmented namespaces for domain specific entities like :user/email
2018:04:07 12:57:58 leongrapenthin But I'd also love if someone wrote a state of the art summary because there is still lots of experimentation going on
2018:04:07 12:59:47 leongrapenthin @kardan consider that in your example you usually don't come up with a single user spec
2018:04:07 13:00:34 leongrapenthin in practice you end up creating different specs with keys in the usernamespace depending on what you need
2018:04:07 13:01:00 leongrapenthin for example an api that changes a user may say all user keys are optional
2018:04:07 13:01:16 leongrapenthin an api that creates a user may have some required user namespaced keys
2018:04:07 13:02:06 leongrapenthin this flexibility is easily missed when you are looking to create "the" ::user spec
2018:04:07 13:04:23 leongrapenthin the great thing is that you can compose and mix the keys freely in the context you need them whereas in classical type systems you need inheritance or whatever to reuse them.
2018:04:07 17:25:42 ikitommi unsegmented namespaces are not safe, :user/id might mean different things in different domains. If you need to integrate the two domains, you'll have conflictings specs.
2018:04:07 19:05:06 seancorfield @ikitommi It all depends on the scope of the data. If you're writing an application, not a library, and you have data whose scope is just that application, and that data is used entirely internally in that application, then something like :user/email might be fine. If that needs to flow through other code for additional processing, then you might want a more unique prefix. If data is truly isolated to a single namespace for processing (and is entirely opaque to everything else), the ::email would make more sense. If the data does lie somewhere in between, a "reasonably" unique prefix should be sufficient. Using namespace aliases makes working with namespaced maps a lot less painful -- and you can decide whether those should be real (code-based) namespaces or just arbitrary namespace prefixes independent of the alias. /cc @kardan
2018:04:07 19:31:17 mathpunk I'm starting a wrapper for an interesting Java library, for which I have domain knowledge of the problem they're working on. I am interested in adding specs based on invariants that I know so as to make it easier to use this library. Advice on how to approach such a goal is most welcome.
2018:04:11 18:55:15 mathpunk @U08EKSQMS This is exactly the kind of thing I was looking for, thank you!
2018:04:11 18:58:16 bbrinck np, good luck!
2018:04:08 12:09:54 leongrapenthin @seancorfield what do you think about keys like :git/sha used in ctd
2018:04:08 12:10:59 leongrapenthin @alexmiller would be interested how you decided that, too
2018:04:08 12:20:46 alexmiller Sorry, what’s the question?
2018:04:08 19:14:32 seancorfield @leongrapenthin when Alex talked about that (on the list? in #tools-deps here?) he said that's a human written format so it needs to be clear but not too verbose. A while different set of considerations compared to code.
2018:04:08 19:50:16 leongrapenthin @alexmiller I'm interested in a rule when its ok to pick unqualified/unsegmented namespaces. "Lib no;app yes" seems what has emerged mostly, but in tools.deps you claim for example "git". When should a lib author choose to do this and how can he not be worried about conflicts?
2018:04:08 19:51:33 leongrapenthin Or should we say unqualifies nses in libs are reserved for core libs
2018:04:08 19:52:48 leongrapenthin @seancorfield thanks, that makes sense from a practical standpoint. I still couldn't do it in my libs without worrying about conflicts ))
2018:04:09 01:04:17 olivergeorge I have an "on the fly spec generation in clojurescript" question.
2018:04:09 01:05:58 olivergeorge I'd like to generate and register new specs but clojure-spec has a macro based api and clojurescript doesn't have eval.
2018:04:09 01:06:28 olivergeorge Can someone point me at a sane approach please.
2018:04:09 01:07:37 olivergeorge It's possible a macro is the right way to do this but I get super confused about when macros evaluate in clojurescript.
2018:04:09 01:08:26 olivergeorge My spec generation looks at existing registered specs. That means my macro needs to consider the correct registry (presumably there's a CLJ one at compile time and a CLJS one at runtime).
2018:04:09 01:08:57 olivergeorge Pretty sure I'm lots in the weeds.
2018:04:10 06:17:51 aaron51 looking to auto-generate an HTML form from spec/schema for non-programmers (not API documentation, with validations). is there a simpler alternative to swagger out there, that works with compojure.api.sweet?
2018:04:11 19:13:30 donaldball What’s the least wrong way of tightening a spec in the registry in the context of another spec?
For example, I’m writing specs for config maps in general, and for config maps for production. The general spec for e.g. :config.mode might be #{:app :lib} but when it appears in my :config.prod spec, I’d like to further constrain it to be simply #{:lib}.
s/and with predicates is a perfectly adequate way to do this, but I seem to recall a better way.
2018:04:11 19:47:37 alexmiller s/and is what I would say
2018:04:11 20:09:40 donaldball It makes generating values unlikely, but this is also a case where I really don’t care about that
2018:04:12 12:49:09 Andreas Liljeqvist How would I spec a map like {:type akeyword :args dependsonkey}?
2018:04:12 12:54:03 alexmiller s/multi-spec is designed to handle cases where you choose the spec based on the data (here the :type)
2018:04:12 12:54:18 alexmiller So that may be a good match here
2018:04:12 12:54:50 Andreas Liljeqvist I can use a multimethod to spec the map depending on type (multimethod methodname :akeyword [m] (s/keys [:args]))
2018:04:12 12:55:27 alexmiller Yes, that’s how s/multi-spec works
2018:04:12 12:55:45 Andreas Liljeqvist But I can't see at the moment how I would specify that :args should have a different spec for that match
2018:04:12 12:56:55 Andreas Liljeqvist Like that it should accept string? if :type is :a, but int? if :type is :b
2018:04:12 13:02:36 alexmiller Is :args actually unnamespaced?
2018:04:12 13:02:57 Andreas Liljeqvist nah, everything should be namespaced
2018:04:12 13:05:37 alexmiller When unsure how to spec something, it’s best to always return to how to represent the truth of your actual data. In this case you’re saying that you have one attribute that can have a variety of different structures
2018:04:12 13:06:07 alexmiller So you need to capture that in the spec
2018:04:12 13:06:54 alexmiller The spec for that attribute is A or B or C
2018:04:12 13:08:24 alexmiller You then have a separate constraint that says that a particular value of :type should co-occur with a particular form of :args and you should capture that constraint as a separate predicate
2018:04:12 13:09:37 alexmiller So you would model the attribute with s/or, the map with s/keys, and the constraint by s/and-ing the map and the constraint
2018:04:12 13:23:27 Andreas Liljeqvist Thank you. s/and-ing is always powerful, only problem is that we have to provide a custom-gen.
2018:04:12 13:23:47 Andreas Liljeqvist But usually I have to write custom-gens anyway...
2018:04:12 14:52:32 guy What would you do for speccing [& args] ?
2018:04:12 14:53:17 alexmiller unless you have other knowledge about args
2018:04:12 17:32:39 kenny Is it possible to have a multi-spec dispatch on the first value of a vector and return a Spec for the rest of the vector? i.e. take [:my-vec 1 "2"]. The multi-spec would dispatch on :my-vec and each defmethod would return a spec for (vec (rest [:my-vec 1 "2"])) - [1 "2"].
2018:04:12 17:36:26 alexmiller no, but you could dispatch on the first value of a vector and return a spec for the whole vector
2018:04:12 17:37:49 kenny Yeah... It's just the Spec for the first part of the vector is uninteresting -- it's always going to be any?. This makes the return value for the defmethods very repetitive.
2018:04:12 17:49:07 kenny That's also possible. The problem there is that once I move that to a macro, I need to move all functions that register a method for that spec to be macros.
2018:04:12 17:54:20 dadair Would x always be any? You could have x be say #{:my-vec} so the spec is more specific to the event spec you are returning?
2018:04:12 17:54:42 dadair More specific for tests around that event
2018:04:12 17:54:52 kenny Yes, always any?.
2018:04:12 17:55:21 dadair but for that specific defmethod isnt’ x :my-vec?
2018:04:12 17:55:58 kenny Yes, but that's already guaranteed because the multimethod is called.
2018:04:12 17:57:04 kenny The API consists of a lot of functions that look like this:
(defn reg-my-thing
[id spec other-stuff]
(defmethod my-multimethod id
[_]
(s/cat :x any? :rest spec))
;; do other stuff
)
In order to do what you're saying I'd need make most of the API macros. That isn't the end of the world but it does make the code base a lot messier to do what seems like such a simple operation.
2018:04:12 17:57:30 alexmiller @kenny re “The problem there is that once I move that to a macro, I need to move all functions that register a method for that spec to be macros. ” - why?
2018:04:12 17:57:57 kenny Because the above code will not work.
2018:04:12 17:58:06 kenny cat needs a form, not a symbol.
2018:04:12 17:58:32 kenny
(defmacro reg-my-thing
[id spec other-stuff]
`(defmethod my-multimethod id
[_]
(s/cat :x any? :rest ~spec))
;; do other stuff
)
2018:04:12 17:59:17 alexmiller I don’t think that macro is correct, but it can be fixed
2018:04:12 17:59:31 kenny It's not - more of psuedo code.
2018:04:12 17:59:39 kenny Essentially in order to program with spec, everything needs to be a macro.
2018:04:12 18:01:33 alexmiller this is a macro already, it’s just not the right macro
2018:04:12 18:01:59 kenny Not sure I understand what you mean.
2018:04:12 18:06:43 kenny My API is defined a bunch of functions that are passed a spec for the (vec (rest [:my-vec 1 "2"])). The functions all do global registration sort of thing (akin to defmethod). Each of these functions needs to register a spec for the whole vector [:my-vec 1 "2"]. I could construct that spec at the macro level based on the spec they passed in, but that'd mean my whole API needs to be defined at the macro level.
2018:04:12 18:07:45 kenny ... because this doesn't work 🙂
(defn reg-my-thing
[id spec other-stuff]
(defmethod my-multimethod id
[_]
(s/cat :x any? :rest spec))
^^^^
;; do other stuff
)
2018:04:12 18:15:04 alexmiller something like this works:
2018:04:12 18:29:40 alexmiller really the same thing you’re doing with a macro
2018:04:12 18:29:51 kenny This is a CLJS project so I'm not sure about the eval usage.
2018:04:12 18:30:21 alexmiller oh sure, throw that in at the end :)
2018:04:12 18:32:06 kenny Should've mentioned that in the beginning 😬 It's essentially adding Spec to re-frame, thus the reg-* API.
2018:04:12 18:32:46 alexmiller well then, I don’t know :)
2018:04:12 18:33:27 alexmiller I don’t understand the constraints in cljs as well. There are some changes coming to spec that will help with stuff like this too but I’m not sure when or how they will play out in cljs.
2018:04:12 18:34:37 kenny I'm guessing everything will need to be done at the macro level. I think the only constraint is the lack of eval.
2018:04:12 23:31:13 mv If I have a list of lists, is it possible to write a spec that enforces that no two sublists start with the same value?
2018:04:12 23:35:37 seancorfield @mv Sure, if you can write a predicate that tests for that, you can use that predicate in a spec (or even as a spec).
2018:04:12 23:35:55 mv So just a standard function?
2018:04:12 23:36:01 seancorfield Yup.
2018:04:12 23:36:39 mv And I’m assuming s/valid?? I’m new to spec
2018:04:12 23:37:18 seancorfield Presumably you already have a spec for "list of lists"?
2018:04:12 23:38:29 mv Not yet, but there is a spec being applied for the individual elements with s/*
2018:04:12 23:39:04 seancorfield OK, so when you have your spec for list of lists, then you just s/and that spec with your predicate and that's your complete spec.
2018:04:12 23:39:52 seancorfield (or something like that)
2018:04:12 23:40:06 mv Neat!
2018:04:12 23:41:02 seancorfield Bear in mind you may not be able to generate data from that spec (you might, but generation may produce sublists with identical first elements quite often so the check on the generated data might fail).
2018:04:12 23:41:25 seancorfield If that's important, you'll need to write a custom generator.
2018:04:12 23:41:42 seancorfield But if you're just getting started with spec, you may not need that. Yet 🙂
2018:04:12 23:49:29 mv Yea I don’t think I need that yet, this is more to enforce a bug won’t come back
2018:04:13 22:00:28 hmaurer Is it possible to use clojure multi-specs with derived keywords (through derive) and ensure that “child specs” are a superset of parent specs?
2018:04:14 14:11:23 alexmiller why do think that’s pre-1.9?
2018:04:14 14:11:47 alexmiller I wrote the guide btw
2018:04:14 14:16:55 matan okay, so the namespace remains alpha, got it 🙂
2018:04:14 14:17:01 matan good guide BTW! 🙂
2018:04:14 14:18:26 matan but don't need to explicitly require it anymore with clojure 1.9 do we?
2018:04:14 14:18:35 alexmiller you do
2018:04:14 14:18:56 alexmiller require just loads the code into the clojure runtime, as you need to do with any namespace
2018:04:14 14:20:24 matan right!
2018:04:14 14:20:40 matan it's not part of core that's why
2018:04:14 21:18:01 seancorfield @hmaurer That says it's very stable in its 'alpha' state! 🙂
2018:04:14 21:20:31 seancorfield I gather that some changes to make specs easier to work with programmatically are on the roadmap @gklijs but it's been great for us to use as-is in our production code. We're very happy with it. And we were using it early on in the 1.9 prerelease cycle, before it became .alpha and was moved out of core (so, yeah, there were some code changes needed along the way -- but overall Clojure doesn't require many of those over the years).
2018:04:14 22:54:41 gklijs That's fine and even if there are changes it will probably not change the main use. However I use it to store and retrieve data with the option to use a backward compatible spec to get the full data back, and I'm a bit scared changes might break it.
2018:04:14 23:45:59 mathpunk Oh, also --- if I'm incorporating generative tests into a library, is the idea that I run stest/check inside a deftest?
2018:04:15 01:03:10 alexmiller as the doc string for s/fdef says: ” :args A regex spec for the function arguments as they were a list to be
passed to apply - in this way, a single spec can handle functions with
multiple arities”
2018:04:15 02:37:28 seancorfield @mathpunk :args should always be an s/cat with each argument specified.
2018:04:15 02:39:43 mathpunk I've been meditating on what @alexmiller said and, i just realized, i think i figured "well a simple string is a 'regex' that matches very few things so, one argument is just a spec for the argument"
2018:04:15 02:39:50 mathpunk now, i think i see it
2018:04:15 02:39:53 seancorfield Also (:args %) in your :fn spec will be the collection of argument values, so you'll likely want (:v (:args %)) -- to match the :v in the :args spec
2018:04:15 02:40:28 seancorfield Read the Spec guide about spec'ing functions again to see how it uses s/cat
2018:04:15 20:37:23 mathpunk When I run something like "(test/check `simplex {:clojure.spec.test.alpha/opts {:num-tests 1}})" it looks like it's still running 1000 tests. 🤔
2018:04:15 21:07:35 mathpunk Mind you, that might not be my problem --- I think I'm tucking way too much into my spec for a simplex, when I should be instead specifying the functions that act on a simplex (i.e. the ones named in the Simplex protocol). I'm a little fuzzy on how to specify functions that should be true in /every/ implementation, though...
2018:04:15 21:55:01 mathpunk hmm, I'm also a little unclear on how to run a generative test on a function from outside of the namespace it's defined
2018:04:15 22:18:35 seancorfield @mathpunk Just specify the fully-qualified name of the function to test/check.
2018:04:16 08:52:15 hkjels I have a lot of cases in my code where I destructure the input of a function using a spec and then extract the keys using a long vector that’s basically the exact same as the spec itself. Is there a way to just use a spec to destructure keys?
2018:04:16 08:52:59 hkjels something like {:keys ::component/params}
2018:04:16 08:57:29 hkjels in other languages you have like a splat or explode that would give you all of the keys as vars in the current scope. I get that it would be a bad idea, but with a spec your still explicit
2018:04:16 15:05:26 borkdude @hkjels This is exactly what I also asked for recently (some weeks back)
2018:04:16 15:05:53 borkdude @seancorfield wrote a macro but I wanted it to do without writing my own let macro
2018:04:17 07:33:50 hkjels yeah, this seems like something that should be natively supported
2018:04:16 15:25:53 bbss I wrote a macro that uses the registry to return a vector with the keys, and use that in combination with C-c C-v C-w eval and replace in the editor. Of course there is some potential erosion as you'd need to maintain it in multiple places. But at least it's explicit.
2018:04:16 18:31:18 mathpunk I'm still working out where to put specs in my project. I'm coming from test-driven development so it seems natural to put fdefs in test namespaces: define what the function should do, separate from the implementation. But then again, does that mean that code that calls the function won't have loaded those specs and will be missing out on the benefits? I'm curious how people are organizing their projects.
2018:04:16 21:47:53 bbrinck I tend to put the fdef right before the defn. That provides the most documentation benefits IMO and also anyone who loads my code is guaranteed to get the specs if they want to instrument them
2018:04:16 21:48:20 bbrinck As you noted, if you put them in test namespaces, then clients will need to load both namespaces in order to instrument, which I think would be surprising
2018:04:16 21:48:30 bbrinck YMMV
2018:04:16 18:37:30 leongrapenthin @mathpunk use :clojure.spec.test.check/opts
2018:04:16 18:37:45 leongrapenthin For some reason they missed to alpha it
2018:04:16 18:39:33 mathpunk Ah ha! Thanks so much
2018:04:17 00:10:55 d._.b I have a few transforms that are related in my application.
What would be cool is if I could somehow link all of these specs together, and control generation. So, an initial spec for phase 1 would refer to phase 2, and phase 2's spec would refer to phase 3.
2018:04:17 00:16:16 d._.b maybe this is more of a controlled generation question?
2018:04:17 00:17:50 d._.b maybe conform/unform is what im looking for here?
2018:04:17 00:20:01 d._.b where a named conformer would be the transform between two phases, so I could (s/def ::phase-1->phase-2 (s/conformer ...))(s/conform ::phase-1->phase-2 {:aH "abc"})
2018:04:17 00:55:16 bbrinck @d._.b Hm, hard to say without seeing the code, but I suspect it’d be easier to just write transform functions with a transformation library and use the specs to validate each transform is working
2018:04:17 01:27:45 d._.b @bbrinck yes, this makes sense to me. it is fine for me to write the transform functions and validate the transform, but for some of these specs, the difference is very small. For instance, "person": {:name "bob"} => {:name (get {"bob" :fun-bob} "bob")} => {:nick-name "fun-bob"}. It would be nice if I could link the specs as an ordering, and then choose which phase I want to generate.
2018:04:17 01:28:20 d._.b from an initial state or spec
2018:04:17 01:38:00 bbrinck @d._.b Yep, I can see how this would be convenient. I can’t speak for spec maintainers, but from what I’ve read so far, they have consistently indicated they want to stay away from transformation (and leave this to other libraries - both libraries that are spec-aware or those that are spec-agnostic)
2018:04:17 01:39:56 bbrinck I can certainly see the use, but this can’t be done in spec today and I doubt it will be added 🙂 . https://github.com/nathanmarz/specter might be a good fit here
2018:04:17 02:29:00 andy.fingerhut Not sure if it is something targeted at what you are hoping to do, but late in the Google group discussion linked above is a mention of the project spec-coerce, which the author says "instead of making conforms that do coercion, it uses the spec and a separated registry to conform the value (similar to what spec does to find the generators for a given spec)". https://github.com/wilkerlucio/spec-coerce
2018:04:17 05:43:07 alexmiller it’s not a good idea
2018:04:17 06:06:32 ikitommi would it be a good idea to add support for spec coercion in the core? or support for generic walking with coercion as one application for walking.
2018:04:17 12:41:05 ikitommi I renamed CLJ-2251 to “Generic Spec Walking for clojure.spec”. Is something like that coming? Would you be interested in a demo / patch of that?
2018:04:17 12:53:11 alexmiller It’s something we’ve talked about but not particularly looking for a patch.
2018:04:17 06:19:17 d._.b Thanks for those recommendations.
2018:04:17 06:27:57 d._.b When an entity like “foo” exists in numerous, ordered forms, it feels superfluous to name intermediate specs, and edges on “types”. I don’t want that extra, unnecessary complexity in my programs, so I’m looking for a way to define a “foo” as a succession of small transformations, where the phases (minor transforms) are named loosely, abstractly. One place to define all of the successions of miniature transforms is better than enshrining stronger, named entities. The names wind up feeling forced.
2018:04:17 06:29:28 d._.b :thing/upcased-name feels like extra nonsense if it’s a short-lived intermediate.
2018:04:17 06:33:09 d._.b Coalescing specs into logical, ordered groupings feels valuable if only to avoid the naming of tiny-delta intermediates.
2018:04:17 07:13:53 mathpunk NB: Every simplex is a complex, but not every complex is a simplex.
2018:04:17 14:39:10 bbrinck The problem seems to be with your :fn spec on dim. The reason the examples work e.g. (dim (complex [])) is that they don’t run the :fn code
2018:04:17 14:39:39 bbrinck I deleted the :fn spec on dim and now (test/check dim)` returns with no errors
2018:04:17 14:41:33 bbrinck It looks like the issue with the :fn is that :ret does not contain the value, it contains the conformed value e.g. not -1, but rather [:empty -1]
2018:04:17 14:52:31 bbrinck I think this will avoid the exceptions, although the definition of dim still fails the :fn test in some cases:
2018:04:17 17:34:39 mathpunk @bbrinck Ah ha! I forgot about conformance.... aaaagain. And orchestra looks useful, much obliged
2018:04:17 07:18:26 mathpunk What's really baking my noodle is, there are a few things I can think of that might just be plain broke in the way I'm writing this, but, it would be broke with MUCH smaller examples than I seem to be getting as examples of failing results
2018:04:17 20:09:06 mathpunk I'm trying to define different specs for different implementations of a protocol, which I don't know if it's even possible. This code passes my example tests, but when I try and test/check any of the functions, I just get an empty sequence: https://github.com/mathpunk/simplexity/blob/complexes/src/simplexity/simplex.clj
2018:04:17 20:12:06 mathpunk The problem I'm trying to solve is: A simplex is a special case of a complex. It would be nice to construct a simplex by going (simplex [0 1 2]) and a general complex by going (complex [[0 1 2] [2 3] [4 5 6]])
2018:04:17 20:12:57 mathpunk That is, construct a simplex from a collection of vertices, and a complex from a collection of (maximal) simplexes
2018:04:17 20:13:37 mathpunk I'm open to other ways of accomplishing this goal
2018:04:17 20:27:35 bbrinck I haven’t tried it myself, but the advice I’ve heard is that protocols should be internal implementation details and you should spec wrapper functions, which become the public-facing API.
2018:04:17 20:27:45 bbrinck I could be misremembering the advice 😉
2018:04:17 20:29:16 bbrinck In this case, you’d have a spec for the wrapper function. I’m not sure that it’d need to be different though - if complex is the general case, could the work in terms of complex? I’m not sure
2018:04:17 20:30:27 bbrinck You could certainly create a generator that creates either simplex or complex values though, and use that
2018:04:17 20:38:45 bbrinck Try wrapping dim in another function e.g. (defn dim1 [x] (dim x)), then write spec for that. test/check works for me on dim1 (poorly named, of course 🙂 )
2018:04:17 20:28:46 alexmiller You can’t spec protocol functions
2018:04:17 21:07:22 kenny Why is the predicate function in the below error message :clojure.spec.alpha/unknown?
2018:04:18 02:22:34 jumblemuddle Is it possible for me to have a spec: s/cat regex where latter parts of the regex use predicates that depend on the conform of prior parts?
2018:04:18 02:29:29 jumblemuddle I guess the alternative would be to have a second spec to run after the first one, so it can depend on the output of the first one.
2018:04:18 16:54:03 cap10morgan If you had some spec'd code where correctness was far more important than performance, a) would it make sense to turn on instrumentation of every fn at production runtime and b) if so, how would one go about doing that?
2018:04:18 17:10:41 seancorfield @cap10morgan instrument accepts a sequence of symbols to instrument as I recall. You'd probably want to just instrument the functions in specific namespaces (your own) to avoid turning on instrumentation for all libraries functions in your app.
2018:04:18 17:11:42 seancorfield (you can "instrument everything" by calling instrument with no arguments but you wouldn't know what functions got instrumented so you'd have no idea about performance impact)
2018:04:18 17:12:16 alexmiller you would know as it returns a collection of all symbols instrumented
2018:04:18 17:12:41 seancorfield I meant, ahead of time.
2018:04:18 17:12:50 alexmiller you do have to ensure the specs have been loaded (by requiring them) before calling instrument
2018:04:18 17:13:26 seancorfield But, yeah, you could potentially get a very long list of instrumented functions that you weren't expecting 🙂
2018:04:19 19:09:38 xiongtx Is there an easy way to modify an existing s/keys spec?
I often find myself needing to take an existing spec and update a few keys. It’d be nice if I could just replace a key instead of copying everything.
2018:04:19 19:50:31 seancorfield Maybe s/merge would work for you @xiongtx? I'm not quite sure how it handles merging over existing keys tho'
2018:04:19 19:51:00 xiongtx I managed to get around it by redefining the specs that differ
2018:04:19 23:04:24 kenny Is it possible to instrument initial function calls? i.e. I have my dev namespace where I enable instrumentation:
The call to reg-event is not instrumented at this point so I do not get a instrumentation error that my call to reg-event is incorrect.
2018:04:20 01:40:24 dadair You could just (st/instrument reg-event) after it’s fdef?
2018:04:20 03:10:39 kenny True but that doesn't scale well and requires special considerations in production.
2018:04:20 19:12:27 kenny I wrote a small library that addresses the above problem -- instrumenting initial/global function calls. It takes a slightly different approach than fdef. During development it immediately wraps function calls with assertions on a function's args and ret value. Thoughts on this idea? 🙂https://github.com/Provisdom/defn-spec
2018:04:20 19:35:56 franquito Neat! Looks pretty similar to orchestra.
2018:04:20 20:15:37 kenny Ah you're right. I did not realize orchestra did more than enable :ret checking for instrumentation. This approach is still different than orchestra's defn-spec. Orchestra's defn-spec also doesn't have great IDE support.
2018:04:21 03:00:08 franquito @U083D6HK9! I pretty much skimmed the README at work. Now I gave it a read and see more clearly the purpose of the lib (BTW, that's a well written README). My thoughts are: I liked it. You just don't care anymore about instrument.
2018:04:21 05:03:56 kenny Thank you. And yeah, I came to a similar conclusion.
``
Is it possible to also have the last predicate error ? (> % 1000)
Many Thanks
2018:04:24 12:12:34 alexmiller In this case, no. Because s/and flows values through if an earlier one fails it can’t run a later one
2018:04:24 12:51:30 ouvasam Thanks. Is there another way to get all the errors ?
2018:04:24 13:44:57 alexmiller not without breaking apart the s/and into pieces and running each individually
2018:04:24 17:23:29 xiongtx Is there a way to express the idea that a spec is any subset of some provided set, and to have generator behave accordingly?
2018:04:26 09:11:30 borkdude Gary talked about an interesting issue recently. What if you instrument all and there are loads of fdefs for core functions. This will create a lot of validation overhead. Anyone experienced this becoming an issue?
2018:04:26 11:54:09 alexmiller Currently, no core functions are spec’ed (just macros, which are done at compile time not runtime, although admittedly those are often pretty close together in repl dev). Having too much instrumented definitely can create a noticeable effect.
2018:04:26 09:46:37 olivergeorge There's definitely overhead. By rights it shouldn't matter so much at dev time which is the common use case for instrumenting.
2018:04:26 12:36:48 gfredericks it'll matter if specs get added to clojure core fns; I can easily imagine a 10000x slowdown depending on what you're doing
2018:04:26 09:54:44 Rose Molina Good morning,
I recently started using spec and I wanted to know the reason behind why s/and conforms values and propagates them.
I hadn't read the spec guide (https://clojure.org/guides/spec#_spec_ing_functions) in full before I found the first example of this in our code base and I found it hard to read.
2018:04:26 11:52:10 alexmiller conforming tells you not just whether something validated (like valid? but how it validated - indicating which alternatives are taken and how value components were parsed). Flowing conformed values allows s/and to also flow conform info out to the user from internal components (which is also done in most other conforms)
2018:04:26 13:03:27 dominicm Is it possible to "redirect" a failure. For example, if a list of numbers must be sequential, this validation can only be performed on the "whole", but the failure is on a particular index.
2018:04:26 13:06:38 alexmiller not without implementing your own spec
2018:04:26 14:20:14 dominicm @alexmiller is that api private currently?
2018:04:26 14:20:30 alexmiller it’s public but undocumented and subject to violent change
2018:04:26 14:21:09 alexmiller as in, I expect it to change, and possibly even be completely replaced.
2018:04:26 14:21:12 dominicm I'll wait on that, knowing it isn't impossible is good.
2018:04:26 14:21:23 alexmiller things like spec-tools are using it extensively
2018:04:26 14:21:27 dominicm It was one frustration I'd had, e.g. for password validation data.
2018:04:26 14:21:41 dominicm Yeah, I wasn't touching it because I know it will break some day.
2018:04:26 15:23:51 ikitommi the Spec protocol will be replaced? Is it possible to comment or discuss on the upcoming changes, from 3rd party library maintainer perspective?
2018:04:26 15:31:10 alexmiller no, I don’t know yet what they will be
2018:04:26 15:31:29 alexmiller it may be replaced, it may not be replaced, it may change, I have no idea yet
2018:04:26 15:32:08 alexmiller this is the alpha part of spec.alpha
2018:04:26 15:32:30 alexmiller my advice right now would be - don’t use that
2018:04:26 15:34:31 alexmiller I expect changes to the main user api of spec to be minimal or at least well thought about
2018:04:26 15:38:49 ikitommi ok. thanks. there is a alpha disclaimer too on spec-tools, but just checked, it has 13k downloads, so someone is using that already. hopefully there is a way to port the features to new spec apis - or remove stuff that is implemented by the new core directly.
2018:04:26 15:41:27 alexmiller I have not been shy about telling spec-tools and others what I said above
2018:04:26 15:41:45 alexmiller spec-tools is almost certain to be broken by future spec changes
2018:04:26 15:42:26 alexmiller I don’t agree with most of what that library is doing
2018:04:29 21:27:58 devurandom Hi! I would like to spec a map where the values have to conform to some spec, but the fields can be arbitrary. How do I do that?
2018:04:29 21:30:32 alexmiller Use s/map-of and any?
2018:04:29 22:07:54 devurandom Thanks, that works! This was the first time with Spec for me and the original code I modified was using cat, which was confusing me.
2018:04:29 22:14:57 devurandom @alexmiller s/cat also matches maps, is that correct? So I need to get the ordering right to make s/map-of match first / instead of s/cat?
2018:04:30 09:13:49 jimmy @clojurians.net thanks for you help. I have tried it and it returns empty []. If I have the fdef in the same ns, and I run (st/instrument) it works
2018:04:30 09:20:08 gnl Just tested it to make sure I'm not missing something, but it works for me. Have you made sure that the external namespace the fdef is in is :required?
2018:04:30 09:37:51 jimmy yes it does. I feel like there is something missing as well, but I haven’t figured it out yet, it should work like your said.
2018:04:30 09:44:55 gnl @nxqd Can you post the head of your fdef, as in (s/fdef <name> ...)?
2018:04:30 09:45:11 guy I found if your fdef doesn’t work correctly you can’t instrument it
2018:04:30 09:45:36 guy if it gives that error saying it cant be satisfied after 100 tries or something, it won’t be able to be instrumented
2018:04:30 09:46:16 jimmy @guy if I put this spec in the same ns, it works.
2018:04:30 09:46:26 guy interesting
2018:04:30 09:46:45 jimmy my purpose is to reuse the same fdef button ns for different functions
2018:04:30 09:46:57 guy are you doing it across clj/cljs?
2018:04:30 09:47:20 jimmy yes, but for now, I test in in clj repl, so it’s only java for now.
2018:04:30 09:47:23 guy kk
2018:04:30 09:47:39 gnl Am I being blind here or is there no (defn button ....)?
2018:04:30 09:47:53 guy i think he had ;;button fn
2018:04:30 09:47:54 jimmy @clojurians.net it’s defined in another namespace.
2018:04:30 09:48:00 gnl ah
2018:04:30 09:48:02 guy dont you have to import it?
2018:04:30 09:48:07 guy some-ns/button
2018:04:30 09:48:17 gnl well then you have to (s/fdef some-ns/button)
2018:04:30 09:49:51 guy Also thats a super common mistake to make so don’t worry about it
2018:04:30 09:50:00 guy I had the same issue before
2018:04:30 09:50:12 guy and spent an hour or two being confused
2018:04:30 09:50:13 guy 😄
2018:04:30 09:50:21 jimmy yeah, most of the time I just define it in the same ns 😄 so I didn’t notice this.
2018:04:30 09:50:30 guy 😄
2018:04:30 09:50:35 jimmy thanks a lot guys
2018:04:30 15:06:05 deansher This is a question about clojure.spec.alpha as an example code base, rather than about using it. In the following code, it seems to me as though alpha.clj is working very hard to avoid multimethods. Does anyone have insight into why?
2018:04:30 16:35:35 ghadi it's not just performance.
2018:04:30 16:53:05 alexmiller in this particular case, the protocol has one method, so that’s not a concern
2018:04:30 17:10:13 deansher Hmm, I’m thinking of it quite differently. The Specize protocol is allowing keywords, symbols, and (to a degree) objects to participate in the multiple-method Spec protocol, instead of using multi-methods. For example:
(defn conform
"Given a spec and a value, returns :clojure.spec.alpha/invalid
if value does not match spec, else the (possibly destructured) value."
[spec x]
(conform* (specize spec) x))
(defn unform
"Given a spec and a value created by or compliant with a call to
'conform' with the same spec, returns a value with all conform
destructuring undone."
[spec x]
(unform* (specize spec) x))
2018:04:30 16:38:19 jimbob is there a good spec function that will return the req-un’ed keys for a spec defined map?
2018:04:30 16:40:12 jimbob example. i have a record blah
{:a a :c c :e e }
and a spec def
(s/def ::blah-fact
(s/keys ::req-un [::a ::e])
whats the best way to call a function to get back
#{:a :e}
?
2018:04:30 16:42:12 ghadi @ben.borders Here's one way:
2018:04:30 16:53:44 eoliphant Hi, I’m working on a system of clojure based services, that passes messages based on clojure maps around. So Im thinking I can just create a common module of specs (or at least a common as they need to be), but I have one other potential wrinkle. We basically need to support ‘tenants’. So I might have the same attribute, that means the same thing throughout the system (s/def :account/number ; like "A123") so far so good. But for another tenant maybe :account/numbers should start with “B”.. Is a good way to handle this ?
2018:04:30 16:53:45 alexmiller note that that won’t work if :req-un contains and or or @ghadi @ben.borders
2018:04:30 16:59:23 jimbob cool that works. thanks!
2018:04:30 21:35:41 ghadi Has anyone experienced... "Overspeccing"?
2018:04:30 21:52:00 seancorfield @ghadi You mean, someone trying to spec "everything"? I've seen some folks (new to spec) go down that path...
2018:04:30 21:54:21 ghadi Maybe not as an intent, but definitely as an effect
2018:04:30 21:54:43 ghadi e.g. spec the wire boundary, spec post-transformation, spec the output (of whatever)
2018:04:30 21:54:54 ghadi then you end up with hundreds of specs
2018:04:30 21:55:48 alexmiller it’s useful to spec core data structures, boundaries, functions amenable to property testing
2018:04:30 21:56:31 alexmiller but it’s totally ok to not spec stuff, or use any? or ifn? etc to under-spec things that are still in process or too annoying to spec
2018:05:01 10:24:42 dominicm This possibly stems from not knowing when to not apply specs, causing an excess of inappropriate specs. It's easy to assume spec = types, ergo spec all the things [meme]
2018:05:01 10:25:59 dominicm I personally struggle at times to know when to, and not to apply certain tools that Clojure provides, and the entire ecosystem even less. e.g. there are still regulars queries about protocols vs multimethods, until recently I reached for protocols more often than I did multi-methods, for no real reason.
2018:05:01 11:15:52 mpenet specs "could" be used everywhere in theory, but in practice gen gets in the way (even via instrumentation)
2018:05:01 11:17:35 mpenet then if you want to s/check everything you're in for some waiting. not really usable in practice that way either. s/check requires to be more selective
2018:05:01 11:25:35 mpenet I personally wish instrumentation would never trigger gen, or at least that we would have the option to disable it without having to dumb down the spec (ex: not having to resort to ifn? for predicates)
2018:05:01 16:39:51 seancorfield We tend to spec arguments to APIs, some data structures around the DB (where we can extract the lists of fields to save/update from the specs), core domain data structures, and a handful of functions that interact with those, either across module boundaries, or for certain "critical" domain functions. We try hard not to overuse them because we want to avoid brittleness.
2018:05:01 17:59:45 drewverlee Whats the ideal way to spec a nested structure? e.g
{:x {:y {:a "hi" :b "bye"}}}
All the examples in the docs are fairly flat data structures. Also most examples assume the key is a key and not a unique value.
Which in most cases, it should be, in this case i was modeling a graph and i’m not sure its reasonable/easy to do it that way.
In a related vein. I wonder how easy it is to do a depth first search with datalolg and datascript. Which would be another way to go about doing this.
2018:05:01 18:37:46 drewverlee Yea. After some more thought, its not the nesting thats the issue. Its the fact that my keys are unique in this example. I’ll have to think about if i want to lose some terseness in order to label things more.
2018:05:01 19:03:11 tbaldridge graphs are going to be hard to spec completely, you'll need a "any?" in a few places.
2018:05:01 19:03:39 tbaldridge spec will consume stack as it traverses the graph, so if you manage to create a really deep graph, or a cyclic graph, you'll have problems validating it
2018:05:02 14:28:15 triss so how do I stop stest/check from generating NaNs all the time?
2018:05:02 14:29:31 triss the spec that fails is: (s/and number? #(<= 0 % 100))
2018:05:02 14:33:37 alexmiller do you really need number? ? or is something narrower like int? appropriate?
2018:05:02 14:34:06 alexmiller it is a known issue right now that NaNs cause issues for certain generators and something we intend to fix
2018:05:02 14:35:54 triss ah ok. I’ll see if int-in works out for me. thanks.
2018:05:02 14:37:04 alexmiller #(not (Double/isNaN %)) pred may be useful to filter too
2018:05:02 14:37:40 guy Sounds silly but does the order matter inside the s/and
2018:05:02 14:37:49 guy in the NaN example
2018:05:02 14:37:59 guy would you have to have #(not (Double/isNaN %)) second?
2018:05:02 14:38:09 guy then #(<= 0 % 100) last?
2018:05:02 15:07:54 alexmiller yes, order matters as the first spec generates, then each subsequent one filters, so the order you suggest is probably good
2018:05:02 15:11:09 guy ok thanks!
2018:05:02 14:44:34 eoliphant hi I have , I guess a ‘phlisophical’ lol question about spec. Im working on a system that uses datomic, etc. So, I’d gone though and created a bunch of specs, now I’m building out datomic schemas, using my specs to name my attributes, etc. So of course, some of this isn’t especially DRY. I’m just wondering about creating my own (or using umlaut, etc) lightweight DSL that in turn gen’s datomic schemas, specs, etc. Is this the’right’ way or is starting from specs ‘better’, etc. Just wondering
2018:05:08 20:09:48 kenny @eoliphant We wrote a library to do this called Spectomic https://github.com/Provisdom/spectomic and have been using it successfully for about a year now.
2018:05:02 15:08:29 alexmiller some people have done this to various degrees
2018:05:02 15:08:46 alexmiller hard for me to say if they found that to be a win in the long run or not
2018:05:02 15:40:07 robert-stuttaford @eoliphant Datomic schema is not the same as a spec. they do similar things - link behaviour to a name, but they are not the same thing (they create different behaviour). conflate them only if you are confident that in doing so, users of your new combined API understand the consequences of this clearly. semantically, they are different. i find i prefer a little extra typing, and having decoupled systems that use very well documented APIs.
2018:05:02 15:41:35 robert-stuttaford my hobby project https://github.com/robert-stuttaford/bridge keeps them separate, and i don’t think it actually costs anything to do that
2018:05:02 15:43:57 robert-stuttaford so i guess the question is, are you actually Repeating Yourself? 🙂
2018:05:02 15:50:34 triss So I’ve got nice s/fdef’s for my functions. Is it possible to use those in a test?
2018:05:02 15:51:09 triss but when I run tests from CIDER it tells me there a no tests defined.
2018:05:02 15:51:27 robert-stuttaford oh, that’s a different question 🙂
2018:05:02 15:51:49 triss 🙂
2018:05:02 15:55:03 triss So stest/check returns a :clojure.test.check.clojure-test/trial how do I tell clojure.test to look out for it?
2018:05:02 16:25:51 eoliphant yeah that’s kind of the way I was leaning @robert-stuttaford
2018:05:02 16:45:14 javi Hi everyone, new to the channel.
Does anyone know of any articles on best practices /process to spec a pre-existing codebase?
I know spec, but so far haven't used it much and I have a "not-small" cljs codebase that i want to start spec-ing, but not sure where/how to start...
2018:05:02 16:46:22 guy oh sorry right larger codebases sorry no
2018:05:02 16:49:24 javi thanks!!
2018:05:02 16:45:33 alexmiller start by writing specs :)
2018:05:02 16:45:58 alexmiller if you have a map that is used a lot, write specs for the attributes
2018:05:02 16:46:13 alexmiller then write an s/fdef for a function that takes that map
2018:05:02 16:46:46 alexmiller if you want to know where to start, pick the most common/stable data you have
2018:05:02 16:46:52 alexmiller work outward from there
2018:05:02 16:47:13 alexmiller or pick the thing you trust the least and add constraints around it by making specs and using stest/instrument
2018:05:02 16:47:49 alexmiller if you get an instrument failure, you’ve learned something (either your data does things you didn’t anticipate, or something is wrong)
2018:05:02 16:48:00 alexmiller if the former, fix your spec. if the latter, fix your code.
2018:05:02 16:48:16 alexmiller if you have data transformation functions, write specs for the inputs and outputs and use stest/check
2018:05:02 16:48:31 alexmiller these are ideas, not a plan
2018:05:02 16:48:32 javi > or pick the thing you trust the least and add constraints around it by making specs and using stest/instrument
ha ha, good one 🙂 i think i ll start there. thanks!
2018:05:02 16:52:16 robert-stuttaford one very useful practice is to spec anything before you make changes. we have code from clojure 1.6 days that we do this to, and it really helps deal with stuff like regression testing, safe refactoring, etc
2018:05:02 20:18:25 djtango Hey folks, probably a noob question but am really struggling to find an answer via search (probably because of my lack of terminology):
If I define a spec using a non auto-resolved keyword (s/def :my/spec any?) how do I use it if I've required that namespace in a separate namespace? I can't get a call to (s/valid? :my/spec 1) to resolve nor does (s/valid? :: 1)
2018:05:02 20:42:50 alexmiller the first should work
2018:05:02 20:43:09 alexmiller the latter is not valid
2018:05:02 20:43:17 alexmiller not a valid keyword that is
2018:05:02 22:20:33 djtango orz it works now. I think I must have had something else wrong in the file - thanks @alexmiller
2018:05:02 22:39:34 alexmiller Cool
2018:05:03 07:56:46 pyr Let me post my twitter question here as well 🙂
> Clojure library writers, what's your position on spec asserts? Leave it to library consumers to set, leaving it to calling code to> choose whether to assert or not, or enforcing it from the library to ensure malformed payloads always throw?
2018:05:03 11:45:48 joost-diepenmaat do you mean using spec.alpha/assert ?
2018:05:03 11:48:15 joost-diepenmaat probably wouldn’t use that in a library, I’d rely on instrument during dev/test. if you’ve got data coming in that must be validated (say user input or stuff coming in over a network) then you do an unconditional (when-not (s/valid ..) (throw …))
2018:05:03 11:48:41 joost-diepenmaat note that assert can be turned off by changing *compile-asserts*
2018:05:03 11:48:58 joost-diepenmaat and you wouldn’t want that for stuf that must be checked.
2018:05:04 00:52:19 jstew What do I use if I just want to ensure that a fn returns a string or something?
2018:05:04 00:52:36 jstew It wants me to add test.check as a dependency
2018:05:04 01:27:18 alexmiller I think you’re mixing syntax here and that’s leading you down a path trying to gen anonymous functions with check
2018:05:04 01:28:49 alexmiller (s/def ::string-fn (s/fspec :args (s/cat) ret string?)) would be better for this
2018:05:04 01:29:52 alexmiller When function specs are validated they use the generator for the anonymous function to invoke the function being validated and verify the outputs. Using that generator requires test.check
2018:05:04 10:38:00 jstew Ah hah. That makes sense. Thank you, Alex.
2018:05:05 22:56:50 madstap What does it mean to call s/fdef with a keyword instead of a symbol?
2018:05:06 00:23:20 alexmiller You shouldn’t do that
2018:05:06 00:23:52 alexmiller But it’s similar to calling s/def with an s/fspec
2018:05:06 22:18:52 gfredericks https://github.com/clojure/spec.alpha/blob/a65fb3aceec67d1096105cab707e6ad7e5f063af/src/main/clojure/clojure/spec/test/alpha.clj#L286
There's a thing here where spec is relying, for no clear reason, on what I'd like to say is a quasi-buggy implementation detail of test.check that I'd like to change.
So I'm wondering if actually changing that is realistic, given the alphas and the whatnot. It would mean that the next release of test.check is not compatible with the current releases of spec.alpha. And it's hard to think of how to keep that from being a confusing pain to users who upgrade in the "wrong" order. 😕
2018:05:07 11:45:51 alexmiller I would greatly prefer this to change in a growth compatible way (new fn if necessary)
2018:05:07 11:49:52 gfredericks there are enough moving parts that I hadn't thought about something like that, but I might be able to make it work; thanks
2018:05:07 02:09:34 alexandergunnarson So, reached what seems to me to be a bug:
(gen/sample (s/gen (s/and (s/? string?) string?))) -> ([""] [""] ["4"] [""] ["Ii5"] ["fye"] ["x4oa"] ["6i"] ["2LH7cR"] ["Y"])
But it seems that this should fail just as (s/and number? nil?). [""], for instance, is not a string?.
2018:05:07 11:40:33 alexmiller s/? matches collections but conforms to a value and s/and flows conformed values so I think this is the expected behavior
2018:05:07 11:43:36 alexmiller Using s/? as a top level regex op often leads to confusing behavior - this is tricky. Usually it gets nested in s/cat or something
2018:05:07 13:27:56 alexandergunnarson Thanks! Yeah I found that it works with s/exercise so that makes sense at least
2018:05:07 13:35:57 alex-dixon Should I expect spec validation errors to be thrown if I call a function that has a spec registered with s/fdef in CLJS?
I have check-asserts on…working with specs defined with s/def and {:pre (s/valid? ….
2018:05:07 13:38:48 alexmiller have you instrumented it?
2018:05:07 13:39:01 alexmiller with instrument?
2018:05:07 13:40:10 alex-dixon I was confused about that as well. An example I saw used s/instrument from the spec namespace….maybe it moved? Showing as undeclared var
2018:05:07 13:40:48 alexmiller it moved long long ago
2018:05:07 13:41:09 alexmiller it’s in the clojure.spec.test.alpha namespace (in clojure) and parallel in cljs
2018:05:07 13:41:27 alexmiller although I think you can just require that same ns in cljs and it will automatically load the right one
2018:05:07 14:43:14 pfeodrippe Hi, guys, we'are having problems spec'ing protocols, we're implementing the protocols with records, so instead of wrap manually the protocol methods with functions, we're defining a defrecord* macro of the kind
2018:05:07 14:44:04 pfeodrippe Is it a good approach for now?
2018:05:07 15:01:41 alexmiller I think it’s a bad idea to wrap protocol functions (as you then destroy many of the benefits of protocol functions)
2018:05:07 16:45:40 bronsa why? that's the advice I've always seen given (even before spec was a thing) : use protocol methods for polymorphism and a wrapping function for validation + varargs + else
2018:05:07 16:45:58 bronsa unless you're talking about a different "wrapping"
2018:05:07 16:49:34 alexmiller I wouldn’t wrap it purely to use spec
2018:05:07 16:50:12 alexmiller there are some good reasons to wrap a call to a protocol but you don’t necessarily need one
2018:05:07 15:02:16 alexmiller rather, I would recommend just not spec’ing protocol functions
2018:05:07 16:35:43 pfeodrippe Ok @alexmiller, thanks
2018:05:07 18:11:04 alexmiller or rather you can’t for the purposes of instrument
2018:05:07 18:11:50 alexmiller I suppose you could spec them regardless for doc (and maybe check)? I haven’t tried it.
2018:05:07 21:18:27 dominicm @alexmiller I thought you were suggesting it would be a bad idea even if it could be done. I wasn't missing any reason it would be a bad idea 😊
2018:05:09 19:03:25 plins any suggestions where to find (libs maybe?) specs for strings contaning datetime in the ISO 8601 format
2018:05:09 19:13:26 ghadi use java.time @plins
2018:05:09 19:14:47 plins not sure if i expressed my self properly, but id like to validate if a strings conforms to the iso, so
(s/valid? ::iso-spec "2018-05-06T13:14Z") shoud evaluate to true
2018:05:09 19:14:49 ghadi #(try (java.time.OffsetDateTime/parse %) (catch java.time.DateTimeParseException e false))
2018:05:09 19:15:00 plins ok! thanks
2018:05:09 19:15:48 ghadi i wouldn't pull in any of the clj date time libraries, they all have serious defects compared to java.time
2018:05:09 21:07:36 seancorfield @ghadi That's a bit of a sweeping statement. Could you provide a bit more specificity?
2018:05:09 21:08:13 seancorfield (we are using clojure.java-time which is a thin wrapper around Java Time that automates the chains of conversions between (Java) types)
2018:05:09 21:09:58 seancorfield The clj-time readme already tells people not to use it if they're on Java 8 or later -- and recommends Java Time instead, optionally using clojure.java-time as a wrapper.
2018:05:09 21:11:20 ghadi I wouldn't even use a wrapper
2018:05:09 21:12:28 seancorfield Fair enough. I find Java Time's raw classes to be a bit ugly if you're doing anything more than basic stuff. I find clojure.java-time's wrapper much more pleasant for more complex date/time arithmetic/manipulation.
2018:05:09 21:21:20 ghadi clojure.java-time pulls in clj-tuple and potemkin, yuck
2018:05:09 21:21:52 ghadi "white gloves" for interop; mostly field accessors
2018:05:09 21:23:27 ghadi so much error-prone wrapper code... looking under the covers
2018:05:09 21:23:35 ghadi sorry I stand by my statement
2018:05:09 21:24:02 ghadi show me a clojure time library and I'll show you a simpler way to do it with java.time
2018:05:09 21:24:31 seancorfield I don't see it pulling in potemkin. clj-tuple has no dependencies.
2018:05:09 21:25:56 seancorfield My teammate is also a fan of direct Java interop rather than wrappers, but some interop code can definitely be made cleaner with wrappers (in this particular context, as we've seen in code reviews during pull requests).
2018:05:09 21:26:04 mg It doesn’t pull in potemkin, it copies the relevant code to java-time.potemkin.*
2018:05:09 21:26:53 seancorfield Yeah, and I'll take that little bit of potemkin code but I agree that I wouldn't want the whole thing pulled in...
2018:05:09 21:27:08 ghadi the dep issue is minor at best
2018:05:09 21:27:34 ghadi these libraries add nothing to your understanding of the primitives, and introduce new names for everything
2018:05:09 21:27:50 ghadi names were very carefully chosen in java.time
2018:05:10 00:53:00 gfredericks what are the issues with clj-time? just the parallel "adds no value over joda"?
2018:05:10 00:53:36 gfredericks I've been using it for years and haven't noticed anything
2018:05:10 02:15:21 ghadi joda itself had serious design issues that the authors rectified as it became java.time
2018:05:10 02:15:45 ghadi there are a bunch of articles about the api differences of joda vs java.time
2018:05:10 02:29:25 gfredericks coolthx
2018:05:10 02:31:30 seancorfield @U0GN0S72R FWIW, the readme of clj-time points to Joda Time's recommendation to migrate to Java Time and that links to several articles I think.
2018:05:10 02:31:44 gfredericks :+1:
2018:05:09 21:12:59 seancorfield (esp. if you're unfortunately enough to have to convert back and forth with java.util.Date and/or any of the SQL date/time types!)
2018:05:10 10:55:52 triss is there a quick way to flush the slipnet library thing?
2018:05:10 17:07:36 devn What's the rationale for having a spec registry?
2018:05:10 17:18:39 devn The Data spec registration and Function spec registration sections in https://clojure.org/about/spec get at it
2018:05:10 17:18:54 devn guess I was wondering if you'd add any additional color
2018:05:10 17:19:24 alexmiller not a lot to add beyond that
2018:05:10 17:21:47 alexmiller there was a desire to not keep shoving stuff into var meta
2018:05:10 17:21:50 devn @alexmiller I wondered "Vars themselves are a kind of registry, no?"
2018:05:10 17:21:52 alexmiller or vars
2018:05:10 17:21:54 devn gotcha
2018:05:10 17:22:07 alexmiller with good names, we are not limited to a single “var” registry
2018:05:10 17:22:56 alexmiller if you use vars, you need namespaces to hold them
2018:05:10 17:23:23 alexmiller with a separate registry, it’s perfectly ok to make a spec named :a.b.c.d.e.f.g/foo without having a related namespace
2018:05:10 17:24:03 alexmiller Rich has some future ideas on managing names and aliases independently from namespaces
2018:05:10 17:36:29 devn this pleases me greatly
2018:05:10 17:37:25 devn we have a spec aliases macro for calling alias + create-ns that we use, and while it works, it doesn't have good mouth feel
2018:05:10 17:38:19 alexmiller yeah, that’s a hack. we have aims to do better.
2018:05:10 23:05:40 olivergeorge This hurts most in clojurescript. Do you think it will be supported in the new approach?
2018:05:11 00:23:58 alexmiller Probably but I don’t know what it is yet :)
2018:05:11 02:52:47 olivergeorge Can't say fairer than that. I look forward to specs becoming less verbose and quicker to write.
2018:05:10 17:38:24 devn there's also the potential for the "oops, someone clobbered a spec name and didn't know about it" situation
2018:05:11 19:12:12 cap10morgan Anyone know why the default generator for (s/map-of keyword? string?) would take 10+ seconds to generate 1,000 samples?
2018:05:11 19:15:41 ghadi it's generating large maps. set (s/map-of .... :gen-max 2) to constrain sizes
2018:05:11 19:16:17 cap10morgan Ah, cool. I hadn't looked for that option in the spec fn itself. Thanks @ghadi.
2018:05:11 19:16:37 ghadi (doc s/every) will show some others
And I’d like to force zipcode and state to be consistent (i have a zipcode->state function so if they are both present, they need to match)
How can I write my ::zipcode and ::state specs?
-- is it how it is supposed to look like, or am I missing some shorthand?
2018:05:12 12:35:23 dottedmag ::header, ::delimiter etc are declared using s/cat themselves, so I suppose s/tuple can't be used here.
2018:05:12 12:37:43 dottedmag Maybe there is a simpler way to spec this data format? The input is a CSV file parsed into a vector of vectors. Header is a number of lines, more-or-less fixed, but some are optional and may be omitted. Delimiter is a single fixed line. Txs headers is another set of more-or-less fixed lines, and txs is the rest of the document.
2018:05:12 13:16:45 alexmiller s/tuple might be better
2018:05:12 13:17:48 alexmiller For vector of fixed position fields
2018:05:12 13:18:39 dottedmag Too bad they are mostly, but not completely, fixed. Banks are, well, banks.
2018:05:12 13:22:31 ghadi I would turn the vec of vecs into a vec of maps, then spec with s/keys
2018:05:12 13:29:22 dottedmag Parsing individual lines is not so bad — they can be handled by s/tuple. Parsing the sequence of lines is worse.
2018:05:12 13:29:46 dottedmag I'll leave it with s/cat for now.
2018:05:13 18:17:57 andy.fingerhut Would adding the links you gave for Cognitect blog articles on spec, and the videos, to one or both of the spec rationale and spec guide articles be of interest?
2018:05:13 18:24:58 alexmiller Nah, most of it is also in the guide
2018:05:13 18:24:19 andy.fingerhut I have read about some operation in spec doing 'sampling' in its validation/checking of a value against the spec, for better performance (but also, obviously, less precise checking). Is there something in the spec documentation I can look for to remind me where this occurs?
2018:05:13 18:25:25 alexmiller s/every and s/every-kv
2018:05:13 18:30:53 andy.fingerhut Thanks. I see those in the spec guide, and good to see the doc strings for s/every and s/every-kv mention that partial checking property.
2018:05:14 06:12:46 gklijs Should be doable at least with cursive I think, it already has a check for having the correct arity.
2018:05:14 13:46:42 jmayaalv what’s the best way to get the keys descriptions for a s/keys :req) form? calling (s/describe) returns a (keys) but not sure how can it be used.
2018:05:14 14:09:12 jmayaalv got it. managed to do what i wanted this way:
2018:05:14 23:08:51 jimbob How can i check against a collection that only certain values of maps in the collection are unique? i.e. that the collection of maps is valid iff all maps in the collection have distinct values for at least one of two fields.
ex:
2018:05:15 04:21:40 nenadalm your datastructures are not valid (parenthesis do not match)
2018:05:15 09:33:47 guy I think you would write a spec that for those values would have to be unique
2018:05:15 09:34:28 guy (s/def :a/unique-field #{"unique 1" "unique2"})
2018:05:15 09:34:56 guy so then the spec for that map would include that field and that spec above and would only allow those values
2018:05:15 09:35:27 guy Or do you mean the maps are only valid when, regardless of the values, they have to all be unique?
2018:05:15 15:43:35 jimbob the first one yes.
2018:05:15 15:55:51 jimbob so mostly just curious if there are ways to spec collections of things, but spec them in terms of the collection, not in terms of the individual items in the collection (so whether or not only certain fields for the items are unique, or that summing fields for all items is within a certain range, etc)
2018:05:15 16:03:06 ghadi custom predicates with s/and is the common mechanism for doing this
2018:05:15 20:24:42 lilactown so I'm trying to do some work where I validate a JSON blob against a swagger spec. I'm thinking about trying to translate the swagger spec into clojure.spec and do it that way
2018:05:15 20:24:58 lilactown basically the opposite of what spec-tools does with it's swagger stuff
2018:05:15 20:25:28 lilactown however, it seems like there's an assumption that clojure.spec's are registered globally with a name and everything. I would be generating these specs dynamically, on the fly
2018:05:15 20:25:34 lilactown is this a good idea?
2018:05:15 20:25:45 hiredman any predicate is a spec
2018:05:15 20:26:24 hiredman so if you have a function that given a swagger spec and some data returns true or false, then you just partial that with the swagger spec and you have a spec
2018:05:15 20:26:55 hiredman it isn't a very nice spec, it won't generate and you won't get vary granular error messages, but it is a spec
2018:05:15 20:28:15 lilactown okay. I was thinking of trying to use clojure.spec to do some of the heavy lifting around e.g. checking whether a map conforms to a set of expected values
2018:05:15 20:30:46 lilactown but obvi this is made more difficult by the fact that this would need to be done for arbitrary number of swagger specs at runtime, so I would have to make sure I'm not clobbering things in the registry...
2018:05:15 20:32:17 guy out of interest why couldn’t you model the data in spec? would it really be dynamic?
2018:05:15 20:33:17 guy I’m probably using the wrong definition of dynamic, but i thought you would know what sort of data you would be getting? so you could potentially spec out the shape of the data?
2018:05:15 20:36:32 lilactown I'm writing some code that you give it a URL to a swagger.json, a URL path, example request and an example response
2018:05:15 20:36:49 lilactown and it validates that the URL path, request and response are valid based on the swagger.json
2018:05:15 20:36:53 guy ohhh i see
2018:05:15 20:37:08 guy so you can give it basically anything, and then you use the swagger to create a spec
2018:05:15 20:37:19 lilactown that's the idea. not sure it's a good one
2018:05:15 20:37:25 guy it sounds pretty cool 🙂
2018:05:15 20:37:38 guy i see ur worries about the registry now
2018:05:15 20:37:54 guy i guess you could test it by creating multiple unique specs and seeing how many you could create
2018:05:15 20:40:16 lilactown the whole thing seems gross. I wish you didn't have to register them 🙃
2018:05:15 20:45:42 seancorfield @lilactown Would it be easier/better to have something that generated spec source code from a swagger URL? After all, you only want to generate the spec once per API end point, but you'll want to validate data against that spec repeatedly...
2018:05:15 20:49:07 hiredman it seems like it would be a better idea to use some swagger validator to validate the swagger, for a swagger validation service
2018:05:15 20:52:58 hiredman I think the direction you see in spec-utils (spec -> swagger) makes sense because the idea is, you can consolidate different ways of specifying data (database schemas, swagger, json schema, etc) as spec specs, and then generate (or using spec to make sure you data matches the contract) those other things from spec as needed. But it doesn't seem like turning swagger in to specs, if all you care about is validating arbitrary swagger specs against arbitrary data, makes a lot of sense
2018:05:15 20:59:18 lilactown yeah I guess I'm looking for a swagger validator, and I decided to try and build my own using clojure.spec? and that sounds like it's not a good idea
2018:05:15 21:02:18 lilactown I'm actually struggling to find such a thing that exists, which is why I'm trying to build one in the first place
2018:05:15 21:03:01 guy If you are doing it for fun, why not give it ago and see how it goes
2018:05:15 21:03:07 guy You’ll defo learn something
2018:05:15 21:03:37 lilactown eh it's not exactly for fun 😅
2018:05:15 21:04:00 lilactown it happens to be fun but also, sprint ends in a week and a half
2018:05:15 21:04:55 guy ah well thats a different matter, if ur doing it for work, i might try and find a different solution
2018:05:15 21:11:42 hiredman googling java swagger validator and jvm swagger validator both seem to turn up good leads
2018:05:15 21:58:52 lilactown @U055NJ5CC scjsv is exactly what i'm looking for! 😄 thanks!
2018:05:16 09:54:08 troglotit using (s/def ::my-spec ..) - is not a hard requirement. Everytime your keys are not about namespaces in your application - i think it’s better to qualify by another namespace like (s/def :swagger.employee-schema/employee ..)
2018:05:18 02:28:05 vlad is it possible to evaluate spec description?
2018:05:18 02:29:26 vlad I woud expect Smaller than 1000 in the description instead
2018:05:18 02:43:03 vlad the following one works, but I want to see this in the explanation
2018:05:18 02:43:07 vlad
(-> (s/form ::small-integer) :description eval)
2018:05:18 03:02:39 alexmiller this is not supported
2018:05:18 05:27:22 mpenet It's easy to hack a (simple) meta registry to do that without abusing the internals of clj.spec.
2018:05:18 05:28:01 mpenet I did that and a few other horrors here if you re looking for inspiration https://github.com/mpenet/spex
2018:05:18 05:28:59 mpenet Look for spex/with-doc in the readme
2018:05:18 07:52:09 mpenet gist of it:
(-> (s/def ::foo any?)
(spex/with-doc "That's a foo"))
(spex/doc ::foo) => "That's a foo"
2018:05:18 13:55:11 sundarj there's a coll-of but no corresponding one-of; i can go from a scalar spec to a collection spec, but i don't think i can go from a collection spec to an element spec. is that intentional?
i'm wondering because i'm currently speccing a text editor, and it's going to have a collection of working documents as well an active document, which would naturally be one of the working documents
2018:05:18 13:59:20 mpenet seems like you just need a set
2018:05:18 13:59:31 mpenet or I am misunderstanding
2018:05:18 14:01:11 sundarj i presently have this:
2018:05:18 14:01:32 sundarj but it feels wrong because active-note isn't just a random note, it's only ever going to be one of the working notes
2018:05:18 14:05:44 alexmiller this would be easier if the working-notes was a set (not sure why it isn’t) but you could s/and an additional check into editor that working-notes contains active-note
2018:05:18 14:07:51 sundarj it's not a set because the active-note is going to be pulled from the working-notes, and sets would be the wrong data structure for that, right? sets are just about membership, as far as i know
2018:05:18 14:08:38 sundarj i hadn't thought about putting an s/and on editor, that makes a lot of sense. thanks 🙂
2018:05:18 14:17:24 alexmiller you’re declaring in working-notes that its elements are distinct. that’s the same guarantee sets give you but they can check membership in (near) constant time rather than via a linear search
2018:05:18 14:20:41 sundarj yeah, i know. i had the :kind as set? before, but then i realised you couldn't pull arbitrary members out of a set; there's going to be a UI that lists all the working notes, and then you click on one to open it. i think a vector is a better choice given that use-case. i suppose i could have a vector and a set, but then i'd have to keep them in sync
2018:05:18 14:22:33 alexmiller what do you mean by “you couldn’t pull arbitrary members out of a set” ? it seems like you can do exactly that
2018:05:18 14:22:51 alexmiller do you mean you need to retain ordering?
2018:05:18 14:29:46 sundarj i meant that to pull an item out of a set you need the item itself; i thought you couldn't iterate over a set, but i realise i was mistaken. in any case, i think i do need to retain ordering
2018:05:18 14:47:38 alexmiller and that’s a good reason to use a vector, not trying to necessarily change your mind, just probing
2018:05:18 14:54:12 sundarj i understand; i appreciate the questions. it did make me more fully analyse why i wanted it to be a vector, so thanks 🙂
2018:05:20 15:28:56 karol.adamiec in clojurescript/javascript when i get json on the wire with UUID as a string, how should i spec that? uuid? passes even with empty string, (but has a correct genereator). I want the spec to be used as a tool for dev time to quickly catch misbehaving API’s. Also want to generate sample data for unit tests. (ie javascript harness reads a file generated by simple clojurescript spec gen and runs a suite of tests for each generated input. i could swap / regenerate entry fixtures as i see fit…)
2018:05:22 13:07:51 troglotit Hey! I want to spec for five characters long numbers: I could use regexes #(re-matches #"\d{5}" %), but I as far as I can understand that could be done in spec. I looked up a bit, but what I found is only s/* s/+ s/? that kind of operators. So, is there that kind of spec? I know about s/cat - but that seem too overkill for when I just want to repeat my regex
2018:05:22 13:09:45 ghadi while they may work for characters of sequences, I wouldn't use s/* s/+ etc. for that
2018:05:22 13:10:15 ghadi if you're parsing strings, a normal regex is more appropriate
2018:05:22 13:10:40 troglotit Oh, I found (s/every :count) - seems like what I wanted
2018:05:22 13:11:38 ghadi like i said, it's a bad idea
2018:05:22 13:12:19 ghadi the regex predicate you have up there is the better approach
2018:05:22 13:13:31 troglotit I want to have more composable regexes and spec seem suited for that, compared to regular regexes
2018:05:22 13:16:33 alexmiller spec is not designed to be a good string regex parser and is unlikely to perform or work well for that use case as actual regex or a parser
Is there a way for me to specify that when I do (gen/generate (spec/gen ::root)) that every group ID for every student has an equivalent group ID under the ::groups key?
2018:05:23 07:00:43 flowthing In other words, I'd like every group ID in ::students to have a corresponding entry in ::groups. I know how to spec that, but I'm wondering what the best way to write a generator like that is.
2018:05:23 07:01:09 flowthing I could always just post-process the generated map, but for more complicated maps, that becomes a bit difficult.
2018:05:23 07:26:00 andy.fingerhut I've not used this before, but have heard that test.check has something called bind that lets you generate one value randomly, e.g. in your case perhaps a set of group IDs, and then use that set to generate other things, e.g. your ::groups map and also the ::students map, which could be generated to have exactly the same set of group IDs in both.
2018:05:23 10:36:01 gfredericks @flowthing you can generate the groups first, and then (using bind) generate students where their group-id is from (gen/elements (map :id groups))
2018:05:23 10:47:09 flowthing Many thanks for the tips! I'll give gen/bind a go.
2018:05:23 10:47:55 gfredericks @flowthing gen/let can do the same thing, and you may find it more intuitive
2018:05:23 10:48:43 flowthing Yeah, I've tried gen/let actually, but I haven't yet come up with a clean solution. Nonetheless, it's good to know I'm heading in the right direction.
2018:05:23 15:18:29 lwhorton hey peoples, i’m not really sure the term/noun for what i’m trying to do with spec, so it’s hard to google… I want to define a spec where an id field that’s present in two places is guaranteed to be the same:
{"id" {:id "id" :other-stuff true}}
2018:05:23 15:18:52 lwhorton I’m assuming I have to write a custom generator to enforce this?
2018:05:23 15:24:02 guy I think you would have to have a predicate that for that map, it checks that key and the :id value is the same.
And i believe yes, you would have to have a custom generator for it too.
2018:05:23 15:24:40 guy Like i don’t know how you would do it with s/keys for example
2018:05:23 15:25:05 guy but saying that, there might be another way
2018:05:23 15:25:10 alexmiller yes and yes
2018:05:23 15:26:17 lwhorton yea i’m not sure how to handle it either, i might just scrap this gen test entirely.. like how do you s/map-of ::id ::some-spec with also ::some-spec (s/keys :req [::id]), then write a generator that builds out two(?) specs. maybe i have to merge the map-of?
2018:05:23 15:28:36 guy yeah let me find an example soz
2018:05:23 16:12:16 rickmoynihan So I’ve written some specs in an existing project with a clojure.test test suite.
Does anyone have any recommendations on incorporating generative tests using clojure.spec.test.alpha/check with clojure.test?
Obviously I could do:
2018:05:23 16:13:38 rickmoynihan A while back I had a similar issue and tried hooking into the clojure.test assert macros but it was never very satisfactory
2018:05:23 16:14:02 rickmoynihan is there a lein test runner for fdefs?
2018:05:23 16:15:25 rickmoynihan actually just remembered I’ve been here before… and have already implemented a lein alias in this project for running specs using instrument etc…
2018:05:23 16:19:27 lwhorton i wrote a little helper for your first problem..
(defn passes?
"A light helper function to evaluate a generative/property test so that
clojure.test spits out the failing result (if any) for easier debugging."
[spec]
(let [res (spec)]
(t/is (nil? (:shrunk res)))
(t/is (true? (:result res)))))
2018:05:23 16:20:22 lwhorton i’m not sure if that’s too implementation-specific with checking shrunk, but it works for me and i didnt have to dig around docs for a long time
2018:05:23 16:22:02 rickmoynihan @lwhorton what are the arguments?
2018:05:23 16:23:17 rickmoynihan obviously it’s a function… but I guess I should ask how do you use it?
2018:05:23 16:23:25 lwhorton ah, it’s just like you would use testing or is/are:
(defspec my-spec my-prop-test)
(deftest some-test
(testing "some generative test should pass"
(passes? my-spec))))
2018:05:23 16:26:37 rickmoynihan ahh ok — that’s not strictly spec though is it?
2018:05:23 16:27:20 rickmoynihan Isn’t that a property test defined with test.check? Rather than through s/fdef etc
2018:05:23 16:29:06 rickmoynihan I appreciate spec builds on test-check and that they’re compatible to some degree; but will that pick up an fdef:fn spec etc
2018:05:23 16:29:29 lwhorton yea there’s a few moving pieces, the spec itself is defined using clojure.test.check.properties and a generator that uses spec/gen
(defspec first-element-is-min-after-sorting ;; the name of the test
100 ;; the number of iterations for test.check to test
(prop/for-all [v (gen/not-empty (gen/vector gen/int))]
(= (apply min v)
(first (sort v)))))
2018:05:23 16:31:49 rickmoynihan yes I’ve seen that before; but I’m not sure it’s what I need — e.g. if you had a spec defined like this from the clojure docs:
2018:05:23 16:32:52 lwhorton ooph, i’m not sure man sorry 😕 … im just getting into the gen testing myself
2018:05:23 16:33:04 rickmoynihan that spec is already defed so to speak; as it’s metadata is registered already with the system.
2018:05:23 16:33:22 rickmoynihan @lwhorton: 🙂 no worries; it’s subtle stuff
2018:05:23 16:34:33 rickmoynihan I have used defspec before… and tbh it seems like it’s a better way to integrate with clojure.test right now
2018:05:23 16:36:51 rickmoynihan @lwhorton in the interests of sharing I have something like this that works as a different entry point i.e. run via a lein alias:
2018:05:23 16:38:07 rickmoynihan it uses st/instrument to enable all the fdef specs and run them through st/check
2018:05:25 16:18:31 kvlt Hey all, I'm seeing something I didn't expect. Could anyone explain this?
2018:05:28 02:31:16 alex-dixon Is there a way to access why a spec passed?
2018:05:28 02:38:14 gfredericks isn't that what conforming is?
2018:05:28 03:02:56 alex-dixon Not sure. The situation that prompted that question is a multi spec that matched on a particular :type value of a map and spec associated with that value. I can tell that it passes and get the original map back, just curious if there’s a way to access explain information when it passes. May be interesting stuff that could be done with it but seems only to get generated upon failure
2018:05:28 13:45:03 alexmiller explain info is only being built up when using explain, so no?
2018:05:29 01:36:03 callum I'm finding myself having to duplicate spec definitions in order to override generators and control the number of generated values. e.g.
(s/def ::name string?)
(s/def ::a (s/coll-of ::name))
(s/def ::b (s/keys :req-un [::a]))
(s/def ::c (s/keys :req-un [::a
::b]))
;; only generate a single element for ::a
(gen/generate (s/gen ::c {::a #(s/gen (s/coll-of ::name :min-count 1 :max-count 1))}))
2018:05:29 01:36:33 callum Is there a way to avoid this duplication?
2018:05:30 12:43:17 raymcdermott I’m reading the results of evaluation via the PREPL which gives back a string for the :val key
2018:05:30 12:44:34 jumar I'm not familiar with prepl so don't know then
2018:05:30 12:46:58 raymcdermott I’m trying to avoid having to run eval on every input
2018:05:30 12:47:12 raymcdermott before I pass it to the REPL
2018:05:30 12:47:41 raymcdermott but yes, that’s a legitimate option @jumar
2018:05:30 12:50:29 raymcdermott @alexmiller I’m using the PREPL from 1.10 and it would be nice (IMHO) if there was a property for spec fails
2018:05:30 13:41:12 alexmiller I’m a little confused by what you’re saying. In the case of an exception, I would expect prepl to be converting the Throwable to data with Throwable->map which captures ex-data in a :data attribute.
2018:05:30 13:55:11 alexmiller trying some things in the repl, I understand what you’re saying I think, although I don’t understand why
2018:05:30 13:57:12 alexmiller oh, I’m testing with io-prepl and it’s happening due to the default valf of pr-str - is that what you’re using @raymcdermott?
2018:05:30 13:59:01 alexmiller if you use a valf of identity, then you’ll get back throwables as data
2018:05:30 14:07:21 alexmiller
Clojure 1.10.0-master-SNAPSHOT
user=> (require '[clojure.core.server :as ccs])
nil
user=> (ccs/io-prepl :valf identity)
(defn foo 1)
{:tag :ret,
:val {:cause "Call to clojure.core/defn did not conform to spec:\nIn: [1] val: 1 fails spec: :clojure.core.specs.alpha/arg-list ...",
:via [{:type clojure.lang.Compiler$CompilerException,
:message "clojure.lang.ExceptionInfo: Call to clojure.core/defn did not conform...",
:at [clojure.lang.Compiler checkSpecs "Compiler.java" 6891]}
{:type clojure.lang.ExceptionInfo,
:message "Call to clojure.core/defn did not conform ...",
#_here :data #:clojure.spec.alpha{:problems (
{:path [:args :bs :arity-1 :args],
:pred clojure.core/vector?,
:val 1,
:via ...
`
2018:05:30 14:19:11 raymcdermott thanks @alexmiller I’ll give it a shot later
2018:05:30 21:01:03 raymcdermott @alexmiller I could be doing it wrong but I’m using io-prepl as part of a socket-server like this…
2018:05:30 21:03:21 raymcdermott I then have a web server to intermediate
2018:05:30 21:04:05 raymcdermott And this takes request from a REPL client in the browser
2018:05:30 21:04:33 raymcdermott which send over strings that are then read and sent to the remote prepl
2018:05:30 21:06:33 raymcdermott Read failclojure.lang.LispReader$ReaderException: java.lang.RuntimeException: No reader function for tag object
1
2018:05:30 21:08:38 raymcdermott is this an abuse? Is there another option to read that is more compatible?
2018:05:30 21:13:54 raymcdermott I’ll keep fiddling but ideas welcome 🙂
2018:05:30 21:18:01 raymcdermott Read fail: clojure.lang.EdnReader$ReaderException: java.lang.RuntimeException: No reader function for tag object
2018:05:30 21:25:59 alexmiller I haven’t put all these pieces together recently and don’t have time to look at it today. you might look at wrapping clj-server/remote-prepl for the client side?
2018:05:30 21:26:55 alexmiller and since stuff is mostly char stream based, you might just want the defaults in io-prepl but to read on the remote-prepl valf side, not sure
For client written in Clojure I guess you should try remote-prepl as Alex suggested, but your problem seems to be related to the fact that read is not able to recognize #object in the response returned by remote repl.
The error looks is caused by sth. like this:
For client written in Clojure I guess you should try remote-prepl as Alex suggested, but your problem seems to be related to the fact that read is not able to recognize #object in the response returned by remote repl.
The error looks is caused by sth. like this:
2018:05:31 12:35:46 raymcdermott I didn’t know about that var, so thanks ….
2018:05:31 12:35:48 raymcdermott I really must look at all of those ear-muff things one day if I’m going to live down here 🙂
2018:05:31 13:00:00 alexmiller This is interesting and probably ultimately needs a change in spec reporting. Rather than send the spec instance, it should be sending the spec name or form where you are seeing the object. For now, I think you’ll need to intercept either on the server or client side and either remove or transform it yourself
2018:05:31 22:29:58 raymcdermott do you consider this as a spec issue rather than a PREPL issue? Or a bit of both?
2018:05:31 22:51:00 alexmiller spec for the reporting
2018:05:31 22:51:55 alexmiller Maybe prepl although it gives you the hook to address it
2018:05:31 13:01:52 alexmiller I think hooking the default data reader to fall back to using tagged-literal is a pretty good dodge that covers many potential issues like this
2018:05:31 13:31:52 Andreas Liljeqvist How would I alias a namespace that isn't required? Circular dependency is a problem if I require it
2018:05:31 13:33:38 Andreas Liljeqvist I want to keep a separate namespace/file for my spec but use another namespace
2018:05:31 13:34:43 Andreas Liljeqvist Like my-namespace with functions and my-namespace.spec with the spec
2018:05:31 13:35:30 Andreas Liljeqvist But using :t/id instead of having to write :my-namespace/id
2018:05:31 13:42:08 guy So do you mean like.
Inside of my-namespace you have some spec like
(s/def :t/id) instead of (s/def ::id) or (s/def :my-namespace/id)
2018:05:31 13:42:53 guy As far as i understood you have to require the namespace the spec is in, to use it in a different namespace. But i could be wrong
2018:05:31 13:44:10 guy So you might want to just move your spec somewhere else to get around the circular dependencies?
I ended up just giving certain specs fake namespaces to and not using ::id. Instead something like :user/id
2018:05:31 13:45:41 Andreas Liljeqvist in my-namespace.spec I have a (s/def ::id nat-int?) . Resulting in :my-namespace.spec/id
2018:05:31 13:46:26 Andreas Liljeqvist I would like the spec to remain in my-namespace.spec but be registered to :my-namespace/id
2018:05:31 13:46:50 guy try this instead (s/def :my-namespace/id nat-int?)
2018:05:31 13:47:18 Andreas Liljeqvist That will work, but it feels wrong when I have a lot of defs
2018:05:31 13:47:32 guy You will still need to require my-namespace.spec in a different ns if you want to use the spec contained
2018:05:31 13:47:43 Andreas Liljeqvist yeah
2018:05:31 13:47:48 guy but then you can refer to it like :my-namespace/id instead
2018:05:31 13:48:52 guy I think my counter argument to "but it feels wrong when I have a lot of defs" would be its a bit redundant to have .spec in the spec name
2018:05:31 13:49:13 guy :my-namespace.spec/id vs :my-namespace/id
2018:05:31 13:49:23 guy like you know they are both specs
2018:05:31 13:49:29 guy so is .spec really needed?
2018:05:31 13:49:42 guy (this is just my opinion though, so someone else feel free to correct me etc)
2018:05:31 13:49:56 Andreas Liljeqvist Only reason for the .spec is that I want to have a separate namespace for the specs vs the functions
2018:05:31 13:50:30 guy Yeah i had user and user.spec too, because it felt nicer to separate them
2018:05:31 13:50:38 guy yeah exactly the same thought process
2018:05:31 13:50:56 guy But i ended up moving away from :: notation to something/id instead
2018:05:31 13:51:56 Andreas Liljeqvist A bit worried about keyword collisions when doing it that way, but everything you say makes sense
2018:05:31 13:52:30 guy I think thats more a case of you having a consistent naming convention really
2018:05:31 13:52:40 guy >keyword collisions
2018:05:31 13:52:41 guy i mean
2018:05:31 13:52:43 Andreas Liljeqvist I would have prefered to do something like (alias 't 'my-namespace)
2018:05:31 13:53:04 guy The troubles i was having were similarly named fields
2018:05:31 13:53:13 guy user/name vs something else /name
2018:05:31 13:53:29 guy so if u had a (s/def ::name) that was a string and one that was something else
2018:05:31 13:53:36 guy it would cause headaches
2018:05:31 13:53:56 guy if that makes sense?
2018:05:31 13:54:12 Andreas Liljeqvist Yeah, but I might be using a third party library that also implement :user/name or something
2018:05:31 13:54:34 Andreas Liljeqvist But that is a problem for future me 🙂 Thanks a lot for the help
2018:05:31 13:54:40 guy sure but then you can prefix to to Andreas.user/name or Projectname.user/name etc 👍
2018:05:31 13:54:47 guy nps! Good luck!
2018:05:31 13:54:53 guy Hopefully i was helpful haha
2018:05:31 14:00:34 Olical Afternoon. I was wondering if anyone else uses sets with their (s/keys...) calls?
2018:05:31 14:01:07 Olical In the docs it mentions vectors, but wouldn't a set of keys make more sense, like: (s/keys :req-un #{::a ::b})?
2018:05:31 14:01:30 alexmiller conceptually, yes
2018:05:31 14:01:35 Olical I ask because I've seen a weird issue where UUIDs are not being coerced by spec-tools if I use a set 😅
2018:05:31 14:05:37 Olical You can reproduce it with
2018:05:31 14:06:26 Olical If you replace the set with a vector it works fine and coerces the UUID string into an actual UUID. I was thinking "maybe this is an issue with spec-tools", but if you're supposed to use a vector anyway this is technically okay, just a confusing gotcha.
2018:06:01 13:25:26 alexmiller I’d use s/alt there instead of s/or as it’s a regex op, but that’s one option
2018:06:01 13:25:47 alexmiller Another is to push the optionality into the cat with s/?
2018:06:01 13:26:35 bostonaholic s/alt is better, thanks
2018:06:01 13:29:21 bostonaholic when I run my suite with lein test and I get the generator seed “Using generator seed: -1186346283”, how do I pass that seed into lein test again?
2018:06:01 13:39:38 alexmiller I don’t know of any way to do that from that level
2018:06:01 13:39:50 alexmiller stest/check takes an option map where you can pass in a seed
2018:06:01 13:40:57 alexmiller you might be able to do it through some test.check dynvar or something, not sure
2018:06:01 16:16:13 kapil Is there an idiomatic way to associate a doc-string with a spec?
2018:06:01 16:21:23 kapil I am trying to generate documentation from spec for non-clojure developers to read.
Something like this
(s/def ::id string? "Id of resource X. Ex. resource_1")
(s/def ::num integer? "Number of results for pagination")
(s/def ::result (s/keys :req-un
[::id ::num]))
(gen-doc ::result) =>
{:id "resource_1" ;; "Id of resource X. Ex. resource_1"
:num 10 ;;"Number of results for pagination"
}
2018:06:01 16:38:20 alexmiller no, not yet but we intend to add that
2018:06:01 18:37:05 kapil @U064X3EF3 Is there any workaround I can do for now?
I tried using with-meta but it doesn't with no success since keywords are not Objects.
I could however maintain a registry of documentation.
2018:06:01 18:43:15 alexmiller there are some external libs that are providing this, but it’s hard to recommend those as what they are doing is likely to break
2018:06:01 18:43:39 alexmiller you could build an external map of spec keyword to doc string
2018:06:01 18:47:20 kapil This is what I am using now. This is least intrusive change I could think. Later on when clojure.spec adds support for documenting specs I can just modify def-w-doc.
2018:06:01 18:59:11 alexmiller sounds good
2018:06:01 16:16:50 noisesmith for the seed, one workflow would be to keep your randomized check, and also do another with a specific seed that creates a known rare corner case
2018:06:01 16:18:10 bostonaholic SEED=<seed> lein test is how to do it
2018:06:01 16:18:49 noisesmith oh that works? interesting
2018:06:01 16:19:17 noisesmith I like to have regression tests for things that have been known to fail in the past though
2018:06:01 16:19:57 noisesmith (I guess it's better to do that by hard coding the data that had been generated in an example based test)
2018:06:01 16:25:09 bostonaholic yup, setting the SEED as an env var works
2018:06:02 13:47:26 gfredericks FWIW, I have never heard of this and have no idea what's going on
2018:06:02 13:47:38 gfredericks my suspicion is it's not a test.check seed, it's something else
2018:06:01 18:21:39 rapskalian I'm curious what others' tastes are regarding namespaced keys in spec definitions? As an example, let's say we're modeling GPS coordinates. Personally, I've been doing something like this:
In which lat and long are both "nested" under the :coordinate namespace, and then the coordinate map itself is defined inside of some kind of :my-project.specs top-level namespace (using the ::coordinate sugar).
Does this make sense, or are there other conventions that have been established that would be clearer?
2018:06:01 18:42:03 seancorfield @cjsauer That's pretty close to my approach. Much depends on how unique you need the names to be and how your code might be used.
2018:06:01 18:43:04 seancorfield In a library, you'd really want all the names to be globally unique. In an application, you can use less unique names.
2018:06:01 18:44:10 alexmiller they should be sufficiently unique :)
2018:06:01 20:58:38 rapskalian @seancorfield good to hear. This is for an application, so I'll trade some uniqueness for readability 🙂
2018:06:02 19:19:15 cvic If I add
[tentacles.core :as c]
lein repl throws
Exception in thread "main" clojure.lang.ExceptionInfo: Call to clojure.core/defn- did not conform to spec:
In: [0] val: clj-tuple/conj-tuple fails spec: :clojure.core.specs.alpha/defn-args at: [:args :name] predicate: simple-symbol?
#:clojure.spec.alpha{:problems [{:path [:args :name], :pred clojure.core/simple-symbol?, :val clj-tuple/conj-tuple, :via [:clojure.core.specs.alpha/defn-args :clojure.core.specs.alpha/defn-args], :in [0]}], :spec #object[clojure.spec.alpha$regex_spec_impl$reify__2436 0x162e29a1 "
No idea why...
2018:06:02 19:29:03 cvic Or... some dependency that has a malformed defn- form.
2018:06:02 19:29:12 cvic Time to do some digging
2018:06:02 19:35:46 cvic Ah, nvm. I'll just use
[irresponsible/tentacles "0.6.2"]
2018:06:02 22:22:22 alexmiller The line in error is clj_tuple.clj line 556
2018:06:02 22:22:43 alexmiller Wherever that is
2018:06:03 18:16:30 cvic Sometimes upgrading a lib is the shortest way of solving it
2018:06:05 04:33:51 Vincent Cantin Hi. I plan to try to use spec for auto-completion. Has it been done before?
2018:06:05 04:41:08 Vincent Cantin Are the specs easy to parse /navigate programmatically?
2018:06:05 06:48:24 ikitommi @U8MJBRSR5 lot of parsers in 3rd party libs, using s/form. As a sample, one parser and a spec visitor in spec-tools (https://github.com/metosin/spec-tools).
2018:06:05 05:20:38 Vincent Cantin … I think that s/explain-data does a lot already.
2018:06:05 10:16:00 Vincent Cantin Is there some kind of s/seq-of operator, or a way to build it?
The idea is to act as a bridge to spec the chars in a string from the string itself: (s/conform (s/seq-of char?) "abc")
2018:06:05 10:35:59 hkjels I don’t have a running repl atm, but I think you can use (coll-of char? :kind string?)
2018:06:05 10:41:54 Vincent Cantin It works, thanks. Need to add :into [] to keep the chars order at conform time.
2018:06:05 11:02:49 Vincent Cantin I still have trouble afterward … I would like to run some regex spec (e.g. s/cat) on the content of a string but without having to apply s/conform on the seq of that string. I don’t know if it is possible.
2018:06:05 14:22:34 bbrinck FWIW, the general advice I’ve seen is that spec is not a great fit for parsing strings. Certainly people have done it by turning the string into a sequence with seq, but often people run into further issues. More detail in comments here: https://www.reddit.com/r/Clojure/comments/7vwpu4/parsing_with_clojurespec/
2018:06:06 02:48:31 bbrinck “we recommend that you not use conformers for coercion. Conformers were added primarily as a tool for building custom composite spec types”
2018:06:06 03:00:34 bbrinck I know people have had success with using spec for parsing strings, just wanted to make you aware of possible issues down the road 🙂
2018:06:06 03:04:03 Vincent Cantin Ah, Christophe again :-)
2018:06:06 00:08:50 noisesmith is there a way to get the individual spec keywords mapped to by an s/or call?
2018:06:06 00:11:04 noisesmith looks like describe gets me most of the way if there's nothing better
2018:06:06 11:37:13 alexmiller s/form will be more complete than s/describe
2018:06:06 21:24:59 arohner Is there a way to get conformed output to be different for non-regex specs? I have a bunch of strings I need to parse, and I’d like to validate a parsed string against the spec, and returned the parsed values, if they conform
2018:06:06 21:44:45 alexmiller in short, no
2018:06:06 21:46:02 alexmiller you would need a custom spec impl for that currently and I wouldn’t recommend that as the guts are likely to change
2018:06:06 21:46:25 alexmiller in the future, maybe
2018:06:06 23:48:57 seancorfield @dpsutton I have seen some occasional bugs reported against spec that seem to have that behavior -- I guess you'll need to dig into that :fhir-type/PaymentReconciliation spec and see what it's doing?
2018:06:06 23:49:52 dpsutton It kinda sorted itself out but no idea what the transient state was. Just wondering if it was known bug or if we were way off the beaten trail
2018:06:06 23:52:14 seancorfield Ah, a Heisenspec? 🙂
2018:06:06 23:52:56 dpsutton I'm guessing it's more simple and just something silly we are doing. But you never know
2018:06:07 00:25:10 alexmiller Generally anything like this should be considered a bug (if it’s reproducible). One thing that could possibly lead you down this path at the repl is changing a spec but not redefining a spec that used it. Specs are compiled at definition time so you can get a mixture of things that way.
2018:06:07 02:08:02 dpsutton Oh. I made a poor man's version of spec using defs and thought you got around that with the central database of specs and always getting a "fresh" copy. I'll make sure to have up to date definitions. Thanks
2018:06:07 02:49:39 Vincent Cantin What is the most idiomatic way to apply s/cat, knowing that it is a macro?
2018:06:07 12:33:55 alexmiller A future way to do this will be to write a spec for s/cat, write conform data, and s/unform back to a spec.
2018:06:07 12:34:16 alexmiller See CLJ-2112 for some early work on that
2018:06:07 02:50:26 noisesmith to apply a macro, you need another macro (and to do it at runtime to something that is not a literal in your source file or created when loading the file you need to be compiling new code, eg. eval)
2018:06:07 03:06:25 Vincent Cantin I will use a macro then.
Do you know why it was made as a macro? (most importantly, why only as a macro)
2018:06:07 03:07:43 Vincent Cantin That would have been good to have the macro and the functions and let the users choose between them.
2018:06:14 19:56:28 johanatan Completely agree. The prevalence of macros in spec is my number one complaint with it.
2018:06:07 03:07:57 Vincent Cantin s/cat and sm/cat
2018:06:07 03:17:25 Vincent Cantin “macro contagion” … it reminds me of the “const hell” in c++
2018:06:07 03:42:38 seancorfield @vincent.cantin spec was designed for humans to write, so macros are fine. There is work in the pipeline to open up programmatic access to spec, which will mean functions.
2018:06:14 19:54:13 johanatan Macros are not fine. They prevent higher-order code and partial application (to name a couple of issues). And of course they result in 'macro proliferation' as well.
2018:06:07 03:43:16 seancorfield So, for now, you might be going through a lot of (macro) pain for nothing, depending on how quickly you need the thing you're trying to build...
2018:06:07 03:45:39 Vincent Cantin The task I gave myself is probably making me a complainer by design: I am trying to do the most I can do with Spec.
2018:06:07 03:46:14 Vincent Cantin You are right, it is not related to the normal use case.
2018:06:07 03:46:14 seancorfield Including lots of things it wasn't designed for, I'm sure 🙂
2018:06:07 03:47:44 seancorfield It's interesting... over time, the Clojure/core folks have produced a number of things that in the rush of new excitement around a feature, lots of people almost immediately try to do stuff with each new feature that's outside the design goals 🙂
2018:06:07 05:40:58 Vincent Cantin @U04V70XH6 I was doing that: https://github.com/green-coder/TIS-100-Spec
My problem with the cat being a macro was for the implementation of my token spec.
2018:06:07 05:42:41 Vincent Cantin I wanted to see if it was possible to do it that way. Well … it is possible, but not convenient.
2018:06:07 05:44:38 seancorfield And how is performance? 👀
2018:06:07 05:44:54 Vincent Cantin eh eh .. I don’t know, it does not matter here.
2018:06:07 05:46:04 Vincent Cantin I wished that cat could accept nil instead of a token for specs which we do not need to know about when we use conform.
2018:06:07 05:46:47 Vincent Cantin in regex terms, it would mean a “non-capturing” expression.
2018:06:07 05:52:33 seancorfield I'm wondering how many times folks have to say "spec is not a parser"... ? :rolling_on_the_floor_laughing:
2018:06:07 05:53:49 Vincent Cantin 😄 that means that the community would like to use it in that way too.
2018:06:07 05:54:24 seancorfield If you get the grammar right and each spec generates then you have the possibility of generating random but syntactically valid TIS-100 programs... that you could then run and see what they do! 🙂
2018:06:07 05:54:47 Vincent Cantin You start to see some of my goals
2018:06:07 12:05:21 dottedmag However spec is definitely sold as a parser, maybe unintentionally. "Check your inputs and, oh here is the parse tree".
2018:06:07 12:05:54 dottedmag It's not a best string parser out there, that's for sure.
2018:06:07 03:48:26 seancorfield I think it says a lot about how the features have been such great building blocks over the years, as well as how laser-focused many of the features have been.
2018:06:07 03:49:14 seancorfield I don't know if you've done much with clj and deps.edn yet? That seems to be attracting the same "let's see what crazy things we can do with this" experimentation.
2018:06:14 19:57:28 johanatan I don't consider wanting to apply or partially apply a macro to be a "crazy experimentation". It's very much just a thing that advanced [meta-]programmers want to do to keep their code concise.
2018:06:07 16:58:22 favila I can't find a ticket for this, but I can't believe it wasn't noticed before
2018:06:07 16:59:06 favila s/every or s/coll-of will interpret :kind as a predicate only (not spec) during s/valid?s/conform etc
2018:06:07 16:59:25 favila but during explain, it does interpret as a spec
2018:06:07 17:00:24 favila I think it is because the macros generate a cpred unconditionally interpreting kind as a pred
2018:06:07 17:00:32 favila example of how this can go wrong:
2018:06:07 17:03:40 dpsutton in checking against the spec, it will call (::vector []) which returns nil and fails the spec. In explaining, it is aware that [] satisfies the ::vector spec
2018:06:07 17:04:29 favila Note that :cpred is generated but nonsense
2018:06:07 17:05:41 favila docs say
2018:06:07 17:05:54 favila > :kind - a pred/spec that the collection type must satisfy, e.g. vector?
(default nil) Note that if :kind is specified and :into is
not, this pred must generate in order for every to generate.
2018:06:07 17:06:09 favila note "pred/spec"
2018:06:07 17:12:34 favila ok, found the ticket, nm
2018:06:07 17:51:33 bmaddy I'm trying to write a spec for a function that takes in another function. I have the spec for the function passed in, but am getting "Couldn't satisfy such-that predicate after 100 tries.". Is that because I need to write a generator for functions that are arguments?
2018:06:07 18:15:10 noisesmith such-that acts as a filter, it's saying it wasn't finding anything that the filter accepted
2018:06:07 18:27:38 alexmiller how is it spec’ed?
2018:06:07 20:03:48 bmaddy (sorry, was at lunch)
Here's how I have it spec'd:
(s/def ::ruleset-id keyword?)
(s/def ::rule-name keyword?)
(s/fdef ::fetching-fn
:args ::ruleset-id
:ret (s/coll-of ::rule-name))
(s/fdef rules-for-rulesets
:args (s/cat :fetching-fn ::fetching-fn
:ruleset-ids (s/coll-of ::ruleset-id))
:ret (s/map-of (s/and set? (s/coll-of ::ruleset-id))
(s/and set? (s/coll-of ::rule-name)))
;; Ensure each rule only appears in the values only once so we
;; don't run them multiple times
:fn #(->> % :ret vals (reduce concat) frequencies vals (every? (partial = 1))))
2018:06:07 20:33:31 taylor it might be the s/ands where the first predicate is set?
2018:06:07 20:34:56 taylor could you try changing that (s/and set? (s/coll-of ::the-spec)) to (s/coll-of ::the-spec :kind set?) @bmaddy
2018:06:07 20:35:32 taylor (because s/and makes a generator from whatever the first predicate is, and set? is unlikely to produce values that will satisfy ::ruleset-id)
2018:06:07 20:39:02 taylor oh, another potential issue: ::fetching-fn's :args spec should probably be (s/cat :whatever ::ruleset-id) instead of just ::ruleset-id?
2018:06:07 20:39:44 bmaddy trying it...
2018:06:07 20:43:26 bmaddy Oh geez, I think it's the :args (s/cat ... thing. picard-facepalm
2018:06:07 20:43:57 taylor yeah I just realized the s/and issue probably has nothing to do with the such-that issue
should be a wrapped type: '(list type) or '(non-null type)
2018:06:07 21:58:51 bbrinck @hlship Ah, yes, I’ve run into this as well. The problem is that defmsg is narrowly applied to predicates, not any type of spec.
2018:06:07 21:59:32 hlship
Associates the spec named `k` with `error-message`.
doesn't make that clear. Hm.
2018:06:07 21:59:42 bbrinck Agreed, and I don’t think it’s a good design
2018:06:07 21:59:52 bbrinck It’s a reflection of the original problem defmsg solved: trying to provide more “human-friendly” messages instead of string?int?
2018:06:07 22:00:13 bbrinck but that’s an arbitrary restriction, and I think it should be generalized to any spec, so you can add messages across the board
2018:06:07 22:00:50 hlship I'm trying to think of a work-around.
2018:06:07 22:01:39 bbrinck I haven’t tried it, but what happens if you replace #{'list 'non-null} with (fn [x] (contains? #{'list 'non-null} x)?
2018:06:07 22:02:09 bbrinck (I realize it’s a hack 😞 )
2018:06:07 22:03:17 hlship Perhaps you could tweak the logic to treat a set as a predicate
2018:06:07 22:03:52 bbrinck Even more generally, I think I should make sure to respect a registered message regardless of the spec type
2018:06:07 22:06:35 bbrinck frankly, I wasn’t sure anyone was really using Expound’s capability to register messages, so I wasn’t sure if it was really important. Glad to know you are using it 🙂
2018:06:07 22:07:48 bbrinck I’ve recently gone down a rabbit hole of trying to generate specs to test Expound but it’s taken up a lot more time than expected. I’m going to pause that effort and switch over to bug fixing for a bit
2018:06:07 22:11:27 hlship I'm very glad you implemented those messages, I think it can make a major difference. In general, I've gotten the rest of the team addicted to Expound ... we use a fair amount of clojure.spec, but previously, spec errors were treated as a boolean — they were so hard to parse, we just looked at code changes to figure out how to make them go away. Now we have a proper guide to exactly what's wrong.
2018:06:07 22:44:02 bbrinck I’m on the fence as to whether this would be useful enough to warrant inclusion
2018:06:07 22:45:24 hlship Well, as a library developer, I want expound to be an optional dependency, so I don't think I could use this.
2018:06:07 22:48:19 bbrinck Oh sorry, I was unclear in my description on this issue: it works like messages today in the sense that there is no effect if someone isn’t using expound.
2018:06:07 22:48:44 bbrinck The only difference is that instead of registering a static string, you can register a function which will be called to generate the string
2018:06:07 22:49:56 bbrinck This would let you, say, augment the existing expound error message, or use the problem to craft a custom message. But perhaps in your use case, the static strings are sufficient.
2018:06:08 17:44:47 martinklepsch Is there a way to find out from what namespace/file a spec has been defined?
2018:06:08 19:54:42 seancorfield I'd be very interested in hearing an answer to this too!
2018:06:08 17:58:18 bmaddy Here's a spec puzzle for someone. I'm trying to get a spec that defines something like this:
([-7806 "rBQUP"] :_T-7/G0X)
I thought I would need to nest calls to s/cat to do it, but those seem to flatten or something:
How do I get a collection of two items, where the first is a collection of one int and one string, and the second is a keyword?
2018:06:08 18:19:48 taylor like hiredman said below, this should work: (gen/generate (s/gen (s/cat :pair (s/spec (s/cat :int int? :str string?)) :keyword keyword?))) just to prevent the "flattening" of the nested regex specs
2018:06:12 08:14:39 djtango late to the party, s/tuple isn't a regex spec so you can nest it as you like:
2018:06:08 18:09:01 hiredman wrap (s/spec ...) around the inner cat I think
2018:06:08 18:14:01 hiredman nested regex specs sort of collapse into a single regex spec (nested cats become subsequences within the outer sequence) so you have to insert something to tell spec where to stop collapsing them, and I think it is wrapping with s/spec, but it has been a while
2018:06:08 19:37:55 bmaddy That worked! Thanks @hiredman and @taylor! 😄
2018:06:12 07:36:55 kurt-o-sys I'm using cljs-spec but I have issues with #objects...:
Call to #'... did not conform to spec:
In: [0 :data :from] val: #object[b 2018-06-11T00:00:00.000+02:00] fails spec: :week.data/from at: [:args :arg-0 :data :from] predicate: vector?
:cljs.spec.alpha/spec #object[cljs.spec.alpha.t_cljs$spec$alpha40486]
:cljs.spec.alpha/value ({:data {:from #object[b 2018-06-11T00:00:00.000+02:00]}})
:cljs.spec.alpha/args ({:data {:from #object[b 2018-06-11T00:00:00.000+02:00]}})
:cljs.spec.alpha/failure :instrument
so, what I try to spec is #object[b 2018-06-11T00:00:00.000+02:00].
2018:06:12 07:37:48 kurt-o-sys how to spec that from-field?
2018:06:12 08:18:20 djtango how robust do you want the spec? Does that date object contain a "type" field? If you want to check if the from field only is valid, you could pass it back into the date library to parse the string to check it's a valid time stamp?
2018:06:12 08:24:05 kurt-o-sys there is no type-field (apparently) - I'm using a js-library for datetime handling (need timezones, and there are not many options).
so, checking if the from field is a valid datetime would do...
2018:06:12 08:24:49 djtango what's consuming the date object further downstream?
2018:06:12 08:25:59 djtango If it's internal to your project, Is it good enough to spec it to be valid only for the code you know is going to use it?
2018:06:12 08:26:07 kurt-o-sys other functions that do operations on it, like .plus etc...
2018:06:12 08:26:25 kurt-o-sys yeah, that's good enough...
2018:06:12 08:27:12 kurt-o-sys for dev, I just want to be sure I get an kind of datetime object, on which I can perform .plus and .minus operations (or something like that).
2018:06:12 08:27:58 kurt-o-sys Now, I'm using any?, but that too generic 🙂.
2018:06:12 08:29:10 djtango ah, fair. You could store your own representation of the date object as a map, then use the library only for doing manipulations (then translate it back to your date representation)
2018:06:12 08:31:58 kurt-o-sys right... could do. that will be quite a lot of translations - not sure about performance impact here. But I may just try to do a .plus-operation on it and check if it works, or something similar. That may do as well... will try. Thx!
2018:06:12 08:35:58 djtango good luck
2018:06:12 08:57:39 kurt-o-sys works fine, like it:
2018:06:12 13:08:04 mpenet cross posting here since it got no answer in #clojure
2018:06:12 13:16:08 alexmiller If you use s/def with a resolvable symbol, the symbol itself is the spec registry key (this is his function specs are stored). So, not a bug, and matches the docs afaict
2018:06:12 13:21:27 mpenet not sure I follow, do you have an example?
2018:06:12 13:30:05 alexmiller maybe I misunderstand what you are asking about, it’s not clear to me what you think the bug is
2018:06:12 13:31:40 alexmiller the purpose of resolvable symbol k in that docstring is to support registering function specs named by their resolvable symbol. the example given is doing something other than that, which is not what it is for
2018:06:12 13:33:16 mpenet ok:
2018:06:12 13:33:17 mpenet (def x string?)
(s/valid? x "sdf")
2018:06:12 13:33:20 mpenet something like this then
2018:06:12 13:33:31 mpenet yeah that makes more sense
2018:06:12 13:33:49 alexmiller no, not that
2018:06:12 13:34:00 mpenet I still don't get it then 😛
2018:06:12 13:34:24 mpenet since we're talking about the first arg to s/def it's a bit odd
2018:06:12 13:34:26 alexmiller x here should be a symbol referring to a var that is a function, to declare a function spec for x
2018:06:12 13:37:14 alexmiller the first is a shorthand for the second
2018:06:12 13:37:26 mpenet oh ok
2018:06:12 13:37:49 alexmiller like, literally s/fdef is a macro that does that
2018:06:12 13:38:23 alexmiller the registry has keys that are either keywords (pointing to data specs) or symbols (for function specs on the function on that symbol)
2018:06:12 16:19:38 kvlt I have written a couple of fdef specs for my functions. What is the best way to test that in a test file? I'm looking at defspec and prop/for-all but the path forward doesn't seem obvious
2018:06:12 16:21:25 taylor You can instrument the function to assert arguments conform to the :args spec on each call. You can also check functions to assert that it's returning valid values for randomly generated inputs.
2018:06:12 16:22:27 taylor there's also exercise-fn
2018:06:12 16:29:08 kvlt @U3DAE8HMG: Thanks for the response.
Are you suggesting something along the lines of:
2018:06:12 16:32:29 taylor that should work, although you probably want more than 1 test, and I believe there's a helper function to make the stest/check result easier to examine
2018:06:12 16:36:55 kvlt The problem with this is that it won't give me any information about why it failed
2018:06:12 18:10:08 bbrinck @U08UTJ5PB maybe something like the following (untested)?:
2018:06:12 16:32:50 kvlt I've been looking at that, it's still not clear to me how I run these in an automated test environment
2018:06:12 17:20:48 bertofer I have a message coming from an API in a format like '(a b c d e f), and I’d like to propagate that message through a channel to my program with a proper format in a map, with keys for each of the properties that come in the list.
I am using spec to conform the message, so I have something like (s/def msg (s/cat :a number? :b number?…)) and then when I call conform I do have a map like {:a a, :b b, :c c, …}, which is the format I want already (a map). Okay now I have a format I like to pass it on to the rest of the program, except I’d prefer to have namespaced keys, but s/conform does not seem to put namespaced keys even if they are specified as keys or values in the s/cat. Is there any other way to make this happen? Is this a valid approach to “transform” messages from the outside world to the inside language? It is really helpful as it is, but I am not sure it is the preferred way to do it. What other way then? How do you usually do it?
Thanks!
2018:06:12 18:02:58 taylor s/conform + s/cat does seem to preserve qualified keyword tags for me:
2018:06:12 18:17:18 bertofer Oh, you’re right, it does for me too. I got confused because proto-repl prints it out like #:bitfinex-clj.spec.data{:ts 1, :open 2, :close 3, :high 4, :low 5, :volume 6}, but I have checked and they are namespaced.
2018:06:12 18:27:16 taylor ya that's just the shorthand for when all the keys in the map share the same namespace
2018:06:12 18:34:00 bertofer Another question: imagine I have all the namespaced keys already as predicates previously defined, so I can reuse them. Anyway to make this to not repeat each key? (s/def a (s/cat ::a ::a ::b ::b …))
2018:06:12 18:35:02 taylor the only solution that comes to mind is writing a macro that outputs a s/cat like you want it, but before I did that I'd probably reconsider doing the transformations outside spec, maybe before/after conforming
2018:06:12 18:37:37 taylor this seems like it might lend itself more naturally to a s/keys spec, and you could just handle the transformation before you conform (using zipmap or something)
2018:06:12 18:40:28 bertofer hm…So the thing is the messages can come in different shapes and I use spec and s/or to tell me which type of message it is. But I probably can, instead of conform, to just check which message comes in without keeping the transformation, and then do the transformation outside. On the other hand, it seems so much related that it feels everything should be together (the same keys used by s/cat and zipmap, etc…), also because conform already does the transformation I want
2018:06:12 18:43:36 bertofer Anyway, I’ll have a rethink about the current structure I am using. Thanks @U3DAE8HMG 🙂
2018:06:12 18:48:50 taylor np, here's a macro that I think does what you want:
2018:06:14 12:01:25 alexmiller set! will only work on a dynamic var if you are in the dynamic scope of a binding call
2018:06:14 12:01:54 alexmiller Repls like clojure.main establish this
2018:06:14 12:02:47 alexmiller So you either need to use binding or modify the root value with alter-var-root
2018:06:14 12:09:21 borkdude clear. Is there also something like *print-length* but for strings?
2018:06:14 12:11:23 alexmiller No
2018:06:14 12:14:40 alexmiller You could override the print-method for String, although that might have some adverse consequences
2018:06:14 12:15:20 borkdude probably not a good idea 🙂 the reason I ask is that a spec error (with expound) flooded my emacs buffer so bad, I had to forcequit it…
2018:06:14 13:04:58 bbrinck @borkdude although it elides potentially useful information, you can configure expound to not print out “failing spec” info, which would likely shorten the output considerably.
2018:06:14 13:45:12 alex-dixon Is there an idiomatic way to check if a spec exists?
2018:06:14 13:46:59 alex-dixon i.e. given a namespaced keyword how to check if a spec is registered for it
Yeah. Working on something where a macro may receive a namespaced keyword with an associated spec…so hopefully this all works out at compile time. Unfortunately have to do actual work now. Thanks a lot for your help…was going to hack something in but I feel way better with this
2018:06:14 20:03:55 noisesmith given that macro expansion is a simplification step before making bytecode, you either end up with a much more complex compiler or taking the huge performance hit of having an interpreted language rather than a compiled one
2018:06:14 20:04:06 noisesmith (in order to have first class macros)
2018:06:14 20:04:52 johanatan @noisesmith i'm not arguing for first-class macros (although that would be nice). i'm arguing for not using second-class macros in libraries
2018:06:14 20:04:58 johanatan and forcing their proliferation onto users.
2018:06:14 20:05:29 noisesmith oh, I misinterpreted you then
2018:06:14 20:06:28 johanatan [sorry, yea, it wasn't clear from without the context]
2018:06:14 20:27:27 rapskalian Can I generate specs en masse? Something like:
2018:06:14 20:39:58 taylor it's registering the specs to the literal symbol k
2018:06:14 20:40:43 rapskalian This was my gut feeling of what was going on. Any way to "force" it to evaluate the keyword? At this point I can see the value of writing them out by hand, but still curious...
2018:06:14 20:40:57 rapskalian Smells like a macro...
2018:06:14 21:17:24 rapskalian @U04V4HWQ4 interesting...using eval does do exactly what I'm after. It's a bit of a hack, but it beats having to repeat these massive lists of keywords over and over again...
2018:06:14 21:20:30 donaldball It would be nicer if spec had an explicit affordance for this use case, but I don’t really think it’s abusive to take advantage of the fact that clojure is a lisp 😛 . I wouldn’t reach for this often, mind, but in my case I had a gigantor table of fields for a structured file and this lets me leverage an edn representation for spec validation as well as actually e.g. generating serializers
2018:06:14 21:23:19 rapskalian Very cool, I agree that there are definitely valid use cases. Not to mention that the prospect of generating specifications from concretions is extremely interesting in its own right 🤔
Appreciate the help (@taylor as well, thank you)
2018:06:14 20:27:38 rapskalian For some reason I'm getting "unable to resolve spec"...
2018:06:14 20:28:55 johanatan @cjsauer most (all?) attempts to meta-program spec will fail-- it's apparently intended to be hand-written in the first-order only.
2018:06:14 20:39:25 rapskalian generates all specs directly in emacs buffer and calls it a day
2018:06:14 20:39:32 rapskalian I ❤️ Clojure
2018:06:14 20:46:02 seancorfield @johanatan as noted, spec is currently built for human written code/input. There are plans for a more programmatic API which will open up meta-programming. I think building the latter first would have led to a lot more breaking changes (since folks would have built code against the API). Make sense?
2018:06:15 06:05:34 ackerleytng what's the difference between gen/hash-map and just using gen/fmap (fn [a] {:a a}) ...?
2018:06:15 14:33:04 bbrinck @ackerleytng hash-map can take multiple key-value pairs
2018:06:15 14:34:35 bbrinck (whereas in your example, the function you pass to fmap only takes a single value)
2018:06:15 18:53:50 kenny Is there a way to only check the keys explicitly written in a Spec? i.e. No implicit validation of keys. I have a case where I need to validate partial pieces of a larger map.
2018:06:15 18:58:49 noisesmith @kenny when would spec implicitly validate your key?
2018:06:15 18:59:18 kenny Any map implicitly validates all keys against existing specs.
2018:06:15 18:59:31 noisesmith select-keys before validating?
2018:06:15 18:59:44 kenny Could totally do that but then I need to write the set of keys twice.
2018:06:16 17:23:06 alex-dixon If I have a spec that calls conform on data that may have been specced by other users, should I expect to receive their conformed results in my conformed results?
2018:06:17 22:58:31 taylor if their specs are loaded into the spec registry in your process, I think so
2018:06:17 08:46:34 misha @kenny you can get it fairly easily from s/form
2018:06:18 11:26:35 zclj If I would like instrument to check my specs, what would be the preferred way of spec:ing a multimethod? Can I use a multispec with the same dispatch fn or do I need an indirection fn with a spec in each defmethod?
2018:06:18 11:48:48 alexmiller You can make the defmulti dispatch function explicit and spec that
2018:06:18 17:58:40 zclj In my case each defmethod gets called with a map and the map should have different contents for each method. As I read your answer it would be the same spec for all methods?
2018:06:18 12:10:12 misha @kenny I wonder if qualified symbols can be replaced like this:
2018:06:18 12:10:50 misha given you already required [clojure.spec.alpha :as s]
2018:06:18 12:11:26 misha then, getting rid of alpha would be just 1 line :require update
2018:06:18 14:39:44 valerauko I recall that spec was inspired in part by RDF. Is there any recommended way for describing XML structures?
2018:06:18 14:51:34 alexmiller xml represented by what data structure?
2018:06:18 14:53:00 valerauko I'm using clojure.data.xml, so whatever that uses internally.
2018:06:18 14:53:58 valerauko Its sexp-as-element is the closest I've seen it get to simple data structures, but even that seems to be pretty complex to spec
2018:06:18 14:55:23 valerauko I'm afraid I don't follow
2018:06:18 14:55:41 alexmiller there are at least two formats there - the input and output of sexp-as-element. they both seem pretty straightforward to spec if you want to
2018:06:18 14:56:38 valerauko the output's clojure.data.xml.Elements, right?
2018:06:18 14:56:58 alexmiller Element is a defrecord, so you can just spec it as a map with 3 known keys
2018:06:18 14:57:57 alexmiller the sexp form is just structured vectors and can be spec’ed with s/cat etc
2018:06:18 14:59:10 alexmiller oh, there actually are specs already in data.xml
2018:06:18 14:59:56 alexmiller that’s for the element forms
2018:06:18 15:02:25 valerauko it's not in the stable release yet though, is it?
2018:06:18 15:03:50 valerauko i see you bumped its clojure dependency up to 1.9 when specs were introduced
2018:06:18 15:04:51 valerauko as you pointed out i can just spec them as maps with :tag:attrs and :content
2018:06:18 18:02:17 lwhorton if I want to declare a with-gen ::spec generator-fn, generator fn needs to be a fn that returns a generator. how does one create a generator that’s simply a function, without using fmap or any of the other test.check.generators?
2018:06:18 18:02:52 lwhorton for example I just want a “generator” that invokes (random-uuid) where the fn itself is already random and doesn’t need any randomness from the generator scaffolding
2018:06:18 18:05:36 lwhorton i had some garbage like
2018:06:18 18:09:19 guy thats using gen/return to always return nil
2018:06:18 18:09:37 guy I'm not entirely sure its what you are looking for though 😞
2018:06:18 18:10:25 lwhorton that seems like the same idea as abusing fmap and an existing gen (int-in), but with different fns. it just feels like there’s a built-in fn like gen/return for this use case, right?
2018:06:18 21:13:21 alexmiller using external sources of randomness prevents shrinking and repeatability so is discouraged
2018:06:18 22:45:41 gfredericks if you want to do the discouraged thing, fmap will work and return won't.
2018:06:19 09:41:24 mpenet you can mix both to limit the awful (g/fmap rand-uuid (g/return nil)) I think
2018:06:19 09:41:37 mpenet but it's still bad
2018:06:19 14:36:57 lwhorton i see. so if you aren’t using the generator’s api test.check can’t work its magic?
2018:06:19 17:08:08 favila test check needs to control all sources of entropy
2018:06:19 17:08:54 favila (if you want the other features of generators eg strinking and repetition to work)
2018:06:19 17:08:54 alexmiller @lwhorton no, test.check uses the same api. the issue here is using external sources of randomness
2018:06:19 17:10:35 favila gen/uuid already exists, is that not ok for you?
2018:06:19 20:45:33 lwhorton oh, i didn’t notice that. i will check it out. my real gotchya was that i have an instance? check against cljs.core/UUID and wasn’t sure how to make that happen randomly.
2018:06:19 22:09:19 gfredericks The builtin uuid generator should only generate instances of that type
2018:06:20 05:41:01 olivergeorge I'm wondering how to avoid a REPL gotcha related to instrumentation. Since stest/instrument wraps existing functions you effectively lose instrumentation by redefining a function with defn at the REPL.
2018:06:20 05:42:08 olivergeorge Re-running stest/instrument fixes this but would be annoying to do manually every time.
2018:06:20 05:43:54 olivergeorge How do you avoid losing instrumentation at the REPL?
2018:06:20 07:27:47 sundarj @olivergeorge define a defn+instrument macro?
2018:06:20 08:11:01 olivergeorge An instrumenting version of defn would be perfect but ideally it'd be seamless.
2018:06:20 12:15:10 gfredericks what tooling are you using? I'm often editing the files and running the tools.namespace refresh function, which means I can either have the call to instrument at the bottom of a given namespace, or else use the callback hooks in refresh
2018:06:20 12:16:09 gfredericks related, what's the purpose of instrumentation? just for running tests, or for interactive development?
2018:06:20 12:21:44 olivergeorge Interactive development. Just starting on a re-natal mobile companion app to a clojurescript re-frame app.
2018:06:20 12:23:03 olivergeorge Figwheel has an "always reload" option for namespaces but a simple cursive repl is the case I was considering.
2018:06:20 12:24:59 gfredericks I had the idea of monkeypatching defn, but I dunno how to accomplish that exactly in cljs
2018:06:20 13:23:09 olivergeorge No immediate joy but I'll give it some more time later.
2018:06:20 14:28:07 gfredericks you'd have to somehow get that to run in the cljs compiler process
also I realized it's possible that defn isn't even a macro in cljs; it could be a special form
2018:06:20 16:20:25 noisesmith I assume that do was meant to be doto?
2018:06:20 16:22:10 gfredericks I don't think so
2018:06:20 16:22:23 gfredericks I mean it definitely wasn't, and I believe it's right that way
2018:06:20 16:22:54 gfredericks I think st/instrument takes a symbol, not a var
2018:06:20 16:34:39 noisesmith oh, so the intention is to no longer return the var but instead return whatever instrument returns?
2018:06:20 16:35:43 noisesmith I am probably weird for doing this, but I have a repl workflow where I count on being able to call load-file and then (*1) to call the last var in the file
2018:06:20 16:42:35 noisesmith anyway all I was getting at was that the original return value of defn was being dropped
2018:06:20 16:59:09 gfredericks Oh good point. Need a let
2018:06:20 12:29:52 gfredericks as long as it only runs in dev mode then it's not the worst thing in the world 🙂
2018:06:20 12:58:03 olivergeorge Thanks I'll give that a try
2018:06:20 13:20:05 olivergeorge I might be able to do something similar via a custom command in cursive. Looks like I can send something like this to the repl with a hotkey (untested):
~top-level-form
(instrument ~current-var)
2018:06:20 13:22:13 olivergeorge Another approach would be using a patched version of clojurescript for development. (Must try and see how hard that is now that deps.edn does github shas)
2018:06:21 04:26:19 alex-dixon Does anyone know if there’s a proposal to add line numbers to spec error messages?
2018:06:21 12:28:32 alexmiller There should be line numbers included now for macro function spec errors
2018:06:21 12:30:00 alexmiller I think there was a ticket filed recently, or at least it came up, for the same with non macro function spec errors
2018:06:22 15:09:39 mpenet @alexmiller It seems spec might get some love given some jira activity. I know there are already a lot of things queueing on that, but do you think there's a chance we might get anything to allow to get relations between specs (like list all aliases for a registered spec, list all specs ancestors depending if it's a s/merged of others etc etc)?
2018:06:22 15:09:57 mpenet it's quite easy to add, but just curious if this is on the radar at all
2018:06:22 15:28:08 alexmiller Deep walk is of interest but I don’t think we will look at that until after Rich’s next batch of impl changes
2018:06:22 18:22:11 bmaddy has there been any indication on what the next batch of impl changes might be focused on? Even if only a general description?
2018:06:22 19:41:33 alexmiller Better for data-oriented functional construction
2018:06:22 19:42:16 alexmiller Due to other priorities for Rich, I think that will slip out a bit but we’re going to try to do some bug fixing in near term
2018:06:23 16:01:37 ackerleytng what's the difference between s/* and coll?
2018:06:23 16:01:53 ackerleytng like if i do (s/def ks coll?), is that the same as (s/def ks (s/* any?))?
2018:06:23 16:22:28 alexmiller s/* is a regex op and will compose with other regex ops to describe the internal structure of a sequential collection
2018:06:23 16:22:47 alexmiller If you don’t have any structure, I’d use coll-of
2018:06:24 04:59:48 ackerleytng thanks!
2018:06:24 11:00:33 martinklepsch I would like to find out where a spec has been defined (source file, namespace etc.) — is this possible?
2018:06:24 12:44:35 ackerleytng hmm sorry not sure about that...
2018:06:24 13:04:32 alexmiller Some specs retain meta, but generally no. There is a ticket about this
2018:06:24 13:20:14 ackerleytng I'm trying to test a function where the inputs are related
2018:06:24 13:20:27 ackerleytng it's actually a bit like update-in
2018:06:24 13:20:54 ackerleytng where ks has to be related to m
2018:06:24 13:22:01 ackerleytng I wrote a generator to generate m, and i want the result of that generator to be inputs to a function that returns another generator for ks
2018:06:24 13:22:33 ackerleytng those already mostly work, but how would i supply gen-overrides in pairs?
2018:06:25 12:56:33 ackerleytng I got it!
2018:06:25 12:56:54 ackerleytng i'm now generating all the arguments to the function as a single list
2018:06:25 12:57:45 ackerleytng another question! is there any "state" that carries over between instances of generators?
2018:06:25 12:58:46 ackerleytng say i define (def string-gen (gen/string))
2018:06:25 12:59:35 ackerleytng if i use that in two different overrides, say {::foo (fn [] string-gen)} and {::bar (fn [] string-gen)}
2018:06:25 13:00:06 ackerleytng would the two overrides generate strings independently of each other?
2018:06:25 13:06:46 guy I think the generators have seeds? which might be different each time?
2018:06:25 14:26:25 ackerleytng ah yes i guess so
2018:06:25 14:26:56 ackerleytng along those lines - will a more complex generator retain any sort of state between two invocations?
2018:06:25 14:32:08 favila a top-level generator instance and all its "sub" generator instances share a common source of randomness; it's not shared with other instances
2018:06:25 14:32:33 guy unless you pass it a seed or something? i think
2018:06:25 14:32:39 guy I can’t quite remember
2018:06:25 14:32:43 guy let me google it 👍
2018:06:25 14:32:57 favila yeah but even then you're regenerating the same randomness independently
2018:06:25 14:33:00 favila it's not shared
2018:06:25 14:33:05 guy ahhhh
2018:06:25 14:33:08 guy yes you are correct
2018:06:25 14:33:13 guy 👍
2018:06:25 14:33:49 favila so if your overrides are part of a larger generator (e.g. you are generating examples for a top-level map and it has these two keys you override) then the generators share the same source of randomness
2018:06:25 14:34:05 favila if these are independent uses of these generators, they do not share
2018:06:25 14:40:05 ackerleytng hmm then why do the overrides have to be wrapped in a function?
2018:06:25 14:41:09 ackerleytng it has to be {::foo/bar (fn [] (gen/return inc))} instead of just {::foo/bar (gen/return inc)} right?
2018:06:25 14:41:52 ackerleytng does wrapping it in a function kind of force a new generator to be returned or something? to avoid state carrying over?
2018:06:25 14:51:12 favila @ackerleytng wrapping in a function is to allow lazy invocation of the entire generator system; It's possible to use spec without a test.check dependency (e.g. in production) if you don't use any of the generation features.
2018:06:25 15:00:52 ackerleytng ah i see, thanks!
2018:06:25 14:51:44 favila if you didn't do this, spec would always require test.check even if you only wanted to use e.g. s/valid? s/conform s/unform etc
2018:06:25 14:52:35 favila this is also why clojure.spec.gen.alpha exists: it wraps all the test.check functions and generators so that test.check itself is never required unless actually used
2018:06:25 18:29:42 blance do you usually define generator together with your spec when the only use case for the generator is writing generative tests? It's nice that you can combine them with with-gen, but on the other hand I don't know if it's right to define something in src that's only needed for testing
2018:06:25 19:02:47 noisesmith @blance consider that a generator for foo will be needed for another library that uses foo, and test files are not transitive like source files are
2018:06:25 19:03:20 noisesmith so you end up hacking and adding the generator test files to exported classpath, or rewriting the generator
2018:06:25 19:03:33 noisesmith of those options, a generator in the source file is the least hackish
2018:06:25 19:04:29 blance thanks! that makes me feel better writing generator in along side with the spec itself
2018:06:26 02:16:43 athos @blance I was working on a library for managing generators separately from spec definitions a couple of months ago. If you’re interested, take a look at it! https://github.com/athos/genman
2018:06:26 02:23:47 blance this looks promising! i'll definitely give it a try next time I write spec
2018:06:26 21:03:18 justinlee question about spec-tools.data-spec. it seems like with data-spec there are now two namespaces: the one in the spec registry and the symbol i define in my own code. this makes sense with the one example in the readme because it is a map. but what about for code like
(def hit-box-spec
(ds/spec ::hit-box [number?]))
It seems like both ::hit-box and hit-box-spec are now symbols that refer to a spec. I’m just getting a bit confused about how to use this library with non-map specs.
2018:06:26 21:09:56 noisesmith ::hit-box is a keyword, specs resolve via a central registry that uses keywords. hit-box-spec is a var, containing whatever ds/spec returns (not necessarily the same value stored under that key in the spec registry, but maybe?)
2018:06:26 21:31:41 justinlee yea. that’s how i understand it. i think i’m just confused why ds/spec doesn’t rely on the fact that it creates a side effect in the spec registry. i’m not sure why i’d want to also have a var in my namespace that points to a value that is apparently interchangeable (at least it is when using s/valid?
2018:06:26 21:44:27 noisesmith but it also points to the literal data,that ds/ functions can use to create other specs, right?
2018:06:26 21:44:32 noisesmith I'd assume that's the reason
2018:06:26 21:45:57 justinlee I don’t think so. In the readme, they use a different var to point to the data used to create the spec. So they might write the above as (def hit-box-spec (ds/spec ::hit-box hit-box))
2018:06:26 21:46:24 justinlee but it might have to do with these transformations, which I’m not using and don’t understand. that would make sense
2018:06:26 22:05:18 justinlee hm what it is doing is more complex than this. my statement above that ::hit-box and hit-box-spec are interchangeable for valid? is wrong
2018:06:26 22:57:11 ikitommi @lee.justin.m side-effects are bad, I think the data-specs could benefit from a local-keys variant that doesn’t need to register the keys. data-specs are anyway not in align to the Spec filosophy of reusable keys, so why not go further down the road…
2018:06:26 22:58:26 ikitommi but for the original:
(ds/spec ::hit-box [number?])
can be written:
(ds/spec
{:name ::hit-box
:spec [number?]})
… and as there are no keys, the :name can be omitted:
(ds/spec
{:spec [number?]})
2018:06:26 22:59:02 ikitommi to register that, you can say:
(s/def ::hit-box
(ds/spec
{:spec [number?]}))
2018:06:26 22:59:07 justinlee @U055NJ5CC ah i see. i should be using the map syntax.
2018:06:26 22:59:20 justinlee as for whether side effects are bad, that’s just the way spec works, right?
2018:06:26 22:59:31 justinlee you have to register your spec with the registry
2018:06:26 22:59:34 ikitommi right.
2018:06:26 23:00:09 justinlee the thing i’m not getting right now is this:
2018:06:26 23:00:40 justinlee i see that the two subspec are registered, but why not the main banana spec?
2018:06:26 23:00:56 ikitommi if you try to put a map somewhere in the data-spec and don’t provide a :name, it will fail-fast:
(ds/spec
{:spec [[[[[[[{:a int?}]]]]]]]})
; CompilerException java.lang.AssertionError: Assert failed: spec must have a qualified name
; (qualified-keyword? n), compiling:(data_spec_test.cljc:380:3)
2018:06:26 23:01:21 ikitommi good question, not sure, maybe it should?
2018:06:26 23:01:34 justinlee i just don’t even know how it works 🙂
2018:06:26 23:04:11 justinlee basically i was expecting an unmunged :seekeasy.app-state/banana given that i provided it that as the name of the spec. although i guess that’s consistent with the readme, now that i think about it. the only thing that got registered were the subspecs.
2018:06:26 23:06:43 ikitommi if there would be the local-keys, there would be no registering of any specs. that in mind, having a function called spec doing registration of the top-level would be bit odd.
2018:06:26 23:07:17 ikitommi there could be ds/def for that…
2018:06:26 23:07:30 justinlee oh i see what you mean
2018:06:26 23:07:41 ikitommi … but (s/def ::name (ds/spec ..)) is the way to do it now.
2018:06:26 23:07:59 justinlee because spec does (s/def ...) so we would expect it to cause a side effect
2018:06:26 23:08:20 justinlee but here it’s just a function so we need to def it ourselves. okay. this is coming together for me.
2018:06:26 23:08:47 justinlee I guess I thought there was a separate data structure for the specs. But they are just stored on normal vars?
2018:06:26 23:09:11 justinlee the existence of the st/registry made me think this
2018:06:26 23:09:41 ikitommi all Specs are just values, you can store them in a Var.
2018:06:26 23:11:23 ikitommi .. unless you register them and get a name than can be used in s/keys. I think it’s the only one that requires a spec to be registered?
2018:06:26 23:11:46 justinlee ahhhhh okay
2018:06:26 23:11:56 justinlee damn that is confusing 🙂
2018:06:26 23:13:32 justinlee basically, the thing that confused me is that if you do (s/def ::something ...) that will show up in the registry even it if isn’t meant to be used as a key but if you do (def something (ds/spec ...)) it doesn’t show up. but both work. i hadn’t appreciated the fact that nothing cares about the registry except for s/keys
2018:06:26 23:16:38 justinlee btw, thanks for spec tools. i really really really prefer the self-documenting format of schema, and now i have my cake and eat it too
2018:06:26 23:21:52 ikitommi thanks! it’s kind of a roque lib, I hope the final version of spec will make much of it redundant.
2018:06:26 23:13:15 ackerleytng what types of functions do you guys normally spec? do you spec utility functions?
2018:06:26 23:19:14 justinlee the answer you’ll hear most commonly is “at data boundaries” or something like that. e.g. reading data from a file or network or moving from one chunk of code to another, rather than doing it wholesale on every internal function
2018:06:26 23:35:22 noisesmith I'd put it as "system boundaries" rather than "data boundaries" but yes, exactly that
2018:06:26 23:35:49 noisesmith where system boundaries are has a lot to do with your design, but if your system is designed it should have some :D
2018:06:26 23:36:13 ackerleytng thanks 🙂
2018:06:26 23:36:23 ackerleytng won't your system boundaries keep changing?
2018:06:26 23:36:38 ackerleytng as you compose functions with functions
2018:06:26 23:37:06 noisesmith if your system boundaries are changing those weren't your system boundaries
2018:06:26 23:37:13 ackerleytng another question! how do you define an fspec where you don't care about the name of the argument, but you care about the type?
2018:06:26 23:37:41 ackerleytng ah thanks that sounds right haha
2018:06:26 23:38:42 noisesmith the idea is that it isn't a system if you don't define some limit or border, that's really the first step. What the boundaries are, and which things cross them, should be one of the first things defined, and often you'll want to define things so it changes relatively rarely
2018:06:26 23:39:27 noisesmith @ackerleytng for the names of things in specs, the reason to have the names is for the error messages you get without a match. Otherwise the output of a failure turns into a soup of data types that isn't very helpful.
2018:06:26 23:39:52 ackerleytng oh! so it doesn't actually match against the name of the actual parameter?
2018:06:26 23:41:02 noisesmith if it is the thing I'm thinking of it's a series of name / type for each arg right?
2018:06:26 23:41:23 noisesmith the name is used in generating the message, it doesn't have to match the arglist or anything
2018:06:28 04:44:30 ackerleytng thanks!
2018:06:28 11:02:33 gnl Introducing Ghostwheel – it makes the writing, reading, refactoring, testing, instrumentation and stubbing of function specs easy; introduces a concise syntax for inline fspec definitions; helps keep track of side effects; and makes general debugging almost fun, with full evaluation tracing of function I/O, local bindings and all threading macros.
https://github.com/gnl/ghostwheel
2018:06:28 13:40:06 andre.stylianos This looks really, really nice! I will for sure give it a try soon
2018:06:28 13:43:07 gnl Great, report back when you do!
2018:06:28 12:24:21 ackerleytng how do i spec a function that has a variable number of arguments?
2018:06:28 12:46:21 ackerleytng but only (s/conform ::helpers/test zzz-var-args) conforms
2018:06:28 14:33:18 favila They can't all conform: these are incompatible function signatures
2018:06:28 14:33:59 favila you can write a spec such that the args for any of those signatures passes, but you can't disambiguate their meaning
2018:06:28 14:35:44 favila nor can you work backwards: generated sample args from this spec won't work for all functions. zzz only works with :c and some :a for example
2018:06:28 14:35:56 favila if you give it something matching :b it will fail
2018:06:28 15:54:25 justinlee @clojurians.net whew! you’ve been busy! i’ve just switched over to using spec-tools for its data-spec features that allow me to specify nested data types using data. I was going to try to get orchestra installed too so I can use their defn-spec, but your library looks like it provides that functionality and more. I think can can mix and match spec-tools with either of these. any pitfalls you can think of?
2018:06:28 16:20:32 gnl @lee.justin.m I honestly don't know.
I have no experience with spec-tools and it is my (very superficial) understanding that they rely on some internal spec workings/APIs which are likely to change and break as spec matures and moves out of alpha, I remember reading a comment from Alex Miller to that effect, if I'm not mistaken. That's one of the reasons it hasn't been near the top of my list of new stuff to check out. Feel free to correct me if I'm mistaken and spreading FUD here, anyone.
FWIW I can say that Ghostwheel is only using public APIs and not doing anything hacky at all. And if you do end up using it - feedback would be much appreciated! I've done a moderate amount of dogfooding, but there's always all kinds of things that tend to pop up in the wild.
2018:06:28 16:21:42 gnl Oh and btw, Ghostwheel also supports orchestra for instrumentation.
2018:06:28 16:22:37 justinlee @clojurians.net that’s true about spec-tools. it says so right in the readme. (doesn’t bother me though. code can always be changed 🙂 )
2018:06:28 16:23:47 noisesmith > code can always be changed
tell that to python3
2018:06:28 16:24:18 justinlee i’ve never kept up with python
2018:06:28 16:25:39 justinlee and at any rate, spec tools is a small library. if they break it, i’ll just not upgrade or i’ll rewrite my types. it’s not mission critical
2018:06:28 16:25:40 noisesmith python3 was introduced in 2008. most of the community still uses python2 because they use libraries that are incompatible with version 3
2018:06:28 16:26:03 justinlee oh. well that’s not exactly comparable. that’s changing out the whole language 🙂
2018:06:28 16:27:43 noisesmith if you are relying on the internals of spec, you need to change your code with updates to spec, and if you aren't careful your users need to change code too - the point here is the way that compatibility issues can fragment a community and get things stuck, and spec marks things as internal to avoid this kind of scenario
2018:06:28 16:31:38 justinlee well spec should have given me a usable tool instead of cramming its way of doing things down my throat. by releasing this thing as an official library schema lost its mindshare so i’m kind of forced to use spec and i’m doing my best. i’m working on a product so there are no consumers of my code.
2018:06:28 16:23:16 justinlee I basically just want to spec out the args inline using a macro. I think I’ll try yours out because maybe I can get tracing too. I never could get debux to work with shadow
2018:06:28 16:24:19 gnl Another thing that comes to mind regarding orchestra's defn-spec is that if I'm not mistaken it's not quite straightforward to do multi-arity functions with varying return specs between the different arities. With Ghostwheel it's just like writing single-arity functions and it automatically does the right thing.
2018:06:28 16:25:04 gnl And last but not least - migration to and from ghostwheel is easy, because you are not interspersing specs between the args, etc, you are basically just adding a single vector to the standard defn syntax
2018:06:28 16:26:48 gnl Regarding debux - I got it to work, sorta, and was considering using that, but I much prefer the clairvoyant+re-frame-tracer output, so I took those and pimped them a bit instead.
2018:06:28 16:29:27 justinlee @clojurians.net does all of this stuff print back to the repl? it seems like shadow-cljs defaults to the browser console. i’m just trying to figure out how it will work with all of this color output
2018:06:28 16:29:57 gnl It prints to the browser console, js/console.log basically.
2018:06:28 17:00:11 metametadata There's also this nice little alternative to orchestra - https://github.com/Provisdom/defn-spec. I like that it's friendly to Cursive IDE syntax highlighting.
2018:06:28 17:11:54 gnl @metametadata So is Ghostwheel! Cursive-friendly that is.
2018:06:30 16:02:02 kurt-o-sys right, thx... will try.
2018:06:30 16:35:43 kurt-o-sys oh right, but will see if I can make that more dynamic in some way... I'm fetching the map from outside. well... will see 😛
2018:06:30 16:40:57 kurt-o-sys @clojurians.net I like ghostweel - just trying it now - but it doesn't validate the return value of >defn. I seem to remember clojure.spec itself doesn't either, but orchestra does so... Any plans to add this to ghostweel as well (validation of return values of functions)?
2018:06:30 16:43:32 justinlee @kurt-o-sys have you configured to do so? there’s a separate config option to turn that on
2018:06:30 17:25:18 andre.stylianos
;; Spec-instrument functions on namespace reload.
::instrument false
;; Spec-instrument functions on namespace reload using orchestra,
;; which spec-checks the output in addition to the input.
::outstrument false
2018:07:01 18:20:38 kurt-o-sys I didn't... - I didn't know it was an option, but I do now 🙂.
2018:07:01 19:43:50 andre.stylianos 😜
Glad to help!
2018:06:30 17:25:42 andre.stylianos from the README
2018:06:30 18:25:08 gnl @kurt-o-sys It's already in there! The option is ::g/outstrument.
2018:06:30 18:28:03 gnl Didn't read the other replies before I answered. By the way there's a bug in the current stable version, where ::g/instrument and ::g/outstrument require ::g/check to be enabled to have any effect. This doesn't really make any sense and is fixed in 0.2.2-SNAPSHOT, soon to be released as a stable 0.2.2 once some Figwheel issues have been cleared up.
2018:06:30 18:28:03 gnl Didn't read the other replies before I answered. By the way there's a bug in the current stable version, where ::g/instrument and ::g/outstrument require ::g/check to be enabled to have any effect. This doesn't really make any sense and is fixed in 0.2.2-SNAPSHOT, soon to be released as a stable 0.2.2 once some Figwheel issues have been cleared up.
2018:07:01 18:22:15 kurt-o-sys nice, thx! I did figure it out concerning that ::g/check issue, not about the ::g/oustrument😛. Thanks a lot. Just one more question: how would you recommend instrumenting only during dev, but not for a prod build? (Just wondering what you consider as best practice)
2018:07:01 18:30:36 kurt-o-sys also, I must be doing something wrong:
(>defn target
"target function to be minified"
{::g/instrument true
::g/outstrument true}
[in]
[any? => double?]
"wrong")
This should fail, right? (return value should be of type double, but it's a string). However, it doesn't fail at all.
Adding the config to the ns doesn't help either:
2018:07:01 18:33:44 gnl Don't use ::g/instrument and ::g/outstrument together, only one will be used, in this case ::g/instrument. What you want here is only ::g/outstrument.
2018:07:01 18:34:47 gnl And regarding not instrumenting in production – just make sure that ghostwheel isn't enabled in your prod build config
2018:07:01 18:35:51 gnl No way to do that in Clojure at the moment, so just don't do instrumentation there in production. 🙂
2018:07:01 01:22:43 codonnell Does anyone know of an efficient way to generate a multi-spec value with a particular tag? (ie. without doing a such-that and hoping to get lucky)
2018:07:01 06:09:44 ackerleytng @codonnell what do you mean by a multi-spec value? have an example?
2018:07:01 06:59:08 ackerleytng If i run a stest/check and hit an error, how do i inspect the value that was generated? my return value is a function. I fspecced it, and one of the inputs to that function, generated during stest/check is causing the spec to fail
2018:07:01 12:39:30 codonnell @ackerleytng
(s/def ::type #{::a ::b})
(s/def ::a-value pos-int?)
(s/def ::b-value keyword?)
(defmulti mymap-type ::type)
(defmethod mymap-type ::a [_]
(s/keys :req [::type ::a-value]))
(defmethod mymap-type ::b [_]
(s/keys :req [::type ::b-value]))
(s/def ::mymap (s/multi-spec mymap-type ::type))
;; How to generate a mymap value with tag ::a here without using gen/such-that
2018:07:01 13:18:38 ackerleytng something like this?
2018:07:01 13:25:12 codonnell I'd prefer not to build up the value manually like that, since the actual spec I'm working with is much more complex than the toy example I posted above.
2018:07:01 13:26:58 codonnell I'll definitely check out that talk, thanks.
2018:07:01 13:44:56 ackerleytng @gfredericks thanks for giving that talk!! I loved the pictures too
2018:07:01 22:20:14 Andreas Liljeqvist It would have been, but there is a bug with providing a generator for the dispatch key - See my bug report and patch https://dev.clojure.org/jira/browse/CLJ-2311
2018:07:01 22:31:25 codonnell thanks @andreas862, that is exactly what I was looking for. Hopefully your patch is accepted!
2018:07:02 00:13:05 ackerleytng s/gen allows you to override a specific generator?
2018:07:02 00:13:58 ackerleytng I see, thanks!
2018:07:02 09:24:20 otfrom morning
2018:07:02 09:26:23 otfrom I have a map where one of the namespaced keys ::foo for example, in the whole program can be one of 3 values #{"foo" "bar" "baz"}, but in one part of the program can only be #{"foo" "bar"} because I've filtered out "baz". Is there a way of expressing this in spec? I've been looking around and I haven't found anything (might be morning brane and solved by more coffee, but I'm not sure)
2018:07:07 11:53:57 rickmoynihan Interesting… I think this is awkward because it’s expressing something that’s brittle and contrary to the growth ideals of spec?
i.e. saying “this set must have only these keys” is analogous to saying “this map must have these keys and no others”.
The solution with maps in this situation is often to use select-keys in your function, so the function is robust to inputs with arbitrary other keys as it will ignore them.
Could the solution in your situation be to intersect the set-argument with #{"foo" "bar"} so anything else is ignored?
2018:07:08 15:59:08 otfrom hmm.. I hadn't thought about the growth ideals properly when thinking about this. It makes me think that the envelope should be very permissive and that I should create new maps for the payload with new specs going through the rest of the system. I think that is why I keep asking these questions as spec affords some forms and punishes others (for good reasons such as growth), but it can be difficult to see it all the time and design accordingly.
2018:07:09 07:58:06 rickmoynihan Yes, if you build specs where you find yourself trying to spec “and nothing else” you’ll struggle.
Obviously I don’t really know what you’re doing, but your new ideas seem to make perfect sense from a general architectural perspective i.e. the transport layer of an architecture typically shouldn’t know about specifics of the application/content layer.
2018:07:09 08:58:12 otfrom I often find if I'm struggling against one of the affordances of something in clojure it is because I'm not thinking about it very well. I think this might be one of those cases. Thx!
2018:07:09 14:17:43 carocad @U0525KG62 I have had that case as well though not for part of the program but rather for part of an api response. My approach was to split the set into the two possible solutions and then use s/or and s/merge to express the parts that are common and differentiate the other ones
2018:07:02 10:07:20 andre.stylianos Depends on what you want out of it I think, and what those things mean in your code.
You can make ::filtered-foo which is #{"foo" "bar"} and ::foo is now s/or of ::filtered-foo or #{"baz"}.
2018:07:02 10:09:21 andre.stylianos Just avoid having ::foo itself mean two different things depending on context, as that breaks specs being globally consistent
2018:07:02 13:03:50 otfrom hmm... makes me want to use unqualified keys for things like that then
2018:07:02 13:16:13 otfrom I basically have a lot of "type fields" and often filter down to particular types in certain parts of the system and would like to constrain things in those sections (esp things like generative testing)
2018:07:02 13:16:53 otfrom and different affordances drive different design styles in maps
2018:07:02 13:38:12 andre.stylianos You could also leave ::foo as being #{"foo" "bar" "baz"} and in specific places just say that ::foo.subset is (s/and #(not= % "baz") ::foo)
2018:07:02 14:21:27 otfrom yeah, it just means that the key has to change going through so any code that does work on the supertype wouldn't be pointing at the right key.
2018:07:02 14:22:32 otfrom it just makes an envelope and payload style tricky to do (as payload might have loads of types)
2018:07:02 14:37:31 otfrom so this is tricky to model in spec then
{::id <some uuid>
::timestamp <some timestamp>
::payload <a number of different things that are maps that vary depending on domain>}
2018:07:07 11:55:21 rickmoynihan isn’t this the use case for a multi-spec on ::payload?
2018:07:02 14:39:05 otfrom if you want to be able to constrain ::payload later as you are only dealing with one of the domains.
2018:07:02 14:40:07 otfrom I suppose you could leave ::payload as just a map
2018:07:02 14:40:20 otfrom and deal with the payload maps separately
2018:07:02 14:45:54 andre.stylianos Well, you can either have ::domain-1/payload, ::domain-2/payload and so on, or you could spec ::payload with s/or, conform it and check if the conformed value is the branch you expected
2018:07:02 14:47:22 andre.stylianos do keep in mind that I'm no clojure.spec expert, just brainstorming a bit :man-shrugging:
2018:07:02 14:58:11 otfrom or have a non-namespaced :payload and define it as :req-un as needed
2018:07:02 14:59:47 seancorfield Well, you can use namespaced variants of :xxx/payload and still define it as :req-un -- that way you can see which version you're working with -- and still just use :payload in the map.
2018:07:02 15:03:44 otfrom I'm not sure I understand you @seancorfield. Does that give me the ability to say "this bit of code here only works with this domain of what might go in payload rather than all the domains"
2018:07:02 15:14:18 seancorfield You can have multiple specs with the same set of (unqualified) keys -- but each spec can use different definitions for those keys by using different qualified versions of a key.
2018:07:02 15:15:03 seancorfield (s/keys :req-un [:foo/bar]) and (s/keys :req-un [:quux/bar]) -- the actual map has :bar in both cases but the specs are distinct.
2018:07:02 15:25:31 otfrom cool. That was what I thought. That you could basically redefine what the *un*qualified keys meant where you wanted, but that you couldn't do that with qualified keys in a map. I didn't know about the :req-un sugar you had their tho (or had forgotten)
2018:07:02 15:32:47 guy I mean you still have to s/def the keys right
2018:07:02 15:33:13 guy (s/def :quux/bar string?) and (s/def :foo/bar pos-int?) for example
2018:07:02 15:33:26 guy u can’t just use :req-un to redefine them
2018:07:02 15:33:33 guy i’m pretty sure
2018:07:02 15:33:47 otfrom yeah, but I can do that per domain :domain1/bar:domain2/bar
2018:07:02 15:33:49 otfrom etc
2018:07:02 15:33:52 guy ye 👍
2018:07:02 15:34:02 guy >That you could basically redefine what the *un*qualified keys meant where you wanted,
Just wasn’t sure what u meant here
2018:07:02 15:34:08 otfrom which is exactly what I want to do here
2018:07:02 15:34:12 guy cool cool
2018:07:02 15:34:13 guy 👍
2018:07:02 15:34:26 otfrom feels like a bit of a cheat, but it does work with what spec affords
2018:07:02 15:34:34 otfrom and I'm all about the affordances
2018:07:02 15:34:37 guy it’s in the spec guide if it makes you feel better 😄
Much existing Clojure code does not use maps with namespaced keys and so keys can also specify :req-un and :opt-un for required and optional unqualified keys. These variants specify namespaced keys used to find their specification, but the map only checks for the unqualified version of the keys.
2018:07:02 15:39:45 guy :+1:
2018:07:02 15:41:07 otfrom thx
2018:07:03 18:28:40 fominok Hi there! I suppose this is a right place for questions and I have one: is it possible to have references with spec generators? Like this: I have an author spec and a book spec, book has an author_id, can I generate one author and n books with proper id?
2018:07:03 18:31:12 noisesmith I think gen/fmap could do that
2018:07:03 20:00:17 taylor @fominok you could also use test.check's let macro like this:
which is generating a map of author + books from their individual specs, and just associng the author's ID over each book
2018:07:03 20:10:27 fominok Thank you, @taylor
2018:07:04 15:13:41 ackerleytng generative tests with specs take really long to run. what are some tips to try and reduce the testing time?
2018:07:04 15:37:45 taylor This can depend on your specs. Recursive specs can be especially costly. Can you give an example that’s taking a while to run?
2018:07:04 23:22:56 ackerleytng it's not a recursive spec though
2018:07:04 17:02:27 seancorfield @ackerleytng I don't feel generative tests should be run automatically as part of your (fast) unit test suite. They can/should be run separately or as part of CI for example. Because generative testing can definitely take a while.
2018:07:04 17:24:11 dominicm Rich has mentioned a number of times that generative tests should only run when the relevant code changes. I'm sure he'd be very happy if someone built that for him.
2018:07:04 18:13:03 alexmiller I have built the hard parts of it, just needs a lot of productization
2018:07:04 21:11:24 dominicm Do you mean to suggest that it's going to be a product that's sold or branded? Or just that it's rough?
If it's rough, is it in such a state that the community might learn much from it by seeing it and seeing where they can take it?
2018:07:04 21:30:00 alexmiller It will be a contrib library. It’s not at a state where I’m ready to publish it yet, both for design and impl
2018:07:04 21:32:11 alexmiller We may end up sharing some code with codeq 2 also, which is similarly unfinished. Just a lot of stuff to shake out and none of us have time to work on either atm
2018:07:04 20:29:05 gnl @dominicm Shameless plug – https://github.com/gnl/ghostwheel runs gen-tests per namespace on hot-reload which is effectively just that (or close to it), with the option to make re-rendering dependent on successful test completion.
2018:07:04 21:11:43 dominicm I hadn't appreciated that's why it did those things. Very impressive!
2018:07:04 21:28:02 alexmiller That’s great but what I’ve been working on is code analysis at the function dependency level
2018:07:04 21:44:09 gnl Function-level granularity would be even better for this of course – looking forward to it. Is this going to be a core thing or a third-party library?
2018:07:04 21:45:36 gnl @dominicm It's really just a fancy (run-tests) but it gets the job done. 🙂
2018:07:04 21:47:42 gnl The important thing is that if you're working on layout/view stuff where quick hot-reloading is most valuable, it doesn't run any expensive event handler gen-tests, etc.
2018:07:04 21:49:19 gnl @U064X3EF3 Oh, I just saw the other thread where you answered that, nevermind.
2018:07:04 23:23:25 ackerleytng @seancorfield Thanks! I'm writing my own custom generators, could that contribute to slower testing?
2018:07:04 23:24:26 ackerleytng even with overrides like {::foo (fn [] (gen/return <something pre-generated>} does not improve testing speeds much. why would this be?
2018:07:04 23:24:42 ackerleytng gen/return probably doesn't run the generator at all, right?
2018:07:04 23:43:43 gfredericks well it is a generator, but everything it does is trivial, so it by itself should never make anything slow
2018:07:06 16:50:37 tws I’d like to try and do this without custom generators… any ideas? I want to spec out a string of digits, with each having certain restrictions. e.g. each digit is (s/int-in 2 10) but I’d like 3 in a row catted into a string. using regexes like #(re-matches #"[2-9]{3}" %) requires me to make a custom generator I believe. Struggling with the syntax.
2018:07:06 16:51:24 tws I can get an s/tuple easy enough, but not sure how to jam that into a string inside the s/def
2018:07:06 17:21:21 guy (s/def ::digit my-pred-fn my-gen)
2018:07:06 17:21:54 guy (defn my-pred-fn [x] your logic)
2018:07:06 17:22:00 guy if that makes sense?
2018:07:06 17:22:18 guy but you need a generator that would then fulfil that predicate
2018:07:06 17:22:34 tws Yes thanks. I’m familiar with custom generators. It just seems like I’m codifying the same rule in two places (pred and gen) which I would like to avoid if I could.
2018:07:06 17:23:02 tws But I’ll just move ahead with a custom gen.
2018:07:06 17:24:18 guy @tws oh sorry just read your first line facepalm
2018:07:06 17:24:24 guy >I’d like to try and do this without custom generators
2018:07:06 17:24:55 tws Communication is hard!
2018:07:06 17:25:09 guy haha nah i just need more coffee so i can read better 😞
2018:07:07 13:43:35 ackerleytng why does this
(ns foo.bar
(:require [clojure.spec.test.check]))
error out with
Could not locate clojure/spec/test/check__init.class or
clojure/spec/test/check.clj on classpath.
2018:07:07 13:45:05 gfredericks I don't think that's a real ns
2018:07:07 13:47:08 ackerleytng hmm there's no file associated with it
2018:07:14 01:32:18 ackerleytng I do this, so previously it was just something like expected: true and actual: false
2018:07:14 01:32:21 ackerleytng which is meaningless
2018:07:14 01:32:39 ackerleytng but now it shows #error and dumps the error from spec
2018:07:14 12:55:15 gfredericks hmm
2018:07:08 04:52:29 ackerleytng also, youre right, it should be :reporter-fn
2018:07:11 13:27:24 roklenarcic does calling instrument with no arguments instrument all symbols in current namespace or all instrumentable symbols in all the loaded namespaces?
2018:07:11 13:28:51 taylor all instrumentable vars in all loaded namespaces AFAIK
2018:07:11 13:31:06 alexmiller yes
2018:07:12 10:09:30 Matt Butler Is there any way to use predicates inside the literal comparator to achieve something like this in spec.
(s/valid? #{["foo" "bar" string?]} ["foo" "bar" "baz"])
2018:07:12 12:10:36 taylor you could spec that with s/cat
2018:07:12 12:44:41 alexmiller Or s/tuple
2018:07:12 14:10:05 Matt Butler so (s/tuple #{"foo"} #{"bar"} string?) ? but if I wanted to combine that with other fixed values where I know all 3 elements, I'd have to use an s/or like so?
(s/or :set #{["a" "b" "c"]} :tuple (s/tuple #{"foo"} #{"bar"} string?))
2018:07:12 14:10:20 Matt Butler There is no way to next dynamic values inside the #{}
2018:07:12 14:13:57 Matt Butler Thanks btw 🙂 jut wanted to check my understanding.
2018:07:12 14:17:35 taylor if I understand the use case, s/or seems reasonable to me
2018:07:12 14:19:54 Matt Butler Yeah I have a set of triple store values, most are known but for some subset the final key is freetext.
2018:07:12 15:58:49 akiel Is there a way to override key-specs in test like it is possible for fn-specs with instrument?
2018:07:12 16:00:51 ghadi see the second argument to instrument, specifically :overrides :stubs :gen
2018:07:12 16:03:42 akiel I don’t see :overrides in the doc string of instrument. I also don’t see a possibility to override key-specs.
2018:07:12 16:04:21 akiel With key-spec I mean (s/def ::a int?). I like to override ::a.
2018:07:12 16:04:29 ghadi sorry, I goofed, that's not an actual config key
2018:07:12 16:05:59 ghadi Overriding the key spec doesn't really make sense conceptually -- you'd be changing the contract but preserving the same global name
2018:07:12 16:10:14 akiel I have a function which gets a map of values and I don’t like to generate the real values in test. So I would like to override the specs for the keys in the map. But if I think more about it, why should a supply a map to the function at all? So I can change the fn-spec to take a custom test value instead of a map which custom test values in it. That should work.
Oh I also stub the function.
2018:07:13 14:48:20 roklenarcic How can one spec maps with string keys?
2018:07:13 14:51:27 mpenet (s/map-of string? ....) if you want to use s/keys you will need to convert them first
2018:07:14 21:26:46 roklenarcic Hm I thought this would work:
2018:07:16 09:38:07 bbrinck @U66G3SGP5 Let me know if you encounter any problems with the above code
2018:07:16 11:12:28 roklenarcic I tried set! before, but it's been my understanding that it only sets var value for current thread
2018:07:16 11:18:24 bbrinck Hm, I’m not sure what happens in threads started within the REPL - I can check later. You might have to call ‘set!’ in each thread. But in any case, alter-var-root! won’t work in the REPL context since the var is bound by Clojure before your code is run, so I think ‘set!’ is the only solution.
2018:07:17 06:59:47 athos A quick workaround for this is to try (alter-var-root #'s/*explain-out* (constantly exp/printer)) in your user.clj. Clojure runtime should automatically load user.clj (if it exists) before launching the REPL.
2018:07:17 11:54:20 mpenet maybe some external lib does, but not spec itself afaik
2018:07:17 12:14:26 ikitommi @markt you were using schema-tools.core/select-schema? By design, Spec doesn't support closed keys. 3rd party?, there is at least spec-tools.core/select-spec for this
2018:07:17 12:16:30 markt Ah thought that would be the case, it's a bit of a odd case we have.
Cool thanks for the heads up, i'll have a look at spec-tools 👍
2018:07:17 12:37:24 ikitommi I think it’s the odd case not to strip out extra data at the borders.
2018:07:17 12:50:37 markt Interesting, so you would prefer to have a function only return what the spec dictates?
2018:07:17 14:14:49 markt Just noticed your the author of the library!
Thanks for writing the library 😄
2018:07:17 15:26:37 ikitommi not at function level, but at system boundaries: Reading JSON sent by a (potentially rogue) third party, or when writing to a (document) database. Not removing extra keys is a security risk, stripping keys manually is both error prone & extra work.
2018:07:18 12:25:52 markt absolutely agree, and thanks for all the help
2018:07:18 18:31:15 gklijs It's not that hard using specs to implement losing the rest. I rather have a bit of code than a dependency. That's the road I took, also added the option to have default values for keys, to keep specs backwards compatible.
2018:07:19 12:35:56 ackerleytng May I trouble someone to explain conformer to me? Don't really understand the docs
2018:07:21 01:24:29 jkrasnay Hi folks, I have a Re-frame app that is nicely checking my function invocations with spec but it’s not showing file names / line numbers.
2018:07:21 01:25:43 jkrasnay
Call to #' did not conform to spec:↵
<filename missing>:<line number missing>↵
↵
-- Spec failed --------------------↵
↵
Function arguments↵
↵
(... :foo ... ...)↵
^^^^↵
↵
should satisfy↵
↵
string?↵
↵
2018:07:21 01:26:05 jkrasnay Is this normal, or should I be able to get a file name and line number here?
2018:07:21 02:31:20 seancorfield @jkrasnay Are you using Expound there? That doesn't look like a raw clojure.spec error... Just wondering if it's a bug in the reporter itself...?
2018:07:21 03:32:18 jkrasnay Looks like Expound has that code there waiting for the day when CLJS will send caller info. For now it’s just teasing me.
2018:07:21 04:18:15 seancorfield @jkrasnay Good to know. I don't use cljs myself but I know there are all sorts of weird edge cases/differences from clj...
2018:07:22 04:32:10 alex-dixon Is it possible to define a spec with a custom generator but have it separate from the spec itself? Something like
2018:07:23 11:58:56 roklenarcic So when callbacks have a fspec defined, they get called with random data if they are instrumented?
2018:07:23 12:03:04 gfredericks yep
2018:07:23 12:03:18 gfredericks that's how the function is validated
2018:07:23 12:03:48 gfredericks it is a common source of surprise
2018:07:23 12:29:47 roklenarcic I guess i need to instrument just a specific ns
2018:07:23 12:34:37 mpenet there are options also on instrument you can use to bypass this
2018:07:23 12:34:45 mpenet well, to provide stubs at least
2018:07:23 16:30:35 fedreg Is there a way (or another function) to have spec/explain return just the spec errors and not the original data as well?
2018:07:23 16:37:45 samueldev @fedreg why not just pull the errors out of the result of spec/explain? :shrug:
2018:07:23 16:37:57 samueldev or create your own fn explain-errors that does that and use it instead :shrug:
2018:07:23 16:39:09 fedreg @samueldev I was about to do just that but then decided to check if I had missed something that already existed. Guess that’s a no… thx!
2018:07:23 17:02:17 seancorfield @roklenarcic Yeah, I may revisit some of those fspecs in clojure.java.jdbc.spec as I get more feedback from people. As @gfredericks says, the behavior you're seeing is what clojure.spec instrumentation is intended to do, but it can be surprising at first. And of course your specific :result-set-fn may well not accept any old collection -- but given that :row-fn can transform the result set data into, essentially, an arbitrary collection of any type of element, the :result-set-fn could potentially be called on any? types... We don't have parameterized types so we can't constrain the row function to map? -> T and then the result set function to coll-of T -> U...
2018:07:23 19:38:29 manutter51 This surprised me
1. In a CLJS namespace, define a function with a name that matches a valid HTML tag (e.g. "table" or "br" or whatever)
2. Create an (s/fdef ...) spec for it.
Result: every function in that namespace now evaluates to nil.
2018:07:23 19:38:59 manutter51 Just a local fluke in my current project, or something that other people can reproduce?
2018:07:23 19:39:12 manutter51 If the latter, known issue, or new?
2018:07:24 07:49:00 claudiu There seem to be a lot of nice tools/libs like expound, ghostwheel, orchestra, phrase etc... Is there an official/unoficial list with most of these like tools.deps wiki 🙂 ?
2018:07:24 09:27:33 claudiu Cool, but still just partials in multiple sources 😞 Was banging my head with tools.deps stuff and noticed that there was a really nice page. seems to be pretty complete. Was hoping for something similar for spec. https://github.com/clojure/tools.deps.alpha/wiki/Tools
2018:07:24 09:28:45 cvic Yeah. In an ideal world you would find all of these on https://clojure.org
Work in progress (I hope).
2018:07:24 15:02:57 avi @U064X3EF3 I tried to add a Tools page to the wiki in this repo:
https://github.com/clojure/spec.alpha
but can’t navigate to the wiki… I just get redirected back to the home page of the repo. Never seen that before. Any idea whether it’s a repo setting problem or a bug in GitHub maybe?
2018:07:24 15:25:17 eggsyntax https://cljdoc.xyz/
is trying to become the new center for documentation of clojure/script libraries. But doesn't especially help with finding good libs.
I know https://www.clojure-toolbox.com/ was updated with at least one lib recently, so it's not abandoned.
2018:07:24 19:25:32 alexmiller It’s a repo setting. Happy to change it, but Stu and I are both on vacation atm. Ping me again next week or send me an email or I’ll forget.
2018:07:31 20:43:07 avi @U064X3EF3 ping! ☝️
2018:08:02 17:56:43 alexmiller hey, fyi email is always better than dms as it can stick around in my inbox rather than disappear
2018:08:02 17:57:03 alexmiller I don’t have admin access to that repo
2018:08:02 17:57:17 alexmiller Stu does, but he is still out
2018:08:02 17:57:27 avi ah ok cool
2018:08:02 17:57:31 avi will do — thanks!
2018:07:24 16:42:10 martinklepsch Is there a reason spec doesn’t have it’s own project in JIRA?
2018:07:24 16:42:22 martinklepsch I assume it’s mostly historical?
2018:07:24 19:24:09 alexmiller Because spec and the core specs are tied into Clojure, we manage them in the context of Clojure even though it’s a different repo. That was an intentional choice. It may not always remain so.
2018:07:24 19:34:45 claudiu In the next release will spec graduate from alpha ?
2018:07:24 19:36:10 alexmiller I hope so, no promises
2018:07:25 12:47:28 jumar Let's say I have a strange function that has two arities:
1-arity computes some results and returns them
2-arity uses 1-arity to compute the same results and save the output to file.
How can I write :ret spec for this?
I tried :fn but struggling with reusing an existing spec
Here, the ::result-spec isn't properly used - I need to apply it only to the :ret key value, not the whole map passed to :fn.
2018:07:25 12:55:18 gfredericks if it were me I'd have two different functions
2018:07:25 12:57:18 jumar That was my idea too, but let's say it's a "convention" to do it this way. Can I still somehow reuse the ::result-spec or is there a better way to write :ret spec for this function?
2018:07:27 08:12:24 jumar Nice, thanks! I guess the only problem would be if I had more than one such function in the same ns (conflicting ::ret specs) but that's not an issue in my case.
2018:07:26 18:40:27 Ethan Hello, I'm working on creating specs and I would like to use ::bindings from the core.specs.alpha in the alpha.clj file in my code. I was wondering how I would go about doing that without all copying the code to my file.
2018:07:26 18:54:25 alexmiller Just refer to the full spec keyword
2018:07:27 13:44:04 guy Is resolve a spec function?
2018:07:27 13:44:06 guy 🤔
2018:07:27 13:44:11 guy I’ve not seen it before
2018:07:27 13:47:55 burbma resolve isn’t a spec function it’s in core. Suppose I started with "string?", how can I get to the clojure.core$string_QMARK_...? (resolve (symbol "string?)) won’t do it as we’ve seen.
2018:07:27 13:48:49 guy really? ill check it out thanks!
2018:07:27 13:49:11 guy ohh right got ya
2018:07:27 13:49:12 guy neat
2018:07:27 13:49:17 guy TIL;!
2018:07:27 13:50:01 guy if u want to see what it does, you can generally just try s/exercise and see what it produces
2018:07:27 13:51:42 burbma s/exercise gives the same error for the (resolve (symbol "string?")) version of the spec.
2018:07:27 13:56:24 guy Yeah i mean just for (s/exercise string?)
2018:07:27 13:56:26 guy sorry
2018:07:27 13:57:49 burbma Ah, no worries. Thanks for the pointer.
2018:07:27 14:01:43 burbma Using (var-get (resolve (symbol "string?"))) does the trick.
2018:07:27 16:39:09 noisesmith or deref, or the shorthand @
2018:07:27 15:57:39 alexmiller In queue to get screened
2018:07:27 16:46:50 dspiteself Thanks
2018:07:27 18:03:59 avi Last week at clojure/nyc I shared my experience, as a spec n00b, using spec to specify other people’s data structures … here’s the video if anyone’s interested: https://www.youtube.com/watch?v=eqfSifXaXnw
2018:07:27 18:14:20 fabrao well, it´s for dynamic size
2018:07:27 18:14:46 noisesmith right, but how can you use that directly as a spec predicate? where would the other arg come from?
2018:07:27 18:28:57 fabrao @noisesmith I´ll check many specs from vector, like fields in form -> [[:code "Error code" ::limite-tamanho 6] [:name "Error name" ::limite-tamanho 15]]
2018:07:27 18:32:15 noisesmith maybe I'm missing some spec feature that would make this work, but I still don't understand how you'd make spec supply two args to your spec
2018:07:27 18:32:49 noisesmith and as far as I know you can't use a neighbor or parent data in the check for a key
2018:07:27 18:33:30 fabrao Well, I´m thinking use like this so, [:codigo "Código do vendedor" #(<= (count %) 6)]
2018:07:27 18:34:15 noisesmith which is equivalent to what I proposed with partial above, right?
2018:07:27 18:34:33 fabrao yes, thanks a lot
2018:07:28 03:00:26 burbma How can I get the forms to evaluate before going into the macro?
2018:07:28 06:41:43 valerauko not sure if this is what you're looking for, but i have something like this
2018:07:30 17:33:03 djtango So I am pretty sure this is absolutely not how spec was intended to be used, but has anyone gotten mileage from leaving stest/instrument turned on in production to use spec for runtime contract assertion? Even more so with jeaye/orchestra
2018:07:30 17:33:59 djtango One reason I can think of why this may not be such a great idea is that stest/instrument validates fspec using generated inputs
2018:07:30 17:35:38 djtango are there any other reasons not to leave stest/instrument turned on?
2018:07:30 17:48:45 seancorfield @djtango The generative validation of fspec is one reason. Performance is another. You'll also get bare AssertionError exceptions instead of anything meaningful you might normally catch and handle (assuming you try to (catch Exception e ...))
2018:07:30 17:59:02 noisesmith yeah, my understanding is that AssertionError is for things you expect to throw during development - the vm even has a flag to ignore them
2018:07:30 18:02:47 djtango I suppose, to give more context on my usecase, it is useful (for me) to declare some invariants about a function (e.g. a relationship between it's args and return value) but in a sufficiently large project, it's possible for inputs in runtime to fail that invariant and it's nicer when that failure comes at the point of the invariant failing rather than some arbitrary of steps later like a null pointer error
2018:07:30 18:04:22 djtango spec happens to be a great way for declaring properties about lots of things, and while generative testing is meant to give better guarantees about over your testing space, the associated complexity with them is a harder sell for the team
2018:07:30 18:04:41 djtango If that makes sense?
2018:07:30 18:05:10 djtango Coming from Racket, I'm kind of used to contracts always being on
2018:07:30 18:24:58 noisesmith the pragmatic thing might just be overriding fspecs for the instrumenting
2018:07:30 21:57:19 noprompt it’d be great if there were a spec-impl that “merged” two s/ors or s/alts without having to reify the protocols (which is a dangerous game as i understand it).
2018:07:30 22:01:41 noprompt for example, if you want to express that clojure.core/unquote-splicing forms are not permitted at the top level you’d have a union for your legal top level forms and then another union for your subforms in, say, a list that includes unquote-splicing forms in addition to what is legal at the top level.
2018:07:30 22:02:35 noprompt to fully leverage all of spec, including explanations, it seems like i have to spell this out entirely in both cases if i want to avoid an additional layer of rettags.
2018:07:30 22:05:04 seancorfield @noprompt Or wrap s/or in s/nonconforming I guess...?
2018:07:30 22:07:20 noprompt @seancorfield that’s not quite the ticket. lemme demonstrate.
2018:07:30 22:09:08 noprompt what i do not want is the :a and :b tags. i still want to retain the :number and :unquote-splicing tags.
2018:07:30 22:09:22 noprompt by using s/nonconforming i lose the tags. 😕
2018:07:30 22:10:08 noprompt essentially, i want to extend the :clj.form/top-level union without adding an additional layer of rettags.
2018:07:30 22:10:59 noprompt of course, i can do this by using s/with-gen, s/conformer, etc. but i lose out on s/explain-data.
2018:07:30 22:12:56 noprompt if there was an s/explainer this’d be a done deal. 🙂
2018:07:30 22:13:48 noprompt in short, i think what i want is an or-spec-impl or an alt-spec-impl that does not require tagging the options.
2018:07:30 22:14:35 noprompt when it conforms returns whatever the conformers it’s compose of returns.
2018:07:30 22:15:27 noprompt i think there was previous discussion along same lines but for s/cat with an s/concat conformer..
2018:07:30 22:29:44 seancorfield Alex has talked several times about the possibility of adding a specific non-conforming or variant so you'd have that sort of level of control...
2018:07:30 22:32:42 arohner I have a big complicated s/keys, and I’m getting "Couldn't satisfy such-that predicate after 100 tries.". How do I discover which predicate is failing?
2018:07:30 22:58:25 gfredericks if you're using a new enough test.check, the ex-data should have the generator and predicate attached, at least
2018:07:31 18:15:43 lilactown how do I spec a function that takes only one argument?
2018:07:31 18:27:18 seancorfield (s/fdef my-func :args (s/cat :arg ::spec)) (to be more specific)
2018:07:31 18:29:01 seancorfield @lilactown Here's an example from our codebase at work
2018:07:31 20:48:54 lilactown how are people enabling instrumentation in development?
2018:07:31 20:49:43 lilactown I'm using CLJS. I'm thinking of doing something like:
(when js/goog.DEBUG
(stest/instrument))
but I'm worried that clojure.spec.test.alpha is going to end up in my release bundle
2018:07:31 21:04:00 justinlee let me know if you figure out the right solution 🙂
2018:07:31 21:55:44 seancorfield (we use clojure.spec in production code for validation and conformance so I'm puzzled as to why you'd want to avoid it? doesn't cljs use tree-shaking anyway to remove unused code?)
2018:07:31 21:57:04 lilactown validating and conforming can be done without including clojure.spec.test
2018:07:31 21:57:08 lilactown right?
2018:07:31 21:57:35 lilactown and I'm not sure if tree-shaking is smart enough to completely eliminate the lib if it's not used in a release build. will have to test
2018:07:31 21:57:52 lilactown I have a second stupid question: is there an easy way to alias a spec?
2018:07:31 21:58:17 lilactown e.g.
(s/def ::my-spec <point to ::other-spec>)
2018:07:31 21:59:02 hiredman spec validates functions by generating example arguements and calling the function on the examples, which is not something you'll want to do in production
2018:07:31 21:59:18 hiredman if you are strictly validating datastructures that should be fine
2018:07:31 22:00:13 lilactown right. I know I definitely don't want to instrument my fdefs in prod 🙂
2018:07:31 22:09:44 avi 🤔 I don’t think instrumentation involves generators … not saying instrument is appropriate for runtime, just might be helpful to keep things clear…
2018:07:31 22:17:40 noisesmith doesn't it, for the case where the arg is specced to be a function, it would be checked by exercising it
2018:07:31 22:18:02 noisesmith (let me know if I misunderstand something here...)
2018:08:01 13:51:45 avi :man-shrugging: I dunno, I think I spoke beyond my level of understanding — sorry
2018:08:01 13:52:57 avi your example is fascinating!
2018:08:01 13:53:51 avi thanks!
2018:07:31 22:31:09 noisesmith
user=> (require '[clojure.spec.alpha :as s])
nil
user=> (require '[clojure.spec.test.alpha :as stest])
nil
user=> (s/fdef bar :args (s/cat :f (s/fspec :args (s/cat :x int?) :ret int?)) :ret int?)
user/bar
user=> (defn bar [f] (f 0))
#'user/bar
user=> (stest/instrument `bar)
[user/bar]
user=> (bar (fn [x] (println "called with arg" x) x))
called with arg 0
called with arg 0
called with arg 0
called with arg 0
called with arg 1
called with arg 2
called with arg -1
called with arg 0
called with arg -2
called with arg -24
called with arg 3
called with arg -2
called with arg 9
called with arg -1
called with arg -757
called with arg -1
called with arg 3
called with arg 1963
called with arg 1165
called with arg -359
called with arg -3956
called with arg 0
0
2018:07:31 22:31:39 noisesmith example of the generation behavior with instrument
2018:08:01 04:48:37 tianshu Hi, how can I write a spec for arguments like [name & opts], opts is a hash-map here, I already have a ::opts spec for a hash-map, what a spec looks like for this function?
2018:08:01 04:56:41 tianshu It's likely I can use (s/cat :name ::name :opts (s/keys* :req-un [...]))
2018:08:01 04:58:07 tianshu but If I do (s/def ::opts (s/keys* ...)), I can't use ::opts here. It requires me to provide a vector
2018:08:01 05:00:42 tianshu oh, my mistake, if I use s/keys* it's ok, but can't use (s/and (s/keys* ...) (...)).
2018:08:01 17:08:09 arohner If I have a multi-spec, what’s the best way to get an instance of a single dispatch value? Occasionally (-> (s/spec ::foo) (s/gen) (gen/such-that [f] (= :bar (:type f)) will “couldn’t satisfy such-that”
2018:08:01 17:09:23 arohner but if I call the multimethod directly, it can return nonsense values, because (defmethod :foo [_] (s/keys :req-un [::type])), the :type doesn’t necessarily match :foo
2018:08:02 04:28:04 andy.fingerhut @noisesmith Do you know why instrument of function foo causes a function arg of foo to be called additional times with arguments that the intrumented call is not itself calling? I can understand doing that if you were doing generative testing on foo, but don't see the rationale for why one would want to do that for instrument.
2018:08:02 04:35:04 noisesmith My understanding is that's the only way spec has to validate a function argument
2018:08:02 09:38:21 mpenet It could validate it at invoke time, just once.
2018:08:02 09:39:02 mpenet At least in theory. A few people suggested that on jira among other places. I can imagine that being an option
2018:08:02 04:49:33 andy.fingerhut Makes sense. 20 times may be more than needed for that purpose, but probably tunable.
2018:08:03 16:34:42 kvlt So question:
I have the following...
2018:08:03 19:31:28 favila ExceptionInfo Couldn't satisfy such-that predicate after 100 tries. almost always means there's an s/and with values generated from earlier predicates could not satisfy later predicates
2018:08:03 19:31:57 favila very often you need a custom generator
2018:08:03 19:32:57 favila btw it would be really nice to somehow have a generator associated with a predicate
2018:08:03 19:33:47 favila is there some trick to this? I end up manually using s/with-gen everywhere I use a predicate (and having to remember to do that)
2018:08:04 05:03:21 kvlt Hey guys, sorry for the slow reply. I had ended up with this:
2018:08:04 06:45:18 misha what is the practical purpose of s/key’s :opt and :opt-un?
- documentation(?)
- include :opt keys during s/exercise
- anything else?
does s/conform conforms all known keys in a map, or just listed in :req/:opt/-un?
2018:08:04 07:23:40 valerauko in my understanding it's that the presence of the key is optional, but if it's present, its shape has to fulfill its spec
2018:08:04 07:48:54 guy ^
2018:08:04 07:49:07 guy Thats what ive used it for before
2018:08:04 08:20:34 misha @vale s/keys checks against all the known key-specs, you dont have to list them in the s/keys. empty s/keys will test all the registered keys
2018:08:04 08:21:18 valerauko are you sure about that?
2018:08:04 08:21:56 misha yes
2018:08:04 08:22:50 misha this is one of the goals or benefits of having global specs registry and namespaced keywords
2018:08:04 08:26:38 valerauko i see... well it's still meaningful for the un ones where you have to be explicit
2018:08:04 08:26:53 valerauko otherwise the opt is probably just for what you said
2018:08:04 08:27:27 misha true
2018:08:04 08:27:49 misha anything else?
2018:08:04 08:37:02 guy >you dont have to list them in the s/keys.
I thought part of the reason u list them in s/keys is to show easily what the shape of the data is?
2018:08:04 08:37:29 guy using (s/keys) like you did seems to be interesting, i’ve never seen that before
2018:08:04 08:39:01 misha @guy “show” is documentation + exercise I mentioned above
2018:08:04 08:39:27 guy oh right sorry
2018:08:04 08:39:29 guy 👍
2018:08:04 08:39:44 guy yeah i think uve covered it imo
2018:08:04 08:39:58 misha I’d not use empty s/keys, it is just an illustration
2018:08:04 13:23:17 ackerleytng does anyone have a good way of writing specs for ring middleware?
2018:08:04 13:24:20 ackerleytng for every middleware function along the chain, the inputs and outputs change
2018:08:04 13:24:35 ackerleytng seems tedious to spec all the middleware functions
2018:08:04 13:36:25 gfredericks you shouldn't have to fully describe the req/resp at each point though; you could describe just what's pertinent to that piece
2018:08:04 13:39:15 ackerleytng do you mean like... if this middleware augments the request with a new key value pair
2018:08:04 13:39:35 ackerleytng then perhaps the :args request should be just map?
2018:08:04 13:40:17 ackerleytng and the :ret is like a function with :args including just the new key-value pair?
2018:08:04 13:40:34 ackerleytng sort of like only writing spec for the change in this particular middleware?
2018:08:04 14:00:39 gfredericks yeah
2018:08:04 14:01:12 gfredericks you could also write a :fn saying everything else gets passed through
2018:08:04 23:01:26 ackerleytng thanks!
2018:08:05 00:23:59 ackerleytng What's an example of a well or appropriately specced codebase?
2018:08:05 00:58:33 seancorfield Note: that deliberately puts all the specs in a separate namespace so that they are optional and clojure.java.jdbc can still be used with earlier versions of Clojure.
2018:08:05 00:59:33 seancorfield Also, if you instrument that library, there's a noticeable overhead in performance (running the test suite takes much, much longer) and that's partly due to the complexity of some of the specs. I have a ticket open to look into optimizing them.
2018:08:05 01:03:33 seancorfield Another point to consider: those specs focus on functions. Spec is amazing with data structures -- and that's mostly how we use it at work: to specify data structures (including API parameters) and to validate values against those specs (with some coercions). We also use specs for tests, but that's less of our focus.
2018:08:05 11:53:39 hmaurer So you use specs to validate inputs from APIs?
2018:08:06 00:20:00 seancorfield Yes, we have specs for all our (public/external) APIs and then s/conform the data on entry to the API. If that is s/invalid? then we use s/explain-data to construct appropriate error codes and messages for the client/caller, otherwise we go forward with the conformed data.
2018:08:06 00:23:09 seancorfield Our specs include some coercion, since API input data can be either strings or "values" -- so we conform strings to values (and pass values through). So true is acceptable where we expect a Boolean, but so is "true" and that is conformed to true. Same with long, double, and date values. We have custom generators that produce strings (mostly they just use the "expected" value spec as a generator and then call fmap str on the result).
2018:08:06 10:48:12 hmaurer @U04V70XH6 thanks for the detailed reply! considering that specs are arbitrary predicates, do you find it easy to construct human-readable error messages?
2018:08:06 10:48:26 hmaurer (I am aware of expound)
2018:08:06 16:04:52 seancorfield It took a while to develop heuristics for walking the explain data but, given our domain and the specific set of specs and predicates in use, it isn't too bad.
2018:08:05 01:54:11 ackerleytng i think i've a vague idea of what you mean by focus on functions, could you elaborate on what you mean by focus on functions vs focus on data structures?
2018:08:05 01:54:46 ackerleytng also, whats the difference between s/nilable and s/??
2018:08:05 04:26:38 seancorfield @ackerleytng Sorry, I was off having dinner... s/nilable wraps a spec or predicate and produces a new spec that allows a value to be nil or conform to the wrapped spec.
2018:08:05 04:27:06 seancorfield s/? is a regex spec (in an s/cat sequence) for something that is optional.
2018:08:05 04:27:49 seancorfield So for a data structure (or argument list) that may have two or three elements, the third one is optional and would be specified with s/?.
2018:08:05 04:28:44 seancorfield for an element that could be nil or a string, you would use (s/nilable string?) -- it's a value either way, it's just allowed to be nil or else conform to string?.
2018:08:05 08:06:44 ackerleytng Thanks! So it's like... Elements can be nothing, nil or something. s/nilable allows the element to be either nil or something, but s/? allows the element to not be there at all.
2018:08:05 04:28:47 seancorfield Make sense?
2018:08:05 04:30:59 seancorfield As for focus on function vs data, it's about whether you're spec'ing function arguments (and return values and the relationship between them), vs whether you're using s/def to spec entities that are part of a data structure (and the data structures themselves). Not sure how to explain it beyond that. Does that help @ackerleytng?
2018:08:05 08:02:55 ackerleytng But without spec'ing functions, there's no way to trigger checks unless you manually call functions and use s/conformon the outputs right? So we're better off spec'ing the functions then running check in a deftest
2018:08:05 08:04:03 ackerleytng As in, I thought whether or not you spec functions, you would have to spec the data going in and out of functions
2018:08:05 08:04:52 ackerleytng Since you kind of want to exercise the specs in some way, it makes sense to just spec the functions to run tests, does that make sense?
2018:08:06 00:24:36 seancorfield We call s/conform explicitly because we're using spec in production code. And we conform just the data (arguments) we want to, whereas instrument checks all of them -- so you have to write specs for all the arguments.
2018:08:06 00:26:00 seancorfield Spec'ing functions and spec'ing data are very different approaches. Typically the former is for dev/test since the overhead of instrument can be high -- and it will use generative testing on fspec function arguments. The latter, however, is a great way to do data validation in production code.
2018:08:08 23:47:07 ackerleytng I see, thanks! So if i'm using spec in production, then s/conform is the way to go, and if i'm using spec for testing, then fspec is a good idea
2018:08:05 20:20:00 ikitommi could the Specs have a concise toString representation? example with expound:
2018:08:05 20:24:33 ikitommi … or print-method implemented.
2018:08:06 13:31:12 ben.mumford hi spec, i have two types i want to spec that have some fields in common. is it possible to derive specs from a common base spec (like a superclass in java)?
2018:08:06 13:31:52 ben.mumford in schema i could merge the base schema with new stuff.
2018:08:06 13:46:45 ben.mumford nvm sorted it using a combination of s/and s/keys
2018:08:06 14:32:08 favila s/merge is a thing also
2018:08:06 16:44:08 alexmiller s/merge is better for this
2018:08:06 20:08:33 hmaurer @U064X3EF3 @U09R86PA4 I was wondering a couple of months back if there would be a way to do this using derived keywords. i.e. with two specs :foo/a and :foo/b and the relation :foo/b derives :foo/a, enforce that :foo/b has to include :foo/a’s spec (or pass both, equivalently)
2018:08:06 20:18:43 favila if :foo/b and :foo/a are both map specs, s/merge is better because it can intelligently combine the key's generators
2018:08:06 20:19:19 favila s/and of two map specs likely cannot create a usable generator
2018:08:07 11:43:40 alexmiller There is no support for keyword hierarchies in spec, sorry
2018:08:07 12:14:44 hmaurer @U064X3EF3 out of curiosity, is that a design choice? or just a “not for now” kind of thing?
2018:08:07 12:34:49 alexmiller We’ve never talked about it
2018:08:07 12:35:08 alexmiller So, neither :)
2018:08:07 12:01:43 scaturr Is there a conventional way to describe a spec as an argument? Like if I have a function that takes a spec? I see s/spec? - but that doesn’t seem to work with something like (s/spec? ::my-spec)
2018:08:07 12:36:39 alexmiller There is no single predicate for it right now
2018:08:07 18:57:26 hmaurer o/. Is there a reason why s/def does not accept an optional docstring?
2018:08:07 19:00:26 favila Probably an accident of implementation; A spec is put in a global registry atom, which contains a hashmap from key/symbol to spec objects. There's no clear place to put metadata like there is with vars.
2018:08:07 19:01:36 hmaurer @U09R86PA4 isn’t it possible to attach metadata to the keyword representing the spec?
2018:08:07 19:01:52 favila you cannot attach metadata to keywords
2018:08:07 19:02:08 hmaurer @U09R86PA4 ah 😕
2018:08:07 19:02:14 favila
(with-meta :mykw {:abc 123})
ClassCastException clojure.lang.Keyword cannot be cast to clojure.lang.IObj clojure.core/with-meta--5142 (core.clj:217)
2018:08:07 19:02:22 hmaurer it’s really unfortunate; being able to attach doc to specs in this way would be really neat
2018:08:07 19:02:30 hmaurer (yep I just tried that :()
2018:08:07 19:02:51 favila they could do something, but it wouldn't be the normal var thing
2018:08:07 19:02:56 favila that's all
2018:08:07 19:02:58 hmaurer especially since the doc function is already able to pull specs from the registry and show the predicate
2018:08:07 19:03:11 hmaurer it would be super nice if it could also show an extra description
2018:08:07 19:03:17 hmaurer useful in cases where the predicate doesn’t speak for itself
2018:08:07 19:04:05 hmaurer I guess the implementation doesn’t really matter; it could be an extra global hashmap from keywords to strings
2018:08:07 19:05:57 alexmiller It’s the most voted issue in jira and we are def on board with fixing it
2018:08:07 19:06:36 alexmiller But there are impl factors meaning it will be a while before it’s implemented
2018:08:09 14:46:40 triss Is there an easy way to flush the clojure spec library? I’m refactoring and am never sure I’ve got ll my specs right until I restart clojure.
2018:08:09 14:50:49 favila reset! the registry to empty then re-evaluate all namespaces with specs in them?
2018:08:09 14:52:24 triss ah ok. Just reset an atom… but where is the registry atom?
2018:08:09 15:36:10 favila it's private so you do need to do some poking
2018:08:10 00:07:41 ackerleytng What's a good way to kind of namespace specs internal to a file? So the issue is that s/keys needs to have a spec named, say foo
2018:08:10 00:09:28 ackerleytng So I would do ::foo. However, my function returns a modified map, hence the :ret also has to be something that has s/keys with ::foo
2018:08:10 00:09:51 ackerleytng And many functions in that file return modified maps
2018:08:10 00:10:18 ackerleytng So I kind of want to namespace the spec for this function
2018:08:10 00:11:03 ackerleytng I'm now doing :function-name-pre/foo
2018:08:10 00:11:30 ackerleytng Anyone has a better solution?
2018:08:10 00:12:16 taylor do you need qualified keys in the map? or are you going to have multiple */foo keys?
2018:08:10 00:12:35 taylor if not, why not use unqualified keys?
2018:08:10 00:13:28 taylor or maybe I misunderstand the question
2018:08:10 00:15:21 ackerleytng The s/keys already spec it as unqualified
2018:08:10 00:15:27 ackerleytng Because of that
2018:08:10 00:15:50 ackerleytng The actual key has to have a name that matches the key in the map that is specced
2018:08:10 00:17:38 taylor sorry, not sure I understand what the question is then
2018:08:10 00:25:52 ackerleytng Thanks! Haha
2018:08:10 00:26:31 ackerleytng Maybe in short it's "how do you group specs for a function?"
2018:08:10 00:37:19 seancorfield @ackerleytng I'm not sure I understand the problem you think you have... why does :ret come into this and cause you a problem?
2018:08:10 00:37:50 seancorfield Perhaps if you shared a concrete example?
2018:08:10 00:38:06 ackerleytng Sure hmm I'll work one out and post it later
2018:08:10 17:01:32 bja is ::s/invalid part of the public api?
2018:08:10 17:02:22 seancorfield I don't believe so but there's a function s/invalid? you can use to test for that value.
2018:08:10 17:03:07 seancorfield (that said, I've written conforming predicates that return either an updated value or ::s/invalid a few times so...)
2018:08:10 17:18:13 dadair I have a spec that checks that one of the keys is a core.async channel; I’d like to write a custom generator for that specific key spec, but I can’t seem to determine how to write a generator that returns a channel; I feel like I’m missing something obvious here
2018:08:10 18:18:29 dadair Hmm that would probably work! Thanks!
2018:08:10 18:10:06 bja thanks @seancorfield
2018:08:10 20:17:14 alexmiller yes, ::s/invalid is part of the public api @bja @seancorfield
2018:08:10 21:04:36 seancorfield Thanks for the clarification @alexmiller!
2018:08:10 23:47:10 bbrinck I’ve created a new channel for expound-related questions - #expound
2018:08:11 03:41:10 myguidingstar this spec used to work in 0.1.x (s/explain (s/and map? (s/+ (s/or :pair (s/cat :key keyword? :value boolean?)))) {:foo true}) but with 0.2.168 it fails predicate: (or (nil? %) (sequential? %))
2018:08:11 03:44:39 myguidingstar I use that spec to conform some hash-map's key/value
2018:08:11 03:47:53 myguidingstar but it looks like new clojure.spec use sequential? for s/+ therefore doesn't allow hash-maps
2018:08:11 03:48:20 myguidingstar what should I do?
2018:08:11 03:52:53 myguidingstar that spec is actually used to describe a flexible dsl composed of vectors and hash-maps (the dsl itself is used as part of om.next queries)
2018:08:11 03:53:36 myguidingstar I can't think of any viable workaround
2018:08:11 03:55:33 myguidingstar also, is there any reason clojure.spec use sequential? there instead of coll? which is almost identical except that it returns true for sets and maps?
2018:08:11 04:54:17 seancorfield @myguidingstar Why not just use (s/map-of keyword? boolean?)
2018:08:11 04:58:21 seancorfield (it took me a while to read your spec -- what does s/or even mean with just one named branch?)
2018:08:11 05:05:28 lilactown is it possible to have a recursive spec?
2018:08:11 08:59:15 myguidingstar @seancorfield yup, that s/or is for named branch
2018:08:11 08:59:55 myguidingstar I need to tag the key and value, hence the use of s/cat
2018:08:11 09:00:58 myguidingstar so s/map-of doesn't work for me
2018:08:11 09:10:09 myguidingstar that spec's purpose is to used with s/conform, kinda like a dsl parser, not just to check validation
2018:08:11 18:31:47 seancorfield @myguidingstar I wonder if (s/and map? seq (s/+ ...)) would make it so what you need?
2018:08:11 22:28:36 beta1036 I have specd a domain entity in my application and have to read instances encoded as JSON where the keys are without namespaces. What's the recommended way to validate and convert the JSON encoded instances to the internal representation?
2018:08:13 13:11:17 rapskalian @U4844LY6A I wrote a little library that uses spec to automatically qualify/unqualify domain entities. Given that JSON uses string keys, you’ll have to walk/keywordize-keys first on your map as this library currently only works with keyword keys.
https://github.com/cjsauer/disqualified/blob/master/README.md
2018:08:13 17:01:05 beta1036 You use qualify-map first and then conform the result to validate the input, right?
2018:08:13 20:46:50 beta1036 I've had a look. I'm afraid in order to extend it to my use case (embedded maps, collections, conjunctions, etc.) I'd have to re-implement a lot of spec. handling or this way seems to be particularly tricky.
2018:08:12 03:53:20 ackerleytng @seancorfield I have an example here: https://pastebin.com/Kuat3MPV . That's what i'm doing now, but is there a better way of "namespacing" specs?
2018:08:12 04:40:25 seancorfield @ackerleytng OK... and what's the question again? That seems OK to me on the face of it...
2018:08:12 04:45:40 ackerleytng ah ok. I just thought there might be a better way of namespacing all of that
2018:08:12 04:45:46 ackerleytng thanks then!
2018:08:12 15:36:41 lilactown so I have this spec:
2018:08:12 15:48:26 lilactown what’s the difference between writing :projection ::projection and :projection (s/spec ::projection)?
2018:08:12 15:50:59 taylor by default all the regex-type specs “flatten” themselves when nested
2018:08:12 15:52:21 taylor wrapping a regex-type spec in s/spec prevents that flattening, so you can use them to describe nested sequences of things
2018:08:12 16:08:13 lilactown I don’t understand what you mean by flattening
2018:08:12 16:09:13 lilactown or why something “flattening” would cause a stackoverflow
2018:08:12 17:00:56 taylor the stack overflow is just because of how spec is implemented, if you have a certain form of nested/recursive regex specs it recurses infinitely trying to evaluate it
2018:08:12 16:48:53 taylor @lilactown FWIW you can also define your spec using s/or instead of s/alt, and then you don’t need the s/spec call to prevent the regex flattening:
2018:08:12 16:49:59 taylor by “flattening” I mean that when you nest the regex-type specs, by default they merge/compose/combine/maybe-a-better-word to describe a single sequence
2018:08:12 16:57:43 taylor but what if you wanted to use regex specs to describe nested sequences? the “flattening” behavior wouldn’t allow it, so you could use s/spec to avoid it:
2018:08:13 01:48:31 lilactown I think I understand. thank you for the thorough explanation!
2018:08:13 15:12:51 triss so I’ve got a few namespaces - each one responsible for managing the contents of particular type of map. Is there a convention in clojure for what to call your “constructer” method? Do you name it after the type of the type of object you’re going to create (already in the namespace?) or use a word like create or construct?
2018:08:13 15:38:08 guy (def job {:name "some job"}) like that you mean?
2018:08:13 15:38:49 guy (defn job [some-args] ...)
2018:08:13 15:42:37 lilactown @triss I think this is a better discussion for #clojure , but I try and elide create or make whenever possible
2018:08:13 15:44:01 triss ah yes you’re probably right re: #clojure…
2018:08:13 15:45:06 triss the annoying thing is most of my constructers just do (s/conform ::definition args)
2018:08:13 15:45:41 triss but that’s way too much to type…
2018:08:13 15:46:06 lilactown I see what you’re saying though. If i had a particular namespace, like my-app.job and in there were all functions that work on the job entity type, then I would probably just have a create fn
2018:08:13 15:46:07 triss or is it… maybe I should just call (s/conform directly) in other namespaces?
2018:08:13 15:46:24 lilactown that might be my OCaml bleeding through tho
2018:08:13 15:47:31 lilactown it’s probably better to hide the fact that all you’re doing is s/conform in case you ever want to do more?
2018:08:13 15:48:06 triss that was my thinking I guess
2018:08:13 15:48:16 triss but it is very rare I do more these days…
2018:08:13 19:28:19 mikerod Is it necessary to forward-declare specs that are referenced from other specs in a case such as mutually recursive specs?
Contrived example:
2018:08:13 19:29:03 taylor not in my experience, as long as they're both registered by the time you use them
2018:08:13 19:29:06 mikerod I believe, at least in some circumstances, that the forward-use of a spec isn’t resolved until later anyways, so the forward declaration isn’t needed (maybe I’m wrong)
2018:08:13 19:29:19 mikerod @taylor that is what I thought I was seeing as well
2018:08:13 19:29:47 mikerod and looking at how the references are used in a various impl’s I checked in clojure.spec.alpha it seemed like it was ok
2018:08:13 19:30:58 mikerod but I haven’t seen any direct discussions on it really. I’ve seen at least one posts in the wild that did something like
2018:08:13 19:31:27 taylor agree, seems unnecessary
2018:08:13 19:32:12 taylor the keyword names are like "pointers" to the actual specs that get stored in the spec registry, so as long as the pointer points to a registered spec by the time it's used, I'd assume all's well
2018:08:13 19:52:15 mikerod yep
2018:08:14 16:45:35 triss is it just me or does spec cause me to end up with circular dependencies in namespaces? what’s the workaround?
2018:08:14 17:35:28 guy instead of using ::my-spec i create fake ns’s
(s/def :guy-project/user (s/keys :req-un [:guy-project/name]))
2018:08:14 17:35:57 guy Then u can store those spec wherever u want
2018:08:14 17:36:08 guy spec/user.clj
2018:08:14 17:36:24 guy and import it where u need it
2018:08:14 17:36:36 guy if that makes sense? 👀
2018:08:14 17:43:49 manutter51 That’s how I do it too. I think it makes for more fine-grained namespacing.
2018:08:14 18:50:39 Ethan I am working on a project to rewrite error messages and I was wondering if anyone knows any libraries that are well specced so I can test my spec error rewriting on more than just my specced functions and the specs in clojure.spec.alpha
2018:08:15 20:06:54 ericnormand Can someone tell me what :opt is for? How is validation affected by its presence?
2018:08:15 20:07:22 ericnormand I'm talking about :opt as an option to keys
2018:08:15 20:10:28 guy As far as i’ve understood it,
if you have say (s/keys :opt [::name])
If the key ::name is present it will check it against the spec.
2018:08:15 20:11:03 guy So i’ve used it in the past to say, If this thing is present then it has to look like this spec. Otherwise carry on.
2018:08:15 20:12:57 guy So if i just make up an example with some pseduo code
2018:08:15 20:14:45 guy
(s/def ::name string?)
(s/def ::type string?)
(s/def ::person (s/keys :opt [::name] :req [::type]))
(s/valid? ::person {::type "human})
=> probably true
(s/valid? ::person {::name 124 ::type "human"})
=> false because name is not a string
2018:08:15 20:14:59 guy I’ve just written these by hand so don’t trust the code 1 for 1
This registers a ::person spec with the required keys ::first-name, ::last-name, and ::email, with optional key ::phone. The map spec never specifies the value spec for the attributes, only what attributes are required or optional.
When conformance is checked on a map, it does two things - checking that the required attributes are included, and checking that every registered key has a conforming value. We'll see later where optional attributes can be useful. Also note that ALL attributes are checked via keys, not just those listed in the :req and :opt keys. Thus a bare (s/keys) is valid and will check all attributes of a map without checking which keys are required or optional.
A rather large snippet
2018:08:15 20:16:41 guy Hope that helps!
2018:08:15 22:02:05 ericnormand Hey @guy! Thanks!
2018:08:15 22:02:31 ericnormand However, if I do (s/keys), it still checks that ::name is valid.
2018:08:15 22:04:28 guy Yeah but what does ::person check for
2018:08:15 22:04:34 guy :man-shrugging:
2018:08:15 22:04:39 guy part of why i use opt is documentation
2018:08:15 22:04:46 guy your point is valid though and its been asked before
2018:08:15 22:04:47 ericnormand yes, documentation is good
2018:08:15 22:04:49 guy let me find it
2018:08:15 22:04:52 ericnormand thanks
2018:08:15 22:05:21 guy ah bummer the historys gone
2018:08:15 22:05:31 guy Another person asked something similar but a different way
2018:08:15 22:05:48 guy saying why use :opt at all when (s/keys) does the job
2018:08:15 22:06:05 guy I think for me its documentation + being explicit about what you want to check for
2018:08:15 22:07:11 ericnormand okay, thanks
2018:08:15 22:07:36 ericnormand the spec guide on http://clojure.org hints that it'll explain later, but it never does
2018:08:15 22:07:43 ericnormand I thought it might be more than just docs
2018:08:15 22:07:55 mpenet Also gen
2018:08:15 22:07:59 ericnormand oh!
2018:08:15 22:08:06 ericnormand good call
2018:08:15 22:08:11 mpenet Kind of important ;)
2018:08:15 22:08:37 guy I think when it comes to speccing out ur fn as well it might be awkward depending on how u use (s/keys)
2018:08:15 22:09:17 guy I’m unsure if you would really generate a spec like (s/def ::person (s/keys))
2018:08:15 22:09:34 ericnormand for sure
2018:08:15 22:09:54 guy you might just use (s/keys) as an anonymous spec inside a (s/fdef ..)
2018:08:15 22:10:03 guy i’ve probably got the terminology wrong
2018:08:15 22:10:26 guy but that would also lead to harder reading when it comes to spec messages when they have failed
2018:08:15 22:10:56 guy but yeah
2018:08:15 22:11:18 guy i would favour :opt over (s/keys)
2018:08:15 22:11:25 guy just from a readability factor
2018:08:15 22:11:30 guy but im just one guy 😄
2018:08:15 22:34:54 lwhorton trollface
2018:08:15 22:47:58 gfredericks is there also a difference when the spec is not defined?
2018:08:15 22:48:27 gfredericks i.e., putting things in :opt forces you to define the specs you listed (i.e., prevents you from accidentally forgetting them)?
2018:08:15 23:47:13 ericnormand @gfredericks a little repling shows that opts does not check if the spec exists
2018:08:16 15:15:25 guy Can you give an example?
2018:08:16 15:15:49 guy Like do you mean just doing something like
(s/keys :opt [::fake-spec]) ?
2018:08:16 15:24:09 dadair That fact allows me to avoid circular spec dependencies; the s/keys doesn’t validate that the spec exists
2018:08:16 18:59:19 ericnormand @guy that's what I mean
2018:08:16 00:37:08 gfredericks oh weird
2018:08:16 00:37:11 gfredericks does :req?
2018:08:16 02:20:22 ericnormand don't think so
2018:08:16 02:20:39 ericnormand there's a pretty strong separation between checking keys and checking their values
2018:08:16 02:20:49 ericnormand all values are checked, even if they're not required
2018:08:16 02:21:02 ericnormand and all keys are allowed, even if they're not defined
2018:08:16 12:03:55 gfredericks yeah I guess that's consistent at least
2018:08:16 18:59:38 ericnormand yeah
2018:08:16 18:59:56 ericnormand I think that's why it's called "keys". It's just about the keys, and whether they exist.
2018:08:16 19:34:39 joost-diepenmaat though you will get errors if you try to generate values for keys specs with unspec’d keys
2018:08:16 20:13:57 guy ^ yeah thats why i think i’ve never found it to be a problem
2018:08:16 20:14:10 guy Whenever i’ve tried to exercise it for an fdef or whatever
2018:08:16 20:14:18 guy it would throw that 100 tries error etc
2018:08:16 20:14:21 guy iirc
2018:08:16 22:04:51 richiardiandrea can I stest/instrument a function that has been replaced with with-redefs ? would the spec still work on it?
2018:08:17 01:50:38 alexmiller I think both are working by swapping vars, so probably, but it might get weird. try it and see.
2018:08:17 02:03:38 richiardiandrea Seems like it is when the with-redefs happens before the instrument. But I've got mixed feelings about this idea 😄
2018:08:17 02:04:12 richiardiandrea Probably better not to do that
2018:08:17 02:04:16 alexmiller :)
2018:08:17 10:08:44 olivergeorge Is this a common pattern: isolate a fn by stubbing everything else and running stest/check on it.
2018:08:17 12:10:16 alexmiller First time I’ve seen it, but interesting
2018:08:18 01:28:35 olivergeorge I was thinking it would give you confidence that the inputs and outputs through the code match up. So essentially a cheap type checking substitute.
The downside of that approach is that for full code coverage the generated data needs to trigger all branches of conditionals… I think that’s probably not something you can rely on.
2018:08:17 17:21:09 colinkahn Looks like it isn't an option, but curious if there's a way to specify an fspec for check like you can for excercise-fn
2018:08:17 17:25:09 colinkahn My goal is to check a fn with an fspec whose :fn is specific to a more narrow generator.
2018:08:17 18:00:17 alexmiller you can do that with instrument first
2018:08:17 18:01:09 alexmiller with the :replace functionality
2018:08:17 18:01:16 alexmiller if I understand your goals that is
2018:08:17 18:19:40 colinkahn Looking at the docs for instrument I think :spec should do what I want, but trying the following still uses the registered fdef for files-reducer instead of the override fspec
2018:08:19 06:54:07 fedreg Hi all, Is there a way to see the data that is returned from a function that I call stest/check on? For example, I can see all the data generated for that function’s inputs by passing in a reporter-fn to the opts but I can’t seem to find a way to access that data for the returns. I’d love to pass a function in to format the output data as needed. thx for any pointers!
2018:08:19 11:32:19 gfredericks is there a coll-of?
2018:08:19 13:39:23 pablore yes, but it doesn’t work well with s/cat.
2018:08:19 13:44:25 gfredericks oh so you want something that can go in a regex in particular?
like (){12} from string regexes?
I can imagine a macro that expands to an s/cat with 12 instances of the spec, but that'd be unfortunate. seems like something that could be done better built-in
it does seem uncommon though; where did this come up for you?
2018:08:19 11:32:27 gfredericks sorry, that was for @pablore
2018:08:19 11:34:42 gfredericks @fedreg there might not be a supported way, but if you make a TCHECK jira ticket I'll make sure it's at least available to the reporter-fn
2018:08:19 13:47:39 gfredericks Actually, now that I look closer, I don't think this can be fixed in test.check -- it's spec that's hiding the return value
2018:08:19 13:48:23 gfredericks spec creates a test.check property that just returns true when the test passes
2018:08:19 15:59:32 fedreg @gfredericks Thanks for taking a look!! (and making the ticket!).. Ya, I saw that it was coming back ok from test.check… Seems like there should be a way to see the results… I wonder if the :result is blocked by spec intentionally or if it is just an oversite…
2018:08:19 16:00:10 gfredericks it'd have to be wrapped if anything
2018:08:19 16:00:33 gfredericks spec is returning true, and test.check checks for truthiness; so to test functions that can return falsey you'd have to wrap with {:return ___} or something like that
2018:08:19 16:01:32 gfredericks does stest/check return that true also, or do you only see it in the reporter-fn?
I think it's probably only the reporter-fn, in which case I don't see a big downside to spec doing that wrapping
2018:08:19 16:02:11 gfredericks it can't reasonably leak anywhere else, because the return value from a full quick-check run can't include the return value since there's n of them instead of 1 of them
2018:08:19 16:30:25 fedreg @gfredericks sorry, not sure which true you mean… The return data is evaled here but then just a true is returned if everything is ok. So, if all is good with the specs, just a true is returned. Details only returned if there is a spec validation
2018:08:19 16:32:23 gfredericks If you replaced that line with {:return ret} I think you'd have what you need and nothing else would break
2018:08:19 16:32:35 gfredericks Or at least nothing that wouldn't be easily fixed
2018:08:19 16:32:54 gfredericks But I'm not a spec lawyer so I dunno
2018:08:19 16:39:57 fedreg Yes... I'll file a ticket. Maybe I'm the only one who would find that useful.. thanks for taking a look regardless @gfredericks !
2018:08:19 22:39:00 jeaye What's going wrong here? It seems odd that args would be treated any differently than the s/valid? checks prior.
2018:08:20 00:01:54 alexmiller per the s/fdef doc string, ” :args A regex spec for the function arguments as they were a list to be passed to apply - in this way, a single spec can handle functions with multiple arities”
2018:08:20 00:03:35 alexmiller or I guess maybe the confusing thing is that regex ops nested inside each other are combined to describe the same sequential collection
2018:08:20 00:04:21 alexmiller so you have an effective foo args spec of (s/cat :aa (s/cat :name keyword?)), however nested s/cats like this combine to form a spec for a single sequential collection (the args list)
2018:08:20 00:04:49 alexmiller what you’re wanting is that ::any-args described a new nested collection
2018:08:20 00:05:54 alexmiller you can do that with (s/fdef foo :args (s/cat :aa (s/spec ::any-args)))
2018:08:20 00:06:09 alexmiller the s/spec wrapper here forces a new (non-combined) collection level
2018:08:20 01:43:10 jeaye Thanks, Alex. Got it.
2018:08:20 01:45:28 jeaye @alexmiller Is there any issue with using (s/spec ...), or unintended side effects? I'm thinking about defn-spec, within Orchestra, which provides fn specs inline. It seems like defn-spec should expand every arg spec to be wrapped in s/spec, based on your explanation.
2018:08:21 17:01:15 djtango I've run into issues with regex flattening in arg-specs so I think you're right here
2018:08:21 17:01:32 djtango (w.r.t. using s/spec how you've described)
2018:08:20 01:46:18 jeaye Since none of them should ever flatten to affect the outer s/cat for the :args.
2018:08:20 02:53:08 alexmiller I can’t speak to what Orchestra or defn-spec does, sorry. There are possible use cases for wanting the flattening here, although I agree that’s probably unusual.
2018:08:20 21:49:25 richiardiandrea We were discussing a bit on #clojurescript about having the possibility to specify a custom printer (a la *explain-out*) for check results.
2018:08:20 21:49:43 richiardiandrea Is it something worth working on, or at least opening a JIRA about?
2018:08:23 09:13:32 lmergen i'm struggling a bit on how to design my spec rules. specifically, i have a complex object that i want to spec, and this object can have several traits, depending upon where it is.
so, for example, let's say i have an animal object with the following definition:
so here we can see that an animal has a optional medical information about vaccinations. most of my functions just work with animals, and do not care whether some specific nested property it true. however, some other functions do care whether an animal is vaccinated. for now, i have this:
but this feels a bit hacky, and doesn't scale very well when you want to compose multiple properties into it. it also feels like i'm developing a poor man's trait system here, which is maybe the intention, maybe not
regardless, what would be the proper way to organize this stuff be?
2018:08:23 09:14:55 guy are you trying to do s/merge maybe?
2018:08:23 09:17:11 lmergen i'm not sure how to structure that in this specific case
2018:08:23 09:18:15 lmergen would i s/merge the ::animal into an ::vaccinated-animal ? but how would i re-define a property of a nested map in this case (i.e., :req-un the ::vaccinations rather than :opt-un) ?
2018:08:23 09:19:19 guy Can you show me an example shape of the data?
2018:08:23 09:19:28 lmergen hmm ? the spec is up there ?
2018:08:23 09:20:01 guy (def animal {:medical [{:foo :bar}]})
2018:08:23 09:24:49 guy this is sorta what i would do in ur case
2018:08:23 09:24:58 guy Naming conventions for nested maps
2018:08:23 09:25:06 guy If you read that part
2018:08:23 09:27:14 guy The reason i think the above is useful to you is it allows you to use ns’s to sorta describe what you are talking about
2018:08:23 09:27:27 guy Which is ultimately vaccinated animals vs just animals
2018:08:23 09:27:27 lmergen right! this is useful
2018:08:23 09:27:41 lmergen ok
2018:08:23 09:27:43 guy Does that help at all?
2018:08:23 09:27:45 lmergen yes
2018:08:23 09:27:48 guy ok great
2018:08:23 09:27:50 guy 😅
2018:08:23 09:28:02 lmergen so basically, additional namespaces
2018:08:23 09:28:13 guy its defo one answer
2018:08:23 09:28:27 guy I like it as it makes your specs predictable
2018:08:23 09:59:10 conan yes, this! no surprises!
2018:08:23 11:21:52 dominicm I don't suppose anyone has created a spec for "edn" have they?
2018:08:23 11:49:12 guy I wanna see it 😱
2018:08:23 13:13:21 dominicm I want to test aero better, there's gaps where certain data structures don't work.
2018:08:23 13:13:45 dominicm One really hard part is getting it to generate #ref
2018:08:23 14:40:48 bbrinck It depends on what you want to test, but on strategy for building generators like this is to some from a small set of valid keys OR random keys
2018:08:23 14:42:06 bbrinck Or am I misunderstanding what is difficult about generating refs?
2018:08:23 14:56:31 dominicm I want to generate valid nested paths, e.g. #ref [:adjfkf :bjkadf :foobar]
2018:08:23 14:56:54 dominicm which can have many levels of nesting
2018:08:23 14:58:18 bbrinck Is the goal of the generator to produce refs that will always resolve, or just be valid syntax (but maybe not resolve), or or a mix of resolved and unresolved refs?
2018:08:23 14:59:57 bbrinck i.e. do you want all paths to be valid (assuming “valid” means “it resolves to some actual path in the map”) or is it OK if some are invalid?
2018:08:23 15:14:17 dominicm A mix would be useful, but you can't spec that an exception happens can you?
2018:08:23 15:14:36 dominicm I think nil is the current behaviour, but I don't like that for other reasons.
2018:08:23 15:16:06 bbrinck That’s true, you can’t spec an exception occurs, although you could generate EDN with a mix and then use a normal generative test (i.e. not using check directly) to say either an exception should occur or some other thing should happen
2018:08:23 15:17:50 bbrinck Anyway, if you want a mix, one thing you could try is to normally name keys from a small set in the generator e.g. {:a :b :c} and then generate refs of a usually small length from those same keys e.g. (s/coll-of #{:a :b :c} :kind vector?). Basically you’re just assuming you’ll get some number of “collisions” but accepting some refs won’t resolve
2018:08:23 15:19:40 bbrinck If you want to ensure every path is going to resolve, then I think you’ll have to build a custom generator for the entire config. I’d start by generating a set of paths that will be included in the map. Then, you can take those valid paths and write a function that fills in the values for each path, which may contain a ref to another valid path
2018:08:23 16:50:25 dominicm Starting with valid paths sounds like a great idea!
2018:08:23 16:50:47 dominicm Especially as some things will be sets or vectors.
2018:08:23 17:16:45 bbrinck If the generator works a high percentage of the time, you could also just generate a lot of examples and filter out ones where the refs are invalid
2018:08:24 10:40:33 misha true is because (s/keys :req (var-get some-keys)) is equivalent to empty (s/keys)
2018:08:24 10:41:18 misha (var-get some-keys) this does not work inside s/keys the way one would expect.
(first of all, it should be (var-get #'some-keys))
2018:08:24 11:07:19 guy @conan this is good to know!
2018:08:24 12:49:09 conan ah yes, thanks for pointing this out!
2018:08:24 10:41:41 misha at least in clojure. mb in cljs it does
and it becomes hairy very-very soon with very little nested structures.
and unless you will use -un (which defeats whole purpose of namespaces in just few calls down the call stack) - your get-ins and update-ins become unreadable and unbearable
2018:08:24 10:47:26 misha instead, I went with shorter namespaces, reflecting domain objects/classes, w/o including nestedness info into spec namespaces.
2018:08:24 10:49:22 misha example above does not look too terrible, but it is really a hello-world grade one, and is already whole line long.
2018:08:24 11:07:28 guy Thanks @misha really good to know
2018:08:24 12:52:55 conan we have complex nested structures and don't really encounter a problem, but maybe we have more complex but less nesting - going down our tree, we quickly hit refs to other top-level entities, so maybe that keeps it ok. I always prefer having predictability over terseness, i don't really see much value in the latter.
2018:08:24 12:54:32 conan in that example above for instance, it's likely i wouldn't chooes to model an employee as something in the company hierarchy. i'd be more likely to have a :conan/company containing :conan.company/departments, and each of those containing lots of :conan/employees (rather than continuing the nesting).
2018:08:24 12:55:31 conan or at the very least, if employees were in the company hierarchy, i'd look to break out of it and give each employee a :conan/person, i.e. a thing that exists independently of the company hierarchy but that can be attached.
2018:08:24 12:59:41 conan The pattern where :my/a has a collection of :my.a/bs, each of which is a :my.a/b, which has a collection of :my.a.b/cs, each of which is a :my.a.b/c and so on works very well for us
2018:08:24 13:00:36 conan but from a modelling perspective it's often useful to consider breaking that pattern when possible, so that a :my.a.b/c ends up with a collection of :my/zs
2018:08:24 13:00:53 conan it's tricky to read this stuff without a concrete domain in mind
2018:08:24 13:16:26 misha @conan I am not choosing terseness for the sake of terseness, I am suggesting to think twice before embedding "how one domain object is nested in another one maybe-only-in-just-one-out-of-many-existing-contexts" into spec namespaces
2018:08:24 13:19:21 misha for example, if there is a code which manipulates some leaf of a deep nested structure outside of the nested structure context, does it really need to know that it is :foo.bar.baz.quux/fred and not just :quux/fred?
2018:08:24 13:19:39 conan absolutely, gotta pick what works for you. everything we do is stored in datomic, so this is always our graphy structure
2018:08:24 13:20:13 misha ofc, the easy escape would be :*-uning all the keys. but that is just robbing yourself of power namespaces gave you
2018:08:24 13:20:18 conan in your example though, your function may well not need to know that - but as a developer, I do, and I don't like surprises
2018:08:24 13:20:41 conan yeah unqualified specs are just for data coming from external systems imho
2018:08:24 13:21:22 conan i'm definitely not saying this is the "right" way to do it, not even the only way - it's just intended as a useful starting point if you're unsure. I'll update the post to make that more clear
2018:08:24 13:21:27 misha I agree with what you are saying about knowledge, but only in very limited set of usecases
2018:08:24 13:22:41 misha and I'd say, those use cases are solved by namespaces for functions foo.bar.baz/quux-fred-manipulator vs e.g. quux/fred-manipulator (dont read too much into it)
2018:08:24 13:24:11 conan really appreciate the feedback though
2018:08:24 13:24:18 misha the same way as you can have your own update fn, more specific than clojure.core/update one, in another ns
2018:08:24 13:24:51 misha with similar semantics, but with more knowledge about thing it sh/would be applied to so you-as-developer-would-know
2018:08:24 13:25:07 conan yep, i can see that; we benefit a lot from transacting data generated from these specs into datomic, and it may colour my thinking more than i realise
2018:08:24 13:26:29 misha oh, I tried keeping datomic db for such nested nss - was tough. and again - you are robbing yourself of a graph, replacing it with rigid tree
2018:08:24 13:26:30 conan it's really quite astonishing that i can often write specs for a domain, write schema from those specs, and the generators will produce data that transacts first time into datomic; i don't think i've ever once come close to that level with other databases
2018:08:24 13:26:46 conan we have a very graphy structure, and this works well with it
2018:08:24 13:27:01 misha that is true, that potential is awesome!
2018:08:24 13:27:49 conan essentially, anything that has isComponent ends up being another level of namespace hierarchy (i.e. :a.b.c/d), and anything that is not usually resets back to the top of the hierarchy (i.e. :e/f)
2018:08:24 13:29:39 misha it is very close to that, yes, I agree.
although company.department/employee, we crucified above - does not fit that opieop
2018:08:24 13:30:31 conan yeah i'm terrible at examples, i'll think up a more evocative, less boring one over the weekend
2018:08:24 13:31:44 misha that's fine, it is an example from linked article anyway
2018:08:24 13:33:45 misha I have an example, contradicting deeply-nested isComponent rule: :geopoint/lat/long
2018:08:24 14:36:37 bja is there anything like the plumbing.core/defnk (which used Schema) for spec?
2018:08:24 16:08:59 snowell I have a function I'm trying to spec that takes 2 args, a map and a string that needs to be a key in that map. How do I reference the first arg in the spec for the second?
2018:08:24 16:17:14 taylor @snowell you could use s/and with s/cat to check the args:
2018:08:24 16:18:07 snowell Yeah, I'd gotten there actually, but I was trying to get something that worked with generators, and apparently that's a lot more fickle
2018:08:24 16:18:25 snowell Unless it happens to randomly generate the right string(s)
2018:08:24 16:18:42 taylor hmm yeah because the OotB generators are created using just the first spec in the s/and
2018:08:24 16:19:26 taylor but I don't think it'd be too hard to make a generator that will do this
2018:08:24 16:19:31 snowell It's not the end of the world. Just playing with a new toy 🙂
2018:08:24 16:27:07 taylor @snowell this might work:
2018:08:27 09:40:52 ikitommi the result doesn’t conform anymore to the spec.
2018:08:27 11:51:22 alexmiller that’s often true regardless
2018:08:27 12:09:28 misha @alexmiller have you considered doing this (qualify keys) as part of s/keys or s/conform behavior?
2018:08:27 12:18:08 alexmiller I am confused by your question
2018:08:27 12:31:08 misha opieop will such qualify be a part of clojure.spec someday? if "definitely not", why?
2018:08:27 20:26:33 eoliphant hi, quick question. what's the best way to handle getting 'context' into spec functions (or is this even a good idea?) For instance, I'm playing around with a little data-driven schema definition tool, and I want an attribute type to be either one of a set of primitive types, or another type defined in the schema. so I want to do something like (s/or ::attribute-type #{:type1 :type2} (fn [val] (is-val-in-list val external-list)). where external-list might have been setup right before a call to s/valid?, etc
2018:08:27 21:38:44 misha you might keep context in an atom, and access it inside of your predicate
2018:08:27 23:40:47 eoliphant yeah i'd been thinking about that, but it feels weird lol. I'd really like to have the context scoped down to what's going on at that time, but I may give that a whirl to start
2018:08:28 05:41:46 misha You control what goes into that atom, and when. After all, global spec registry is an atom too.
2018:08:28 17:30:58 Ulrich I do not understand this mismatch, as I see it. Do you know the reason why :gen takes a function and not a generator? (s/exercise boolean? 1 gen/boolean) ok but (s/exercise (s/spec boolean? :gen gen/boolean) 1) bad. So I write (s/exercise (s/spec boolean? :gen (constantly gen/boolean)) 1). Is there a better way?
2018:08:28 18:11:24 alexmiller Specs defer gens via functions to avoid loading clojure.spec.gen.alpha when not necessary. When you call exercise, it takes an overrides map which is a map of spec name to function returning generator.
2018:08:28 18:13:40 alexmiller (s/exercise boolean? 1 gen/boolean) is not ok, that last gen/boolean is just not being used (boolean? has a built-in generator)
2018:08:28 18:15:13 alexmiller I usually do something like (s/exercise (s/spec boolean? :gen #(gen/boolean)))
2018:08:28 18:16:58 alexmiller I suspect something is wacky in the macrology that’s making the constantly version not work there
2018:08:28 18:40:04 seancorfield @alexmiller I'm curious as to why the unwrapped version worked in my REPL then...?
2018:08:28 18:40:36 alexmiller unwrapped version of?
2018:08:28 18:41:26 seancorfield Seems to work both with and without the #( ... ) around gen/boolean...
2018:08:28 18:42:02 alexmiller not sure - since this one has a built-in, that might be masking something too
2018:08:28 18:52:47 alexmiller oh, so gen/boolean (where gen is clojure.spec.gen.alpha) is actually already a thunk
2018:08:28 19:01:36 alexmiller so gen/boolean works as is and #(gen/boolean) is a function that invokes gen/boolean and returns the generator, so is exactly the same thing
2018:08:28 19:02:16 alexmiller but (constantly gen/boolean) is a function that returns gen/boolean not invoked, which is a function rather than a generator
2018:08:28 19:03:35 alexmiller so going way back to the top @ulrich.becker said “(s/exercise (s/spec boolean? :gen gen/boolean) 1) is bad”, but it’s not bad - that’s exactly what you want here (the missing piece of information being that clojure.spec.gen.alpha already wraps gens in thunks
2018:08:28 19:05:22 alexmiller tumbleweed
2018:08:28 19:38:41 seancorfield Ah!! 💡 Now it all makes sense. That also explains why this did not work: (s/exercise (s/spec boolean? :gen (fn [] (println "Gen!") gen/boolean)) 1)
2018:08:28 19:39:44 seancorfield (I wanted to see how often the thunk was called and when I corrected it to call (gen/boolean) the answer was "just once", regardless of the exercise size)
2018:08:29 20:30:04 Chris Has anyone done any research on 'learning' specs given a data set?
2018:08:29 20:31:22 Chris Wow, awesome! That was so fast I thought you were a bot!
2018:08:29 20:32:05 Chris Thanks!
2018:08:29 20:37:42 ghadi See ambrose bonnaire-sargeants talk from last year's Conj
2018:08:30 16:50:52 Daniel Hines I'm writing a web app that writes to our CRM. We have a bunch of third party plugins installed on this CRM that I'm not allowed to touch. One function of these plugins is to validate incoming user-inputted data; however, somewhere in the validation, the plugins are occasionally hitting fatal exceptions, but they give no clues as to what it might be. The third party won't provide any documentation, and decompiling their code is expressly forbidden in their EULA. Through significant trial and error, I've narrowed down the cause to specific combinations of input values, which appear completely arbitrary to me. There are about 100 different inputs, some of which have hundreds of possible values, and the order the inputs seems to matter, making this very difficult to address by hand. Would it make sense to use spec in this situation to systematically explore combinations of data cause errors?
2018:08:30 17:10:43 jeaye I've hit an issue with instrumentation wherein the instrument macro generates and returns a vector of symbols of those fns instrumented which is too large for the JVM to appreciate. So, I see Method code too large! failing the compilation instead. This seems like something spec will need to address, in the long term, as more people end up spec'ing more fns. Right now, looks like this project is around 300 spec'd fns and the issue is arising. Both Clojure and ClojureScript seem to be subject to the issue.
2018:08:30 17:14:54 ghadi can you give a repro for the case @jeaye?
2018:08:30 17:28:22 jeaye Yup. I'll have one up in a bit.
2018:08:30 17:56:25 jeaye A repro case in 20 lines or so. Couldn't get it going with Clojure, but it certainly happens with ClojureScript. When I look at the implementation of instrument for each, it'd seem both would be subject, but perhaps not.
2018:08:30 17:56:31 jeaye 100% repro here though.
2018:08:30 17:56:35 seancorfield @d4hines That sounds very much like the scenario Stu Halloway was talking about in one of his presentations on spec...
2018:08:30 17:58:22 seancorfield I don't know. I don't think I attended SL in 2016. May have watched that online tho'. I thought it was a talk from one of the Conj's or West's...
2018:08:30 17:58:51 Daniel Hines Ok, I'll look for it. Thanks!
2018:08:30 19:48:27 bbrinck Keep in mind that exercise will generate values that match the spec, but to get the power of shrinking (which is what you want, I expect), you’ll want to combine spec with test.check
2018:08:30 19:53:00 bbrinck I really like the test.chuck library for this, you can write something like the following (totally untested and written in slack, so beware 🙂 )
(deftest find-bug
(checking
"all valid inputs don't cause exception"
100
[args (s/gen ::args-spec)]
;; replace 'string?' with whatever the actual expected return type is
(is (string? (apply third-party-api/some-fn args)))))
2018:08:30 18:07:50 taylor IIRC there might be a talk he gave about writing specs for a Java charting library and finding a (JVM?) bug by "fuzzing" it with random spec checks?
2018:08:30 18:19:34 jeaye @ghadi Is that something which you can repro?
2018:08:30 18:27:56 dadair Has anyone encountered an open-source HL7-FHIR spec library? I have been creating one for work but I’d be interested in pulling it out into an open-source project if others wanted to collaborate on it.
2018:08:30 18:37:25 ghadi we would find that useful @dadair
2018:08:30 18:39:25 ghadi i think the compiler special case's top-level do's but not nested ones
2018:08:30 18:39:46 jeaye That macro's just making the fns though. That macro doesn't cause the error; the error is with instrumentation.
2018:08:30 18:42:32 jeaye If you comment out the (stest/instrument) form, it compiles just fine.
2018:08:30 20:42:59 pablore Is there anyway to decompose a spec definition? I want to use a s/keys spec to define a database table
2018:08:30 20:49:27 pablore oh using s/describe
2018:08:30 21:19:14 alexmiller or s/form
2018:08:31 09:51:06 misha sanity check: how do you share specs between libraries/modules-of-app?
2018:08:31 09:53:25 Ulrich Help! I need my generated test data to have unique ids in various places (not uuids, though, just positive integers). I hacked a stateful generator (using an atom) for that which cannot be the right way…? How do I do stateful generators? exercise and sample seem to have no way to allow a state…
2018:08:31 09:53:48 Ulrich @misha require the namespace - it’s magic
2018:08:31 09:54:38 misha meaning: package it as a shared lib itself?
2018:08:31 09:56:33 Ulrich maybe i misunderstood. If you want to use specs defined in one namespace, just require that namespace where you wish to use the spec. If it is in a lib the declaring namespace should be in the lib’s jar … Is that what you asked?
2018:08:31 09:58:37 mpenet @ulrich.becker you can do an ugly hack with gen/fmap with a closed around atom and a (gen/return nil)
2018:08:31 09:58:44 mpenet there's probably a better way tho
2018:08:31 09:58:59 mpenet something like that: (gen/sample (let [a (atom 0)] (gen/fmap (fn [_] (swap! a inc)) (gen/return nil))))
2018:08:31 09:59:21 codonnell @misha we share specs between a backend clojure repo and a frontend clojurescript repo at my work by putting the specs in a third repo that has the specs in cljc files. It works pretty well IMO.
2018:08:31 09:59:47 mpenet @ulrich.becker it's similar yeah
2018:08:31 10:00:14 mpenet but messing with (gen/return nil) is usually a smell
2018:08:31 10:00:35 mpenet this kind of breaks the purpose of using prop testing in the first place
2018:08:31 10:02:18 Ulrich But the need for unique ids is very real, somebody must have a better solution? (except using UUIDS)
2018:08:31 10:02:31 mpenet why uuids are bad?
2018:08:31 10:03:06 Ulrich not bad just ugly…
2018:08:31 10:03:06 mpenet you can also use something like gen/vector-distinct of ints
2018:08:31 10:03:27 Ulrich nope vector/distinct does not work
2018:08:31 10:03:30 mpenet ah?
2018:08:31 10:21:47 mpenet right, it would return a collection, not really useful
2018:08:31 10:22:35 mpenet I guess another alternative is to let the random part untouched and keep track of values used in one way or another, I guess again you can do that with an atom or something. Still feels a bit odd
2018:08:31 10:23:36 mpenet but that atom will grow, and that might hurt
2018:08:31 10:25:28 mpenet it can also fail to generate valid values after a while
2018:08:31 10:26:04 mpenet so yeah, (g/return nil) 🙂
2018:08:31 10:29:59 mpenet just for fun another dirty trick: (gen/fmap hash (gen/uuid)), but you would get neg values, so (gen/sample (gen/fmap #(bit-and (hash %) 0x7FFFFFFF) (gen/uuid))) would work (with potential collisions tho)
2018:08:31 11:15:33 misha @codonnell will sending specs as an edn make any sense?
2018:08:31 22:56:17 codonnell I'm not sure what you mean by this. Could you elaborate?
2018:09:01 06:26:48 misha front-end sends "back-end, give me your specs"
back-end responds with subset of spec registry as edn
front-end reads, evals
2018:09:01 06:27:43 misha this skips fron-end re-compilation if specs change
2018:09:01 12:25:18 codonnell I suppose you could do this if you need to update specs at runtime. I haven't personally run into a scenario where building a static registry at compile time was insufficient.
2018:08:31 15:07:42 lilactown spec-tools is super impressive
2018:08:31 15:17:59 lilactown general question about spec:
one of the things that keeps coming up for me, is that I’d like a language to describe a schema or shape of data that I would need to translate to some other schema language (e.g. GraphQL, dynamoDB, etc.).
As a spec neophyte, my first instinct/desire is to reach for clojure.spec as the schema language, and somehow convert the spec to the other schema language.
But the more I think about it, the more it feels like spec is much better setup to be the tool to do the conversion, and it’s left up to me to create my own schema language in Clojure.
Which one is correct?
2018:08:31 15:23:41 dadair I think there was some project that went from spec to various schema formats, but I can’t seem to recall it right now
2018:08:31 15:24:22 dadair Or maybe it was datomic schema to other schema
2018:08:31 15:25:28 lilactown I was just looking at spec-tools, which can convert specs to JSON Schema and Swagger Schemas
2018:08:31 15:25:53 lilactown which might work for my use case, but then I need to write JSON Schema => GraphQL or DDB or datomic…
2018:09:01 01:26:59 spieden has anyone done anything with strictly validating some data against a spec? by this i mean not allowing anything other than what’s specified. e.g. extra map keys, etc.
2018:09:01 02:34:09 seancorfield @spieden Several libraries have facilities for that and it's fairly easy to do. It's just not necessarily a good thing to do.
2018:09:01 06:32:14 misha @spieden one example of such strict map validation would be s/map-of. if it specifies keywords-to-strings map - you can't have string key in there
But, as Sean pointed out, it is kinda trivial to implement yourself with a particular shade of behavior you are looking for.
2018:09:01 18:35:04 jeaye I've a repro for a bug where ClojureScript's spec instrument fn generates too large a function for the JVM and compilation fails. This happens when too many fns are spec'd. https://github.com/jeaye/instrument-too-many
Looking for some confirmation.
2018:09:01 22:18:59 andy.fingerhut It appears specific to ClojureScript. When I tried a similar thing in Clojure/JVM, there was no problem I noticed.
2018:09:01 23:14:38 jeaye @andy.fingerhut Thanks. I've also tried it on Clojure JVM and I couldn't reproduce.
2018:09:02 18:27:00 bbloom what do people do when you have different keys for different specs, but want to handle them generically? do you just use an extra key to index back in to that map? for example, i’m trying to model a WebAssembly AST, which has “modules” which are made up of “sections” and each section has a vector of “fields”. but while fields have common structure across types of sections, i’d like to have ::funcs and ::types and ::imports instead of just ::fields — my least bad idea is to add (s/def ::fields-key #{::funcs ::types …}) and then indirect through that
2018:09:02 18:28:34 bbloom the two other more bad ideas are 1) just use the abstract name ::fields and rely on the spec’s global name validation. This isn’t ideal because (A) it fails to spec the fact that fields in the funcs section must conform to ::func and (B) precludes unname-spaced keys, which is what i’m actually doing & would have to do a much bigger refactor for that
2018:09:02 18:29:22 bbloom and 2) just copy the fields to the other name like (let [{:keys [fields]} ast, ast (assoc ast ::funcs fields)]
2018:09:02 18:29:26 bbloom this is really nice
2018:09:02 18:29:31 bbloom but it makes rewrites basically impossible
2018:09:02 18:29:40 bbloom b/c if you edit ::funcs, then ::fields is stale
2018:09:02 18:30:16 bbloom is this problem explanation clear? if not, why not? if so, what do you guys do about this sort of problem?
2018:09:02 18:32:48 jeaye > what do people do when you have different keys for different specs, but want to handle them generically?
We'll define it somewhere common, like (s/def ::field blah?) and then alias it to match the expected keys where it's used like (s/def ::my-field ::common/field).
> ... while fields have common structure across types of sections
Depending on how common they are, you may define a (s/def ::base-field (s/keys :req [...])) and then do a (s/def ::field (s/merge ::common/base-field (s/keys :req [...]))) in the namespace tied to that specific field.
2018:09:02 18:33:52 bbloom yeah - so that much i’ve mastered. the tricky part is the collection of these things. for example:
2018:09:02 18:37:56 bbloom the challenge isn’t making the spec - it’s using the data that has been spec’d
2018:09:02 18:38:13 bbloom i want to be able to write (visit-section-fields some-section-of-any-kind)
2018:09:02 18:38:32 jeaye Ah
2018:09:02 18:38:51 bbloom where visit effectively does something like: (update-in section [:fields] #(mapv f %))
2018:09:02 18:39:03 bbloom my “least bad idea” from above is to do:
2018:09:02 18:39:04 jeaye Yeah, that's beyond me. spec-tools allows for better introspection though. Might be of help.
2018:09:02 18:39:10 bbloom (update-in section [(:fields-key section)] #(mapv f %))
2018:09:02 19:12:15 misha @bbloom can you share a snippet of an ast? as json or whatever.
2018:09:02 19:12:47 bbloom there’s a bunch of different designs i’m experimenting with, but it basically is something like:
2018:09:02 19:14:09 bbloom {:type-section {....}, :funcs-section {....}, :exports-section {....}} where each of those are something like {:env {$someid 0} :funcs [....
2018:09:02 19:14:18 misha while you are at it, how about multimethod for visit-section-fields?
2018:09:02 19:14:50 bbloom @malch the logic for visit-section-fields is trivially monomorphic if a section is defined as {:env ..., :fields [...]}
2018:09:02 19:14:51 misha so you want same field name but different spec forms for it?
2018:09:02 19:15:20 bbloom honestly, i don’t want different field names at all - i’m happy to have them all called :fields, but that makes the specs less strict
2018:09:02 19:16:37 misha will you work with qualified keys in data itself? or unqualified?
2018:09:02 19:17:06 misha if unqualified – you just need the same (name kw) for those different specs: :foo/bar :baz/bar
2018:09:02 19:17:06 bbloom i have no need for extensibility, so i was hoping to use unqualified keys to just avoid that entire syntactic headache
2018:09:02 19:20:25 misha there is also (s/keys :req [(or :foo/bar :baz/bar]), but I am not really sure what the behavior contract there
2018:09:02 19:21:08 bbloom > :wasm.type/fields, :wasm.funcs/fields
i don’t understand what you’re trying to say with this
2018:09:02 19:22:34 misha I think I did not understand your initial question and examples opieop
2018:09:02 19:41:12 bbloom ok - new least-bad idea: don’t try to spec my whole AST - just spec pieces of it on an ad-hoc as needed basis ¯\(ツ)/¯ take advantage of a-la-cart, i guess
2018:09:02 19:41:37 bbloom i need to write a validation pass anyway
2018:09:02 19:42:38 bbloom my experience with spec seems to be 1) it forces me to consider my designs & often improves them and 2) then doesn’t really add much actual value for checking at all
2018:09:02 19:45:50 misha how do you intend to use spec for this?
2018:09:02 19:46:20 misha gen-testing? inbound data validation? destructuring?
2018:09:02 20:09:53 bbloom i was hoping to use it to validate the inputs/outputs of each compiler phase by just conforming the root node of the tree - but i need to hand-craft validation logic anyway, so i’ll use spec piece meal in there if it makes sense
2018:09:04 16:40:02 djtango Has anyone come across issues with reader conditionals and spec in cljc?
2018:09:04 16:41:45 djtango Say I want to include a spec from a clj-only library can I do this in cljc:
2018:09:04 16:43:37 djtango currently we are seeing this exception:
Caused by: clojure.lang.ExceptionInfo: Invalid keyword: ::clj-time.s/local-date. {:type :reader-exception, :ex-kind :reader-error, :file "/x/spec.cljc", :line 16, :col 54}
2018:09:04 16:47:52 alexmiller Autoresolved keywords use the current env at read-time to resolve the ns
2018:09:04 16:48:22 alexmiller And this is reading, which happens regardless of the chosen branch
2018:09:04 16:48:47 alexmiller A workaround is to use the full keyword with namespace
2018:09:04 16:49:01 misha harold
2018:09:04 16:56:58 djtango Ah so the reader for your cljs doesn't ignore the clj branches?
2018:09:04 21:35:32 alexmiller It’s read before it’s ignored
2018:09:04 16:57:33 djtango and so it gets to the auto-resolved keywords and if the clj-only ns isn't in the env at that point you get an exception
2018:09:04 16:57:43 djtango Have I understood that right?
2018:09:04 17:27:12 seancorfield Yes, the expressions are read and auto-resolving keywords is part of that, prior to the conditional being applied. Then the result of reading is compiled.
2018:09:04 17:28:54 seancorfield So reader conditionals can only select between syntactically valid expressions that can be read in -- :some-ns/some-thing is read "as-is" but ::some-alias/some-thing has some-alias expanded as part of the reading process.
2018:09:04 17:29:04 seancorfield (I hope I explained that accurately!)
2018:09:05 09:33:48 djtango ^ makes sense. Thanks for the explanations @seancorfield @alexmiller
2018:09:05 12:47:52 ikitommi in the example, the ::x and ::y specs are coerced either from strings (the get-endpoint with query-parameters) or from json (the post-endpoint if client sends json), or just validated (the post-endpoint if client sends edn or transit).
2018:09:05 13:17:06 Chris Is s/fspec the proper way to define a spec for a higher order function? When I declare something with s/fspec and then try to generate a value, it appears to be called many times (perhaps by conform?) is there a way to declare a higher order function without it being invoked with conform?
Example:
`(s/def ::post-download-fn (s/fspec :args (s/cat :size nat-int?) :ret any? :gen
(fn [] (gen/return (fn post-dl-fn [size] (log/info (str "Post Download fn called! n: " size)))))))`
(gen/generate (s/gen ::post-download-fn))
Gives a bunch of these before returning the single value generated:
Post Download fn called! n: 0
Post Download fn called! n: 1
Post Download fn called! n: 1
....
Am I missing something obvious?
2018:09:05 14:07:56 Chris I believe it to be instrumentation validating conformance - but it's odd that we can't turn it off in the case of a side effecting higher order function.
2018:09:05 14:29:39 taylor @chris547 there might be a better way, but there's a *fspec-iterations* dynamic binding you can use to sidestep that behavior:
2018:09:05 14:30:50 Chris Oh great, I wasn't aware of that binding - thanks @taylor!
2018:09:05 14:50:28 taylor np, I'm sure there's better advice/guidance on how to handle this for higher-order functions that take side-effecting functions, and I'm sure it's something that must be considered as clojure.core gets specs. One obvious workaround is to sacrifice specificity by just using e.g. ifn? as a predicate
2018:09:05 19:36:04 basti Hi, is it possible to nest specs, rather than creating for each tiny bit a new s/def ?
So an inline version of e.g.
2018:09:05 19:37:23 taylor for s/keys specs, you need registered specs for the keywords. For other non-`s/keys` types of specs, you can nest spec definitions
2018:09:05 19:38:22 basti k thanks for the info - appreciated! how would that look like for that the spec above?
2018:09:05 19:38:40 taylor (s/def ::my-map (s/keys :req [::key1]))
2018:09:05 19:39:16 taylor or :req-un for unqualified keywords
2018:09:05 19:41:57 basti Oh I rather meant to define the spec for ::key1 in the ::my-map definition. I don’t want to reuse ::key1
2018:09:05 19:42:23 taylor I don't think that's possible
2018:09:05 19:42:50 taylor not with an s/keys spec, at least
2018:09:05 19:36:16 smothers Do you know if there is a way for a recursive spec to nest uniquely? So the same tag can’t repeat across a branch
e.g. NO ❌
2018:09:07 01:55:15 seancorfield And specifically this part of the proposal:
* stop printing data into message strings
ExceptionInfo.toString should list keys, not entire map contents
spec.alpha/macroexpand-check should stop including explain-out in message
2018:09:07 01:59:40 seancorfield @bbrinck Interesting. I can reproduce the behavior in a Boot repl but if I use clj I get different behavior:
2018:09:07 02:01:02 bbrinck @seancorfield Good to know - I am still using Clojure 1.9 in my repro. Maybe it’s an issue with using new clojure.spec without new clojure alpha?
2018:09:07 02:01:06 seancorfield Compared to alpha 6
2018:09:07 02:02:00 bbrinck The announcement for spec.alpha 0.2.176 and core.specs.alpha 0.2.44 says “these libraries can be used now with Clojure 1.9.0”, so I expected that to work
2018:09:07 02:03:49 seancorfield Confirmed, yes, the printing has changed:
2018:09:07 02:06:08 bbrinck @seancorfield Thanks! Since it works in 1.10.0-alpha7, I may just add a note to Expound to suggest that people don’t upgrade to newest spec.alpha unless they also upgrade clojure. Or there may be a workaround
2018:09:07 02:10:01 alexmiller I am too tired to grok the issue - can you recap?
2018:09:07 02:45:47 seancorfield @alexmiller The change in 2373 that removes the explain-out call from several spec failures (and instead adds that call to the default REPL?) means that The details of the spec failure aren't run through a custom printer now in Clojure 1.9 or Boot REPL (and probably not in Leiningen's REPL -- I think both lein and boot use REPL-y?).
2018:09:07 02:46:49 seancorfield (it's the right choice but it means that tools that print explain-data in fancy ways now need to hook into the REPL instead of expecting to be invoking in the spec failures themselves)
2018:09:07 03:12:55 alexmiller It’s not that they aren’t run through a custom printer though, it’s that they’re not run through any printer, right?
2018:09:07 03:13:36 alexmiller Previously they were printed into the exception message and now they’re not
2018:09:07 03:15:00 alexmiller The hooking is still exactly the same, its just that repls need to be better about how they print. The 1.10 clojure.main has a new function exposed to help with this
2018:09:07 05:13:31 seancorfield > It’s not that they aren’t run through a custom printer though, it’s that they’re not run through any printer, right?
Yes, right. I could have worded that better 🙂
2018:09:07 05:15:30 seancorfield > The hooking is still exactly the same, its just that repls need to be better about how they print. The 1.10 clojure.main has a new function exposed to help with this
Boot and Leiningen have some work to do. Or REPL-y. 🙂
2018:09:07 14:08:08 bbrinck @alexmiller I suspect I’ll get questions about this for Expound, so I’d like to put a notice for anyone who upgrades clojure.spec.alpha to “0.2.176”. What would you suggest? Should I recommend not upgrading to 0.2.176 unless they also upgrade to clojure 1.10.0-alpha7? Or until they upgrade to a newer version of Lein or Boot that changes the REPL printing?
2018:09:07 14:10:02 bbrinck > The 1.10 clojure.main has a new function exposed to help with this
which function is this? I’m toying around with a custom REPL built on top of rebel-readline
2018:09:07 14:21:53 alexmiller that’s the function that builds the string to print in the repl
2018:09:07 14:22:47 alexmiller for the note, I think the key is to just be clear about the combination of circumstances you’re warning about
2018:09:07 14:27:58 alexmiller as those circumstances will change with subsequent releases of clojure, spec.alpha, and the tools involved
2018:09:07 16:18:40 bbrinck thanks, i’ll add a warning to the readme
2018:09:07 16:53:26 JH How would you spec this map?
2018:09:07 17:13:55 dadair What do you want the spec to express? Have you read the spec guide? https://clojure.org/guides/spec
2018:09:07 17:21:19 dadair “how would you spec this map” is very subjective; it depends on what you need. For example, a perfectly valid spec is (s/def ::map-thing map?), if that is a robust enough spec for your particular problem.
2018:09:07 19:20:54 JH That makes sense, I just read through the guide and was able to figure it out. I wanted to spec that every key in the map was of certain type. Moreover that the :on-click was a function.
2018:09:07 19:21:02 JH Thanks @dadair
2018:09:08 02:32:03 hoynk Can anyone point me to some GitHub code with a good use of spec?
2018:09:08 02:32:22 hoynk Also, should I use a newer version of spec than the one that comes with 1.9.0?
2018:09:08 02:34:27 seancorfield @boccato If you're just getting started with spec, you might as well stick with 1.9 and the included version.
2018:09:08 02:36:29 hoynk That is exactly my case, @seancorfield.
2018:09:08 02:51:14 seancorfield There are also some changes in the latest spec version that sort of assume some corresponding changes in 1.10 (and I think some of the REPL tools will need some changes to catch up as well).
2018:09:09 01:04:49 basti I there a nested version of s/form ?
2018:09:09 01:17:06 basti to back up a little bit.. suppose I have a deeply nested map data structure, how can I inspect its structure without manually traversing all its composed (s/def)s ? I could generate an example, but that doesn’t tell me that a specific leaf is specd with a string? predicate e.g.
2018:09:09 01:18:32 basti plus the example generation is also kinda limited because its predicated on the fact that every predicate I use has a generator
2018:09:09 05:53:07 basti Other question, how do I spec a collection which has the sequence: [string? int? keyword?] .. Whereby int? keyword? can repeat arbitrary. So something like [sting? int? keyword? int? keyword? int? keyword?] would be valid?
2018:09:09 05:56:36 seancorfield @basti untested but I think (s/cat :s string? :rest (s/+ (s/cat int? keyword?)))
2018:09:09 06:00:07 seancorfield Yeah, (s/def ::x (s/cat :s string? :rest (s/+ (s/cat :i int? :k keyword?)))) should do it @basti
2018:09:09 07:10:44 misha well, until skynet comes around, it's up to you to assess which spec is trivial to generate automatically, and which ones need your help, @basti
2018:09:09 07:14:05 misha what exactly do you mean by inspect? there is an s/conform, which destructures and assigns labels to most of the nodes. in which usecase it is not enough for you?
So that I see the whole structure at one glance.
Meanwhile I wrote a small utility function that converts map specs to my desired structure - will write a blog post about that eventually 🙂
2018:09:09 07:56:30 ikitommi @basti at least spec-tools has a spec form parser & visitor, which can walk over all(?) specs. But I guess you already rolled your own.
2018:09:10 16:57:33 snowell When running stest/check on this function, I get an ArityException because the (s/cat a (s/cat b c d)) is becoming 4 arguments
2018:09:10 16:57:51 snowell How can I tell it that the second cat needs to be as one arg?
2018:09:10 17:00:45 favila surround with s/spec
2018:09:10 17:05:49 snowell OK cool...that at least gets me a different failure
2018:09:10 18:50:17 borkdude I listened to the REPL podcast today and the interviewee (Higginbotham) said there were benefits with putting specs in their own namespace. How exactly?
For fdefs I find it useful to have them close to the function as documentation?
2018:09:10 18:55:45 seancorfield @borkdude It means your code can still be used without spec since the specs -- and those namespaces -- are optional.
2018:09:10 18:56:00 seancorfield clojure.java.jdbc does that so it can still support back to Clojure 1.7.
2018:09:10 18:56:09 borkdude for libraries that’s nice yes
2018:09:10 18:56:47 seancorfield Also, most of our specs at work are data specs, not function specs -- and we find it easier to have a namespace for data specs. One place to go read them all (for a given data concern).
2018:09:10 18:59:13 borkdude ok
2018:09:10 18:59:14 misha +1 to data specs mostly
2018:09:10 18:59:53 borkdude do you use instrumentations with these specs at all?
2018:09:10 19:00:00 borkdude e.g. in jdbc
2018:09:10 19:02:38 seancorfield I instrument the java.jdbc lib for the tests within that lib -- which slows things down dramatically.
2018:09:10 19:03:06 seancorfield At work, we tend to instrument specific functions (that have fdef specs) only within their specific tests.
2018:09:10 19:03:20 seancorfield We use the data specs in production code (to validate/conform data).
2018:09:10 19:03:37 borkdude I find it annoying to get the wrong keys. Whenever I encounter that I tend to put an fdef now and instrument it (only for dev)
2018:09:10 19:03:51 seancorfield We also use the data specs in tests -- for generative testing, or just for random example-based test data.
2018:09:10 23:04:18 Jon Walch Is there anyway to force a specific path with a call to s/gen?
Here’s a trivial example:
2018:09:10 23:06:34 taylor you might be able to override ::value there by providing a different generator
2018:09:10 23:10:00 Jon Walch perfect thanks
2018:09:11 14:41:52 slipset Sorry if this is a faq, but I’d like to generate specs at runtime to use for validation eg:
2018:09:11 14:42:03 slipset Say I receive something like:
2018:09:11 14:42:24 slipset
{:id 1, :model-id 4711, :foo 'bar}
2018:09:11 14:42:56 slipset In order to know how to validate this thing, I’d need to look up what the model is in a database, and then construct a spec based on that model.
2018:09:11 14:44:40 ghadi eval
2018:09:11 14:45:01 lilactown unfortunately, clojure.spec is very unwieldy for this type of use case
2018:09:11 14:45:24 slipset What I’d be looking for is something like:
2018:09:11 14:46:42 ghadi that'll give you a spec, if the form is saved in the db
2018:09:11 14:46:53 slipset And with my understnading of spec, this is not possible, but I’d like to flag it as a use case, just in case.
2018:09:11 14:46:56 ghadi and all of what it refers to is already loaded and available
2018:09:11 14:47:16 ghadi it's totally possible, you just have to dynamically load the spec
2018:09:11 14:47:56 lilactown I know that there seems to be some disinclination to adding more support this. it’s my most-desired feature
2018:09:11 14:48:27 ghadi it's not a feature you add to spec, you can already do it
2018:09:11 14:48:48 slipset @ghadi but my specs are not “known” or stored anywhere, and there might be a bunch of models (a bunch being in the thousands).
2018:09:11 14:49:27 slipset While I’m typing this, I guess I think that if specs were data, this would be feasible, right?
2018:09:11 14:49:40 lilactown the ability to have “anonymous” specs / specs-as-data would have a lot more utility as a general purpose way of describing data. but I think that it rubs against the desire of the core team to encourage people to use namespaced keys
2018:09:11 14:50:09 ghadi regardless of the mechanism, your specs have to be known somehow if you want to dynamically load them
2018:09:11 14:51:24 slipset @ghadi: in my db I would typically have someting like this (for a model)
2018:09:11 16:20:10 ikitommi I’m actually looking forward to seeing things break.
2018:09:11 16:20:58 ikitommi any plans on doing that @alexmiller?
2018:09:11 16:21:12 borkdude even the public parts of spec itself may break if it’s for the better, it’s alpha 😉
2018:09:11 16:33:29 alexmiller Rich has been getting back to spec and right now I think it’s likely that we’ll try to get 1.10 out the door, then start working on spec in earnest
2018:09:11 16:33:56 borkdude cool
2018:09:11 16:38:55 borkdude what are the main features of 1.10? - I saw you were working on better error messages, very nice work
2018:09:11 16:43:38 alexmiller updated JVM/JDK requirements and compatibility, prepl (kind of alpha), error messages
2018:09:11 16:45:35 seancorfield Does that mean we'll get some new tooling from Cognitect based around the prepl?
2018:09:11 16:46:51 alexmiller unlikely for 1.10
2018:09:11 16:47:07 borkdude prepl.alpha… what are we going to do with the prepl?
2018:09:11 16:47:18 alexmiller whatever you like :0
2018:09:11 16:47:46 alexmiller there’s a bunch of things in work. rather than wait for them all to reach completion, I think we’ll just release 1.10 and keep working on them
2018:09:11 16:48:02 borkdude anything to read on prepl?
2018:09:11 16:48:12 seancorfield The source code? 🙂
2018:09:11 16:48:39 seancorfield I had a quick play with it when it dropped... I can imagine some interesting tooling options based on it...
2018:09:11 16:48:58 alexmiller there’s a small chance it might even be removed before 1.10 releases
2018:09:11 16:48:59 borkdude I have this vague notion it’s something similar to unrepl, but that’s all I know
2018:09:11 16:49:10 alexmiller it shares many goals with unrepl
2018:09:11 16:49:27 alexmiller both are data-oriented stream repls
2018:09:11 16:50:18 seancorfield We're thinking a bit about building some internal tooling on top of prepl. Automating some of the stuff we currently do today (via a REPL manually).
2018:09:11 16:51:22 seancorfield (we run a Socket Server REPL inside one of our production processes and currently use unrepl to talk to it over an ssh tunnel for certain things)
2018:09:11 16:51:52 borkdude we run nREPL in production, but I only talk to it myself 😉
2018:09:11 16:52:46 seancorfield We used to -- but I didn't like the stack of dependencies that nREPL brought in. Running just a Socket REPL is nicer. And of course we can start any process up with a Socket REPL without needing any code inside the process.
2018:09:11 16:53:23 borkdude I did that once for a Scala project where I added clojure as a dependency 😉
2018:09:11 16:54:44 borkdude it wasn’t a serious project though, but it worked 😉
2018:09:11 16:56:23 seancorfield We have a REPL running inside our legacy CFML app so I won't judge 🙂
2018:09:11 17:08:23 slipset @ikitommi nice! I’ll have a look at the data specs.
2018:09:11 19:01:03 arohner s/keys supports and in :req, but not :opt, right? Is there any other way to express “:foo is optional, but if present, :bar must also be present”?
2018:09:11 19:04:22 misha s/multispec, s/or
2018:09:11 19:06:46 misha btw, the exact behavior of or and and inside s/keys is still a mystery to me. is there something official to read about it, @alexmiller? or is it all unintended side effect, which should not be relied upon?
2018:09:11 19:15:13 seancorfield @arohner That's usually when I reach for s/and wrapped around s/keys and a predicate for the present/not-present relationships...
2018:09:11 19:24:56 arohner @seancorfield thanks. The combinations here get a little gnarly to multi-spec, so I guess I’ll punt
2018:09:11 19:25:06 arohner @misha it’s documented in s/keys docstring:
The :req key vector supports 'and' and 'or' for key groups:
2018:09:12 15:08:34 samedhi Is it possible to write a spec that generates a sequence where each element in the sequence has some relationship to the previous element(s) in the sequence?
2018:09:12 15:09:08 samedhi As an example, like one that multiplies the last element of a sequence by 2 or 3 (random).
2018:09:12 15:09:51 samedhi so [1 2 4 8 24 48 144] and [1 3 9 18] would both be sequences it might generate.
2018:09:12 15:10:05 taylor are you only interested in generation, or do you want a spec to validate inputs too?
2018:09:12 15:10:17 samedhi I guess both
2018:09:12 15:10:57 samedhi I mean I would like to use test.check with this generated sequence, so I assume that means using both (I think).
2018:09:12 15:13:25 alexmiller the way I would handle this is to use gen/bind
2018:09:12 15:13:48 alexmiller generate a series of random multipliers (a sequence of random 2s and 3s)
2018:09:12 15:14:24 samedhi Ah, I see where you are going.
2018:09:12 15:14:30 alexmiller then gen/bind to turn the sequences of multipliers into a sequence of values
2018:09:12 15:14:40 alexmiller or rather, I guess gen/fmap is sufficient here
2018:09:12 15:14:52 alexmiller it’s just apply a function to that sequence
2018:09:12 15:16:00 samedhi Yeah, so what I was really hoping was to generate the initial (seed) state of my program as the first element in the sequence, and generate the rest of the sequence by constantly looking at the last element in the sequence, determining what actions can be taken at that state, and then picking one of those actions and applying it to said state.
2018:09:12 15:16:46 alexmiller yeah, same idea
2018:09:12 15:17:09 alexmiller start from a gen of a tuple feeding the fmap
2018:09:12 15:18:40 alexmiller then gen/fmap a function over that that takes the first element in the tuple and repeatedly applies the multipliers in the second element to produce a sequence
2018:09:12 15:18:56 samedhi Ok, the only wrinkle I see there is that only a subset of all of the actions are valid for some of the states.
2018:09:12 15:19:02 samedhi But that is an idea.
2018:09:12 15:19:25 samedhi Maybe, I can just make it so that if the action taken is NOT a valid action for that state, it just returns the original state (identity).
2018:09:12 15:19:27 alexmiller if you need to randomly choose an action per state, you will need to move to gen/bind
2018:09:12 15:20:01 alexmiller but I think you could make that work too
2018:09:12 15:20:54 samedhi Interesting, interesting. I mean I tried to read through the "rose tree" thing for test.check generators. And... well, it looks like it is generating some sort of tree using something like a recursive algorithm. But, yeah, got lost there. 🙂
2018:09:12 15:20:56 alexmiller in that case, I’d probably gen the initial state and the number of actions to gen
2018:09:12 15:21:18 alexmiller yes, but you don’t need to worry about that - the key is just to capture all randomness via generators
2018:09:12 15:21:25 alexmiller test.check will do the rest
2018:09:12 15:21:48 samedhi Ok, thank you for your help, I'll look at it some more this evening.
2018:09:12 15:22:14 alexmiller this is advanced generating, but it’s where things get interesting
2018:09:12 15:22:33 alexmiller a lot of interesting problems in simulation testing look like this
2018:09:12 15:22:59 samedhi Really, it is just the fantasy I have of letting spec generate every possible way that my program could run and being like "yep, that one thing encompasses all the test".
2018:09:12 15:23:33 samedhi You still get a great deal of benefit out of using the normal fdef and test.check per function imo, but the idea of actually generating all states of the program would be really cool.
2018:09:12 15:28:33 gfredericks @samedhi @alexmiller this is exactly what the stateful-check library tries to automate, I believe. I haven't used it
2018:09:12 15:34:32 taylor would there be a way to spec this kind of sequence without a custom predicate?
2018:09:12 15:34:45 roklenarcic Is there some way to specialize a key in s/keys? Like I have a (s/keys :req [.... ::type]) where type is #{"ORG", "USER"}, but now I'd like that same spec but where type is limited to `#{"ORG"}
2018:09:12 15:35:08 alexmiller @taylor do you mean custom generator?
2018:09:12 15:36:00 taylor no I mean if you wanted to conform/validate a sequence like that, asserting that each element is a multiple of the previous
2018:09:12 15:36:25 taylor I was trying to imagine how you might use the spec primitives to do that but I can't imagine how
2018:09:12 15:37:05 alexmiller not as a pred in the coll spec, but you could s/and with a function that verified that
2018:09:12 15:37:37 alexmiller or, maybe you could put that in :kind if you had a seq-of-mults? pred
2018:09:12 15:35:37 alexmiller @roklenarcic no, but you can s/and it with a more restrictive predicate
2018:09:12 15:36:42 roklenarcic It tried to s/merge it with another keys spec which has key that is more restrictive, with different keyword ns and same name
2018:09:12 15:37:14 roklenarcic it kinda works, in that it will check both
2018:09:12 15:38:12 alexmiller there are pros/cons to those approaches wrt conform and gen
2018:09:12 15:38:30 roklenarcic ah, haven't thought about that
2018:09:12 15:39:03 alexmiller I think s/and is the most direct way to say it: “I have this map AND it must meet this extra constraint”
2018:09:12 15:39:23 alexmiller s/multi-spec can also be handy for some cases like this but it’s a lot of weight to add if it’s only this
2018:09:12 15:45:43 roklenarcic I'll need to check out multi-spec
2018:09:12 15:53:28 roklenarcic Using spec-tools's data-spec solves a couple problems, but sadly data-spec has a bug where they don't name their specs
2018:09:12 19:48:02 misha @arohner well, that's embarrassing opieop (re: or in s/keys)
2018:09:12 19:58:07 taylor what's opieop all about
2018:09:12 20:01:59 misha face expression
2018:09:12 20:02:30 taylor it's so small I can't make out his expression
2018:09:12 20:03:14 taylor I only see it used on this Slack and I feel like I'm missing... something
2018:09:12 20:04:26 misha it is from twitch .tv chats originally
2018:09:12 22:23:29 lilactown clojure.spec has the ability to transform data from one form into another, right?
2018:09:12 22:24:05 lilactown are there any ground-up guides around data transformation?
2018:09:12 22:25:52 seancorfield The basic rule of thumb is "don't do that"
2018:09:12 22:26:45 seancorfield Spec isn't designed as a data transformation system and when folks ask about doing it, @alexmiller usually pops up and says that's not a good idea 🙂
2018:09:12 22:28:59 lilactown bummer
2018:09:12 22:29:01 seancorfield That said, I think there are some use cases where limited coercion is acceptable. For example, we use spec for API / form input validation so data is mostly strings but we want strings, longs, doubles, Booleans, and sometimes dates out of it. I think that sort of spec is "OK" if you're careful (in particular, think about generation of data using such specs, as well as applicability of the spec elsewhere in your system -- just because you accept strings at the API/form level doesn't mean you want to accept strings elsewhere in your domain model).
2018:09:12 22:29:26 seancorfield What sort of data transformation were you thinking about @lilactown?
2018:09:12 22:31:03 lilactown I'm creating an API for creating content schemas in a CMS
2018:09:12 22:32:11 lilactown i have a general purpose edn format I'd like to use, and then coerce it into the JSON representation for the particular CMS we're using
2018:09:12 22:32:31 alexmiller to be specific, transforming data use conformers in spec is imo not a good idea. using spec as a target for driving transformations is potentially a good idea. I think https://github.com/wilkerlucio/spec-coerce is pretty good.
2018:09:12 22:50:50 lilactown thanks! I'll take a look
2018:09:13 02:46:55 donaldball I’m working on a spec for an asynchronous message format and am curious about versioning. My instinct is to declare a versioned namespace for the message envelope - e.g. :my.message.v1/schema, declare all my envelope fields in that same namespace, and then just make a new one if and when I realize the need to make an incompatible change to the envelope. It occurred to my colleague that multi-spec was another way to accomplish this by including the version as a value, e.g. :my.message.schema/version 1 and using multi-spec to choose the correct spec.
Are both of these viable options and is there anything to recommend one over the other?
2018:09:13 03:06:31 alexmiller I’d recommend not making incompatible changes
2018:09:13 03:06:43 alexmiller then you don’t need versions :)
2018:09:13 03:07:24 alexmiller if the attribute needs to change, give the attribute a new name
2018:09:13 03:08:27 alexmiller if you’re talking about adding attributes, that’s compatible. if you need to remove attributes from a map, then you should have a new name for the container, which I guess version is one kind of new name
2018:09:13 14:29:51 donaldball Alas, my foresight is sufficiently imperfect that I don’t believe I can develop the correct message format for all time. Incompatible changes I believe might be reasonably desired include (and are probably limited to?): adding and removed required keys.
2018:09:13 14:33:17 alexmiller well, I’ll refer you to Rich’s Spec-ulation keynote for the long form argument
2018:09:13 14:35:06 ghadi what is an example of adding keys that is a breaking change @donaldball?
2018:09:13 14:35:41 taylor adding new required keys would be breaking
2018:09:13 14:36:21 ghadi important word simple_smile
2018:09:13 14:37:29 taylor but yeah I think the gist is "don't make breaking changes" which I generally take to mean "if you need to break something, make a new something, and leave the old something alone"
2018:09:13 14:38:59 taylor if you need to add a new required key to some map, it's no longer the same thing. Make a new map spec with a different name that requires the new key
2018:09:13 14:40:27 alexmiller ^^
2018:09:13 14:41:49 ghadi if it's for a message on something like Kafka, the new name might be an entirely new Topic
2018:09:13 14:44:45 donaldball I agree that’s very much the route spec wants you to go down and I’m on board the train, but I am curious what makes the multi-spec idea a poor solution. It seems like it does constrain you forever to having a message be a map, and for the map to require the multi-spec keyword, but otherwise the map contents could vary as required by the versions.
2018:09:13 15:17:18 lilactown how do I put a bound on how many times clojure.spec.test.alpha/check generates a test?
2018:09:13 15:17:40 lilactown the documentation is very nebulous
2018:09:13 15:24:11 taylor >The opts map includes the following optional keys, where stc aliases clojure.spec.test.check
>The ::stc/opts include :num-tests in addition to the keys documented by test.check.
2018:09:13 15:24:58 taylor it confused me at first too b/c test.check/quick-check takes the number of tests as its first argument
2018:09:13 15:33:25 lilactown yeah it was completely meaningless to me
2018:09:13 15:34:44 lilactown hm. even specifying {:num-tests 5}, my generative testing is taking minutes
2018:09:13 15:42:50 lilactown in fact, specifying 0 takes minutes? I'm not even sure this is working
2018:09:13 15:49:16 taylor and the :num-tests is accurately reflected in the check result
2018:09:13 15:51:54 lilactown I think some of my specs must be really expensive. is there a way for me to selectively narrow some of the child specs?
2018:09:13 15:51:54 lilactown I think some of my specs must be really expensive. is there a way for me to selectively narrow some of the child specs?
2018:09:13 15:54:11 taylor any recursive specs involved?
2018:09:13 15:54:46 lilactown nope
2018:09:13 15:55:28 taylor hmm hard to say w/o seeing all the relevant code
2018:09:13 15:58:55 alexmiller Expensive to check or to gen?
2018:09:13 15:59:14 lilactown I'm not sure 😞
2018:09:13 16:00:44 lilactown I apologize, it's quite long. but I'm not confident what might be what's causing the slowness
2018:09:13 16:11:37 lilactown definitely seems to be in the generation of :torch/fields
2018:09:13 16:13:42 lilactown I'm going through and removing everything and slowly adding back in. adding back in
:torch.field/key
:torch.field/name
seems to slow it down, but not astronomically
2018:09:13 16:38:59 alexmiller I meant what operation are you invoking when it’s slow?
2018:09:13 16:39:40 alexmiller Sounds like gen
2018:09:13 16:40:18 alexmiller If you have coll-of, it’s good to set :gen-max to something small like 3
2018:09:13 16:51:57 lilactown that did help! thanks!
2018:09:13 17:06:08 alexmiller The default is 20 and that can get a little out of hand
2018:09:13 16:27:30 lilactown how do I selectively narrow the values generated by stest/check?
2018:09:13 16:28:28 lilactown e.g. if my spec has a named child-spec somewhere that is defined as keyword?, but I don't really care about testing 1000s of keywords
2018:09:13 16:29:23 taylor I think you can pass in an override map in check opts :gen
2018:09:13 16:29:54 taylor >:gen map from spec names to generator overrides
2018:09:13 16:30:18 lilactown thanks!
2018:09:14 15:28:13 lilactown how would I represent a constraint on two related fields? e.g.:
{:type :type/json
:cardinality :cardinality/many} ;; invalid! :cardinality/many can only be used with :type/string and :type/ref
2018:09:14 15:30:03 dominicm @lilactown use a predicate on the map?
2018:09:14 15:35:06 lilactown right.... keep thinking of specs at the key level. thanks
2018:09:14 17:01:59 Spencer Apple Hello, is it possible to include external data when using spec to validate data - or would a separate function be better? E.G. validating an api response, and using a key from the request (such as the length of the pagination to validate the number of values returned).
2018:09:14 17:08:33 dadair Could you define a spec that is the combination of the request + the response (i.e., the context), and then use a predicate to check the mentioned constraints?
2018:09:14 17:10:06 dadair something like the following?
2018:09:14 17:13:13 alexmiller I think the question is whether that’s better than just writing the equivalent Clojure code
2018:09:14 17:13:26 alexmiller (I tend to think not in this case)
2018:09:14 17:19:28 Spencer Apple hmm @dadair that is a really good example, and I think it would work. Thanks for writing that out. I was curious about if it made sense to include in the spec - and I kind of think the length of a vector is part of the data shape… but perhaps it still doesn’t fit in the spec
2018:09:14 17:20:45 Spencer Apple I will eventually get to the data generation side of this spec, in which I will want to include the :pagination :length parameter from the request to generate the correct length of ::response ::values. At that point I might need to combine the ::request and ::response specs?
2018:09:14 18:02:35 alexmiller yes, if you want to generate one thing, they need to be in the same spec, although I’m not sure why you need to generate responses (presumably your code is making those)
2018:09:14 18:12:50 Spencer Apple Gotcha, and thanks for you responses!
The request and response are to and from an external API. I want to generate responses to test our data munging flows. Since the data munging fns rely on the request parameters, the generated responses also have to conform (at least partially) to the requests that “generated them”.
2018:09:14 19:46:03 alexmiller gotcha, that makes sense
2018:09:14 19:46:40 alexmiller depending how much stuff there is, you might able to gen both independently, then just copy values that have to match from the request into the response or something like that
2018:09:14 20:02:21 Spencer Apple ah interesting idea!
2018:09:15 18:36:29 richiardiandrea we have found that :: is a bit clunky for refactoring, is other people using the long :ns.foo.bar/quuz for specs?
2018:09:15 18:39:02 mpenet Not really. You change the name just in the ns declaration vs everywhere you use it. I guess it s quite personal
2018:09:15 18:39:53 richiardiandrea well you have to change the namespace and the alias in case it changes
2018:09:15 18:40:18 mpenet Yes that's what I meant
2018:09:15 18:40:19 richiardiandrea while with unqualified you don't have to do anything else, just copy them around
2018:09:15 18:40:47 richiardiandrea I mean fully qualified 😄
2018:09:15 18:41:50 richiardiandrea following along these lines, sometimes you need to make them unique...you cannot use :foo/bar or they would clash
2018:09:15 18:42:07 mpenet The alias doesn't change really, just the thing it points to.
2018:09:15 18:43:03 richiardiandrea this though means you need to give names
2018:09:15 18:43:16 richiardiandrea and naming is hard and requirements change and all that
2018:09:15 18:44:02 richiardiandrea so we found fully qualified better for us - for now... 😄
2018:09:15 18:44:02 mpenet Same argument that's sometimes mentioned with clj.spec.alpha, just use aliases ex s/def and the day it changes to beta or whatever you just update the alias target
2018:09:15 18:45:03 mpenet In the end it s just a search & replace with less/more hits, I guess both ways are fine
2018:09:15 18:45:41 richiardiandrea well my argument would be that you don't even need search and replace
2018:09:15 18:45:51 richiardiandrea at the price of spec verbosity
2018:09:15 18:46:46 mpenet You d need to update every kw vs just ns declarations so yeah, there is some s&r :p
2018:09:15 18:48:11 richiardiandrea my question was something else though.. 😄
2018:09:15 18:49:23 richiardiandrea kind of a feature request, but at this point I don't know if it makes sense 😄
2018:09:15 18:49:49 richiardiandrea it would be great if you could have aliases of keywords that are not tight to a namespace
2018:09:15 18:50:20 alexmiller we do expect to have something like this, but not sure yet exactly what it will look like
2018:09:15 18:50:36 alexmiller didn’t want to couple it tightly to existing namespace/alias support
2018:09:15 18:50:37 richiardiandrea like (defkw :my-long.foo.bar :mfb)
2018:09:15 18:50:42 richiardiandrea oh cool ok
2018:09:15 18:51:33 richiardiandrea is there a JIRA to upvote? I can open one
2018:09:15 18:52:07 richiardiandrea I'll check 😄
2018:09:15 18:52:53 mpenet Would be interesting if spec understood hierarchies, but I already mentioned that
2018:09:15 18:53:45 alexmiller there is one I think
2018:09:15 18:56:16 richiardiandrea will track it and write there if I have good ideas 😄
2018:09:15 18:56:21 richiardiandrea thank you Alex
2018:09:15 20:25:46 lilactown yeah I ended up writing the fully qualified keyword for my specs lately and really am glad I did
2018:09:15 20:25:58 lilactown it made it really easy to e.g. move all of my specs to their own ns
2018:09:15 23:27:48 dadair Does the thing change if it moves namespaces? Is it fundamentally different? If not, I think aliasing conflates location with name with regards to specs. I tend to prefer something like :transport/event because what it is doesn’t change based on where it is.
2018:09:16 06:51:25 ikitommi From reddit:
> There are a couple areas (spec programmability and some map-related stuff) where we have some work to do.
2018:09:16 06:56:19 ikitommi sounds good, despite doesn't say much about what kind of changes. As there are lot of libraries trying to extend spec for different use-cases, it would be great if the need of the libs could be taken into account. Personally would like to see some public spec walking protocols.
2018:09:16 07:00:53 ikitommi and not based on a parsing spec forms, that we can do outside, more like CLJ-2251.
2018:09:16 07:02:02 ikitommi a library developer feature poll/discussion?
2018:09:16 07:03:45 misha @richiardiandrea you can create ns and give it an alias, it will not be tied to an actual file, but you will be able to use that alias to shorten keywords. it is not pretty though. clojure.core/create-ns, clojure.core/alias. I found, that having shorter spec names is less tedious and more readable (possible in app layer, where your system is pretty much the only system which sees them). In libraries you probably would want to use longer "truly" global long keywords for under the hood things.
2018:09:16 15:50:20 richiardiandrea This is an interesting idea, but feels like a hack :) I will give it a try anyways, thank you!
2018:09:16 16:01:23 misha this is sort of what ns does, but it looks alien and hacky, I agree with that.
2018:09:18 18:30:12 richiardiandrea unfortunately alias does not seem to work in cljs
2018:09:18 18:36:57 misha i'd cross-post this to #clojurescript, seems critical(?)
2018:09:18 18:37:21 richiardiandrea done that
2018:09:16 08:49:59 ho0man Hi, I am using spec for the first time. And it takes too long to generate my specs using (test.check/generate (spec/gen ,,, )).
I had previously used test.check for similar data generation and did not had the problem.
The spec if the configuration map having many nested spec/keys and spec/every.
Limitin spec/every with :gen-max did not help. What am I doing wrong ?
2018:09:16 08:50:41 ho0man Heres part of the spec ... the final configuration contains a numbeer of :ecosystem/service-instances
2018:09:16 12:38:02 alexmiller If you have a lot of s/every, they will gen up to 20 elements. Setting :gen-max in those to something like 3 would probably help
2018:09:17 07:38:55 ho0man Thanks, Alex.
I did try that and it helped.
My question is whether there is a fundamental difference in spec's data generation and that of test.check's ?
And should I be concerned about using these specs in large generative tests as their size grows ?
Thanks again
2018:09:17 12:26:07 alexmiller spec uses test.check generators so there is literally no difference. There will always be the need to tune the gens a little bit
2018:09:16 18:00:48 dadair What’s the preferred way of specing a spec? I have an internal lib function that takes a spec and I want the s/fdef to check that the arg is a spec. So far I have qualified-keyword? But that’s a bit of a shallow spec? Tried spec? But it was returning false on s/def’d qualified keys.
2018:09:16 18:02:30 dadair That spec arg is later used in internal machinery to validate some request routing
2018:09:16 18:08:30 seancorfield @dadair Remember that a predicate function could also be (treated as) a spec...
2018:09:16 18:08:56 dadair Yeah, at this point I force using a “defined” spec
2018:09:16 18:09:23 seancorfield Maybe call s/form on it and succeed if you get truthy?
2018:09:16 18:12:14 dadair Maybe expect users to wrap in (s/spec ..) prior to passing argument, then internally spec against s/spec??
2018:09:16 18:13:11 dadair that would also get around the limitation of removing the ability to use a raw predicate
2018:09:16 18:14:19 dadair Are there any downsides to that approach? I’m not super familiar with the effects of (s/spec ..)
2018:09:16 18:18:24 seancorfield There's a get-spec function you can use with the qualified keyword (rather than s/form). You can wrap anything in s/spec: (s/spec 42)
2018:09:16 18:19:13 seancorfield So (s/spec? (s/spec 42)) is truthy -- so it won't tell you it's really a spec that is wrapped.
2018:09:16 18:19:59 dadair Yeah maybe I need to do some code-based validation rather than relying on the fdef here
2018:09:16 18:20:52 dadair It’s not a critical check, I’m just trying to add some guardrails for the other devs that aren’t super familiar with this part of the system
2018:09:16 18:20:54 seancorfield I guess I'd take a step back and ask: Am I over-constraining this function?
2018:09:16 18:21:28 seancorfield Can it accept a predicate and use it like a spec, for example?
2018:09:16 18:22:02 dadair Yeah, that’s what I’m thinking too.
2018:09:16 18:49:11 alexmiller specs can be qualified-keyword?, set?, ifn?, or spec instances (there’s a s/spec? for those)
2018:09:16 20:28:29 samedhi Should I be using (spec/fdef ...) for multimethods (does not seem to work)? Guess I can just wrap the defmulti invocation in a function and test that...
2018:09:16 20:30:19 samedhi Erroneously thought that multi-spec was for multimethods, but it actually appears to be for dispatching data based on "kinds".
2018:09:16 20:33:08 samedhi nm, silly ns error. spec/fdef does appear to work with defmulti
2018:09:17 19:49:19 spieden is anyone aware of a way to strictly validate some data against a spec? e.g. fail on extraneous map keys, etc.
2018:09:17 21:51:16 bbrinck but yes, the general advice is not to make specs closed
2018:09:17 22:27:40 spieden thanks!
2018:09:17 22:27:47 spieden yes generally i leave them open
2018:09:17 22:28:40 spieden in this case it’s closed, though, and the user can make all kinds of interesting typlos
2018:09:17 22:29:25 bbrinck Yep, I think spell-spec is a good fit then
2018:09:18 06:41:29 misha @U0HM5MJ5V you also could do (s/and (s/keys ...) (s/map-of #{<allowed-keys-only>} any?)) to see whether this behavior is actually what you need. saves you custom macro or dependency.
2018:09:18 17:00:05 misha well, #{} is strict, but yeah
2018:09:20 19:22:59 spieden ah of course
2018:09:18 06:22:44 esp1 Is there a way to have predicates return more detailed error messages? I have some complex predicates that operate on subtrees to do things like make sure all branches of the subtree are of the same type. Instead of just having the predicate return false at the root, I'd like to be able to have the predicate provide detailed path information about where within the subtree it detected an error. It would be great if I could have the predicate behave like the built-in spec functions so it could push this detailed information into the explain-data format. Is this possible to do in any supported way without reverse engineering spec internals?
2018:09:18 06:45:58 misha decomposing your predicates in simpler ones, and recomposing them with spec's s/and, s/or and s/*?+ comes to mind. former will bring you closer to "can understand the exact issue from a predicate form w/o error messages", latter will give you exact paths in s/explain-data. @esp1
2018:09:18 07:46:08 esp1 Thanks for the reply @misha! I don't think decomposing the predicate will really work for me though; let me give some more detail about why. The structure I'm spec'ing is essentially a lisp-like expression syntax. For instance I want to validate that in the expression [ := :Foo/bar {:a 1, :b {:c "blah"} } ] that the arguments to the := comparison are both of the same type. The predicate to do this looks up the type schema I have associated with the :Foo/bar argument (which looks something like {:a :int, :b {:c :int} } and checks if the map argument matches this structure. In this example the type schema says that the value of :c should be an integer, but the value is actually a string, so validation should return an error. I'd like the error message to report the expected and actual types, along with the fact that the type mismatch error occurs in the second argument value at the path [:a :c]. If I just have a boolean predicate I'd just get a spec error indicating that a type mismatch error occurred at the root level. The ability to associate error messages with expound is nice - I hadn't realized you could do that. I suppose I could use a custom printer in expound that essentially walks the argument subtree again once a spec error occurs and prints out the error detail. This feels kinda hacky tho. I was hoping I could make the predicate such that it could provide detailed error info directly into the structure returned by s/explain-data.
2018:09:18 12:47:04 Timo Freiberg can i use a s/or spec but have conform only return the value instead of [:key val]?
2018:09:18 13:10:57 Timo Freiberg not sure how ugly this is, but i "solved" it by wrapping the s/or in an s/and and putting a (s/conformer second) at the end
2018:09:18 13:13:30 taylor @UA11M92KE s/nonconforming is what you want, I think
2018:09:18 13:20:36 Timo Freiberg ah, awesome! that makes it easier
2018:09:18 13:29:41 misha @esp1 well, sounds like you already have destructured spec for :Foo/bar, which should give you detailed paths to errors. You just need to figure out how to validate against it w/o enclosing it in an opaque predicate
2018:09:18 13:49:58 misha
2018:09:18 13:54:39 misha number of layers and complexity of this would depend on the diversity of the expressions you want to support.
2018:09:18 13:56:11 misha way simpler version might be just
Will this bite me in the future? (using a defn to define specs) Will it play well with generators once I get to those?
2018:09:18 19:00:39 alexmiller You don’t need that - look into :into in coll-of
2018:09:18 19:03:24 alexmiller (s/coll-of int? :into (sorted-set)) does what you want
2018:09:18 19:03:43 alexmiller With whatever for int?
2018:09:18 19:06:50 jrychter Oh, thank you, I didn't know about :into! Seems like it's exactly what I need. Trying it out.
2018:09:18 19:07:39 alexmiller Will work with gen automatically too
2018:09:19 01:25:41 richiardiandrea what is a good intermediate value for quick-check's :max-size?
2018:09:19 01:26:02 richiardiandrea The default kind of takes forever for me for a couple of generative tests
2018:09:19 01:26:19 richiardiandrea I am afraid 5, which is fast, is too small
2018:09:19 01:26:40 taylor FWIW I think the default is 200
2018:09:19 01:28:44 richiardiandrea yep it is 200
2018:09:19 01:41:18 richiardiandrea wow, I had to reduce :max-size to 10
2018:09:19 01:44:45 richiardiandrea don't even know what or why I am doing this frankly, any feedback is very welcome 😄
2018:09:19 01:45:42 taylor a lot depends on what you’re generating and what you’re doing with the generated data. if it’s any thing non-trivial, it’s not unusual for generative tests to take a little while
2018:09:19 01:45:43 alexmiller the slow part is usually generating (too large) input values
2018:09:19 01:46:16 richiardiandrea I also added a s/coll-of:gen-max to 5
2018:09:19 01:46:21 alexmiller the most common thing I find is large collections (default max size per coll is 20) - usually I set :gen-max 3 in all of my s/coll-of’s
2018:09:19 01:46:33 richiardiandrea 😄
2018:09:19 01:46:44 alexmiller recursive / nested specs can also be a problem but less common than the above
2018:09:19 01:47:29 richiardiandrea I have some kind of nasty :fn checks:
2018:09:19 01:50:49 richiardiandrea right, lemme try that
2018:09:19 01:52:27 richiardiandrea this seems to be definitely faster!
2018:09:19 01:52:48 richiardiandrea thanks! didn't thing about that 😄
2018:09:19 01:53:30 taylor I imagine evaluating that s/and spec in each iteration is more expensive than a single predicate
2018:09:19 01:54:15 richiardiandrea I was able to get up to :max-size 50 with that simple trick, I guess reporting will be different but that's ok for now
2018:09:19 01:48:10 richiardiandrea so I think this is where it's all clogged...
2018:09:19 01:48:29 richiardiandrea in ClojureScript too so I don't know, i might be many things
2018:09:19 01:48:36 richiardiandrea but thanks for the answer
2018:09:19 01:49:10 richiardiandrea if I go :max-size 20 it is taking more than I can wait for 😄10 is good
2018:09:19 01:55:55 richiardiandrea the trick in the tread above by @taylor makes things acceptable again:
2018:09:19 21:22:54 roklenarcic hm I've got a head scratcher. I have two types of keys in a hashmap (keywords and strings). If key is string then spec A needs to be applied to value, if not then spec B is applied to value
2018:09:19 21:23:02 roklenarcic what's the best way to achieve this
2018:09:19 21:23:37 roklenarcic currently I use seq to change map to sequence of pairs
2018:09:19 21:23:49 roklenarcic then it's trivial to spec
2018:09:19 21:44:53 richiardiandrea @roklenarcic I do that with exclusive maps and I use s/or
2018:09:19 21:45:21 richiardiandrea you could also use multi specs
2018:09:19 21:45:35 richiardiandrea (if I understand the problem correctly)
2018:09:19 22:16:20 alexmiller btw, rather than seq to pairs, you can just spec as s/every of s/tuple on the map
2018:09:19 22:17:00 alexmiller that’s a perfectly valid way. alternately, I think given just two choices, I’d lean on s/or over s/multi-spec for this
2018:09:19 22:18:39 roklenarcic thx
2018:09:20 07:07:58 jaihindhreddy Is there a catenation which doesn't care about order?
I have specs ::a, ::b, ::c and ::d.
I want a catenation in which ::a and ::b are required, ::c and ::d are optional and order doesn't matter. Is this possible with spec?
2018:09:20 12:01:16 alexmiller Sounds like s/keys*
2018:09:20 17:21:10 alexmiller or could be simpler if you want to spec less of the options (s/cat :s keyword? :_ #{'->} :t keyword? (s/* (s/cat :op symbol? :val any?)))
2018:09:21 10:44:43 jaihindhreddy @U064X3EF3 thank you sir! picard-facepalm I should have perused the clojure.spec.alpha docs.
2018:09:24 14:05:49 djtango oh wow didn't know that - but I suppose it makes sense for s/keys to work like that
2018:09:20 07:13:21 andy.fingerhut You could probably do the s/or of all the possible orders, but that would be pretty tedious and error prone.
2018:09:20 07:14:35 andy.fingerhut Why do you want a catenation if the order doesn't matter? Are these args to a function?
2018:09:20 07:19:18 jaihindhreddy Trying to implement serialization and deserialization of a protocol using conform and unform.
2018:09:20 07:19:53 jaihindhreddy To see just how much of it can be pushed into spec? This is the last step.
2018:09:20 07:22:09 jaihindhreddy Actual use case contains ~20 specs, so s/oring 20 factorial orders is out of the picture.
2018:09:20 16:31:51 lilactown I think I keep asking roughly the same question in different contexts but hoping someone can help me understand the best way to approach this:
I need to programmatically create a spec based on external data (specifically, a GraphQL schema/query). Is there any good examples out there of how to approach this?
2018:09:20 17:01:05 taylor I think two current options depending on type of spec are either use spec's internals or eval, and I think spec-tools lib has some features for this
2018:09:20 17:38:30 favila We've done this with code-generation
2018:09:20 17:39:36 favila build up quoted lists of code then write it out to a clj file
2018:09:20 17:39:54 favila (pprinted)
2018:09:20 17:40:45 favila if you have a deterministic order of evaluation (e.g. sorting the specs by name or topology or something) you will also see diffs clearly
2018:09:21 15:00:44 hjrnunes Hi,
I have a map for which there are several possible keys the values of which I'd like to use the same spec, for.
So ideally I'd do something like
2018:09:21 16:11:01 taylor in times like these, I restart REPL/computer/get back in bed
2018:09:21 16:11:11 lilactown hahaha
2018:09:21 16:11:30 lilactown I have restarted my REPL and everything. bed is tempting
2018:09:21 16:11:45 lilactown I wonder if there's something weird going on with CLJS instrumentation
2018:09:21 15:58:28 lilactown the test output/stacktrace doesn't give the line # or file or anything. but I'm confused WHY the arguments wouldn't satisfy that spec??
2018:09:21 17:03:58 alexmiller sorry, I was unaware of that one
2018:09:21 17:00:39 jrychter I'm writing up my thoughts on clojure.spec, having tried to use it for several months now in an application. I'm banging my head against multiple issues. So far it seems like most of them are related to data conversion: if I removed all conformers and :into from my specs and used separate explicit functions for data adjustments, I think my life with spec would be easier. But most writeups and presentations about spec tantalizingly mention conforming data as a major advantage...
2018:09:21 17:01:56 jrychter That is something that I think I missed (I don't think I'm the only one).
2018:09:21 17:02:01 alexmiller conforming was always intended to be an advanced tool to use in writing spec implementations, not for data transformation
2018:09:21 17:02:12 alexmiller it is intentionally absent from the spec guide
2018:09:21 17:02:23 jrychter And yet there is :into (in the guide, too).
2018:09:21 17:03:19 alexmiller yes, and is useful for some use cases (+ for gen), but that doesn’t make it a generic transformation tool
2018:09:21 17:13:26 jrychter Hmm. I think there is confusion around the issue (see for example https://stackoverflow.com/questions/45188850/is-use-of-clojure-spec-for-coercion-idiomatic). I think it's worth mentioning explicitly in the guide. I also found that there is not a lot of guides for spec. The "Spec Guide" is great, but I see it as more of a walktrough. I've been looking on tips on how to use spec in apps (e.g. do I use s/* or s/coll-of?). Most online presentations or tutorials are introductory material only.
2018:09:21 17:14:42 jrychter @alexmiller I have a file with my notes on spec (basically documenting the holes I've been falling into). I could E-mail it to you, would you like it? It would let you know what my (erroneous) thinking was.
2018:09:21 17:17:02 alexmiller the answers on that stackoverflow thread seem pretty straightforward and correct to me :)
2018:09:21 17:17:15 jrychter Yes.
2018:09:21 17:17:39 alexmiller you’re welcome to email, but I am just about to enter the Strange Loop black hole and won’t be looking at anything for Clojure for the next week and a half
2018:09:21 17:18:42 jrychter That's fine. I don't want you to answer that E-mail anyway. I'm not looking for support. I just thought that if I was the author of the spec guide, I'd want to know what some misconceptions are in readers' minds.
2018:09:21 17:19:30 alexmiller at the moment, I’m not looking to invest a lot of time in additional spec docs because we are starting to work on some spec changes that are likely to change some of whatever advice we would give
2018:09:21 17:23:37 jrychter An issue it will be, then. Perhaps the holes I'm falling into will influence some of the thinking behind spec changes 🙂
2018:09:21 17:25:18 alexmiller I do have a full day of spec materials I’ve taught several times as a course now
2018:09:21 17:25:49 alexmiller given time, could be turned into some useful advice
2018:09:21 17:33:31 seancorfield @jrychter FWIW, we do use spec for some very limited coercion but, as noted in one of those SO answers, we also have two types of spec: "API (coercing) specs" and "domain specs" (non-coercing). All of our internal specs are non-coercing. And even in the API specs, we only do very limited coercion: we accept strings-that-can-be-converted-to-<T> and produce values of type T, for longs, doubles, Booleans, and date/time values. That's it.
2018:09:21 17:33:58 roklenarcic is it possible to create a generator from a specced function (i.e. one with a fdef)?
2018:09:21 17:35:47 seancorfield We've found that to be extremely convenient because if you have a form field that should be a long, it's going to come in as a string so you either have a spec that attempts coercion to long but still conforms to the string and then you need an actual coercion as well, or you have a layer of coercion first followed by specs for whatever successfully coerces -- and then you have two layers in your error handling which makes for more complicated code.
2018:09:21 17:35:49 jrychter @seancorfield Well, my coercions were very limited, too. Strings to keywords and collection types, basically (to keep a collection as a set or a sorted set). But even then I'm falling into traps. An example is that I though that s/valid? tells you if your data is valid according to the spec. It doesn't. It tells you if the data will be valid if you pass it through s/conform.
2018:09:21 17:37:23 seancorfield Right, well the pattern we use is
2018:09:21 17:37:31 jrychter I'd argue that parsing strings to longs is not really coercion, it's parsing (my form code does it).
2018:09:21 17:39:11 jrychter @seancorfield Yes, I discovered this the hard way. Problem is that if you spec your functions, then data that is not valid (but conformable) will reach your function code.
2018:09:21 17:39:35 seancorfield We don't spec many functions -- we mostly spec data structures.
2018:09:21 17:40:23 seancorfield And, like I say, these are specs at the outer boundary of our system. Inside our system we only have non-coercing specs.
2018:09:21 17:40:54 jrychter I will try to remove all conformers (`s/conformer` and :into) from my specs and use explicit coercion. I'd love to be able to "attach" information about coercions to the specs, otherwise I have to manually define coercion functions and deal with nested coercions.
2018:09:21 17:41:29 seancorfield And our use of spec is almost all explicit and part of our production code. So we don't rely on instrumentation of functions that way.
2018:09:21 17:42:08 jrychter @seancorfield That's how I see it, too, but I'm still considering instrumenting functions, and looking for possible uses of spec.
2018:09:21 17:44:02 jrychter So, at a first glance, that spec-coerce library tries to do too much. I can see parsing from strings to doubles or integers. I'd rather have a way to attach custom coercion fns to specs and a way to walk a spec and apply all coercion fns that I specified.
2018:09:21 17:45:09 seancorfield Instrument (and check) are purely part of testing for us -- as well as using exercise to produce random conforming data for use in some example-based tests.
2018:09:21 17:45:58 jrychter I think I will also remove all uses of s/coll-of ... :into and convert those into explicit (s/and (s/coll-of ::something) sorted? set?) or similar. That will let me use s/valid? to catch instances of data where something isn't a sorted set.
2018:09:21 17:45:59 seancorfield As an FYI, from our Clojure codebase:
Clojure build/config 48 files 2538 total loc
Clojure source 268 files 63902 total loc,
3331 fns, 658 of which are private,
403 vars, 42 macros, 71 atoms,
478 specs, 19 function specs.
Clojure tests 149 files 20002 total loc,
23 specs, 1 function specs.
2018:09:21 17:47:39 jrychter I have ~45k lines of code in my app right now, so not nearly there, but close.
2018:09:21 17:49:32 alexmiller I think it’s useful to think of the purpose of conform as “why does this value conform to the spec?” and not “transform this value into some target value”
2018:09:21 17:50:41 jrychter I do think that s/valid? is not a good name, though.
2018:09:21 17:51:49 alexmiller because…
2018:09:21 17:52:23 jrychter Because it doesn't tell me if my data is valid.
2018:09:21 17:56:20 jrychter Both s/valid? will return true, even though the data, as supplied to s/valid?, will cause problems in functions that expect a set and a keyword, respectively.
2018:09:21 17:58:05 bbrinck For the first spec, would :kind set? be more accurate?
2018:09:21 18:00:11 jrychter Yes, I should remove all conformers and :into from my specs. My understanding right now is: s/valid? tells you if the data will be valid when conformed, so do not use s/valid? to check if the data is valid if you use conformers or :into anywhere in your specs.
2018:09:21 18:00:57 bbrinck Yes, the examples seems like concrete cases where using s/conformer for coercion creates headaches
2018:09:21 18:01:06 jrychter Hence my point about s/valid? not telling me if my data is valid.
2018:09:21 18:01:25 favila validity is only testable after conforming
2018:09:21 18:01:58 favila conforming isn't something separate that exists outside of an abstract expression of the desired result shape
2018:09:21 18:02:16 favila it's a predicate that also transforms
2018:09:21 18:04:10 bbrinck Correct, if you use conformer to do coercion (which is not recommended), the results of valid? are confusing. Also, some details of explain-data get confusing because the paths to data can change. This is one reason why coercion is not recommended, AIUI
2018:09:21 18:04:33 favila conforming is super-handy though
2018:09:21 18:05:05 favila but using it means you are indirectly specing the "input"
2018:09:21 18:05:31 favila the input must be something-that-i-can-make-into-something-else
2018:09:21 18:05:58 favila maybe think of conform as a normalization step
2018:09:21 18:06:04 favila rather than coersion or parsing
2018:09:21 18:06:22 favila that said I have written parsers using conform
2018:09:21 18:06:42 alexmiller conform is not normalization or coercion or parsing
2018:09:21 18:06:54 favila what's left?
2018:09:21 18:07:15 alexmiller destructuring
2018:09:21 18:07:28 favila ?
2018:09:21 18:08:32 alexmiller if the value matches the spec, it will destructure the input value to tell you which alternative was chosen in specs with alternates and which parts of the data matched which components of the spec
2018:09:21 18:08:35 jrychter That's how I understand it now, too.
2018:09:21 18:09:05 favila ok, I'm talking about conformers
2018:09:21 18:09:10 alexmiller tags exist in s/or, s/alt, s/cat so that parts and choices can be labeled in the conformed (destructured values)
2018:09:21 18:09:13 favila nevermind we are talking past each other
2018:09:21 18:09:40 alexmiller conformers exist to build complex composite spec implementations from existing pieces
2018:09:21 18:12:00 favila what would be an example of that? s/keys* does not appear to use conformers
2018:09:21 18:15:10 alexmiller oh, I might have called out the wrong example (s/keys* uses s/&). might be remembering s/nilable (which actually has been rewritten since for performance)
2018:09:21 18:17:58 favila ah but the & predicate is a conformer
2018:09:21 18:18:17 favila it appears to be the only use in spec
2018:09:21 18:19:30 favila so I think you are saying that the intended use of conformers is to act as glue to chain specs together
2018:09:21 18:19:57 favila it's not meant to make s/conform visible alternations
2018:09:21 18:21:37 alexmiller yes
2018:09:21 18:09:49 alexmiller like building s/keys* from s/keys
2018:09:21 18:10:32 alexmiller conform is not transformation, conformers are not coercion
2018:09:21 18:11:05 jrychter So, I'm looking for suggestions: what is the simplest way to add my own coercion functions to certain specs and then walk a spec calling my functions? Metosin's spec-tools do this by wrapping specs, which I really do not like. Is that the only way?
2018:09:21 18:11:57 alexmiller ^^ is vastly preferable to spec-tools imo
2018:09:21 18:12:45 jrychter @alexmiller At a first glance, this seems to try to do too much. I don't want to parse strings to integers by default. I only want to walk the spec and call specific coercion functions that I provided. But perhaps I'm missing something, I'll look deeper.
2018:09:21 18:13:28 favila @jrychter spec really pushes you in the direction of using different keys
2018:09:21 18:14:06 favila i.e. if you care about both the "raw" data and the "cleaned-up" (slightly-parsed, whatever) data as independent specs, you need separate keys
2018:09:21 18:14:51 jrychter It seems that with spec-coerce you have to use coerce-structure to walk a complex (nested) spec, duplicating your data structure in the call.
2018:09:21 18:19:09 jrychter Basically, I'm looking for something that would let me do this:
…without touching anything that doesn't have a coerce-fn defined.
2018:09:21 18:25:19 favila yeah spec-coerce seems backward; I would want to opt-in to coercion, not have universal coersion predicates
2018:09:21 18:26:35 bbrinck Seems like a sensible suggestion to have the option to turn off the default coercions
2018:09:21 18:26:49 bbrinck (a suggestion for spec-coerce project, I mean)
2018:09:21 18:27:23 favila I'll reiterate though that spec really seems to urge in the direction of more specs
2018:09:21 18:27:36 favila i.e. a separate "edge" spec vs an internal spec
2018:09:21 18:28:27 favila so in your example "::kw as a string encoding a keyword" is different from "::kw as a kw"
2018:09:21 18:28:53 favila and then rename keys everywhere
2018:09:21 18:29:04 favila (if using s/keys with namespaced maps)
2018:09:21 18:36:07 bbrinck @jrychter A challenging thing in your example above is that the coerce-fn is tied to something global (the qualified kw), but your data {:nested {:kw "x"}} is presumably figuring out that :kw means ::kw based on context?
2018:09:21 18:36:13 jrychter I need a way to walk a spec, which doesn't seem to exist. I thought about defining multimethods dispatching on spec keys, but that won't work for unqualified keys.
2018:09:21 18:37:33 jrychter @bbrinck I'm passing all information about the context as the first argument to coerce: the ::data spec knows everything.
2018:09:21 18:37:59 bbrinck right, but you’d have to walk the spec to know what spec :kw means
2018:09:21 18:38:19 bbrinck like you said above, that adds a requirement to walk the spec to know where you are in the spec
2018:09:21 18:40:19 jrychter Yes, but I thought that is what spec does anyway (when, say, conforming).
2018:09:21 18:41:27 bbrinck correct, you’d need to duplicate this process. it’d be an interesting project, I’d need to dig more into the spec impl to understand how viable it’d be to duplicate
2018:09:21 18:42:11 favila doesn't spec-tools have a spec visitor?
2018:09:21 18:42:28 bbrinck OTOH, if you could assume that the name unqualified keyword always used the same coercion, the problem is simpler
2018:09:21 18:43:37 bbrinck which doesn’t seem entirely crazy. Perhaps naively, I’d think that if you want to, say, convert :zip-code from string->int in one place, you probably want to do the same coercion elsewhere
2018:09:21 18:43:47 bbrinck maybe that’s oversimplifying the real world case 😉
2018:09:21 18:44:32 jrychter Not necessarily, but I could stick with that for a while as a workaround. That would mean I could define a multimethod coercer and walk the structures myself. But then spec provides me very little value in the end.
2018:09:21 18:45:35 bbrinck Hm, well spec would still let you validate the result of coercion (and conform to use destructuring)
2018:09:21 18:46:27 favila if you used the "option" keys in a consistent way, s/conform could perform the tagging for you (except for s/keys req-un and opt-un)
2018:09:21 18:46:37 bbrinck Note that if you don’t to have to call two different implementations of def for :kw, then you could just make a macro that does both e.g.
2018:09:21 18:47:52 bbrinck @jrychter (an example of a def macro that calls the normal def but also registers a message. In your case, you’d be registering a spec + coercer)
2018:09:21 18:51:20 bbrinck @jrychter whoops, I messed up the example in that gist. Please reload, I’ve updated. The example should have been: (coerce-structure {:nested {:kw "x"}}) ;; => {:nested {:kw :x}}
2018:09:21 18:51:34 jrychter That is interesting. It's not as good as clojure.spec calling my coercions (clojure.spec would know exactly which function to call), but I guess it is a workaround.
2018:09:21 19:01:24 jrychter @bbrinck Thank you for that gist. I am trying to implement that right now to see how it goes.
2018:09:21 19:03:30 bbrinck @jrychter np. Good luck! Definitely a workaround, but hopefully covers the 80% case. Let me know how it goes 🙂
2018:09:21 19:11:47 jrychter @alexmiller Can we hope that this use case (explicit coercion using user-registered coercion functions) will be considered in future spec work?
2018:09:21 19:12:11 alexmiller you can hope for anything you like :)
2018:09:21 19:12:22 alexmiller not something we’re working on currently
2018:09:21 19:18:48 jrychter Let's narrow it down a bit: at least a way to walk a spec, calling a function with the keyword and value at each point. That would still mean maintaining a separate registry, but that's fine.
2018:09:21 19:27:50 jrychter @bbrinck I'm discovering more limitations as I go — for example, a certain spec might be redefined as s/nilable in some places under the same (when unqualified) keyword.
2018:09:21 19:30:25 bbrinck @jrychter Right, it’s probably worth making all coercing functions a little bit defensive i.e. if the pre-coercion type isn’t what you expect, just return the existing value
2018:09:21 19:30:47 bbrinck so, untested, but something like (if (string? x) (keyword x) x))
2018:09:21 19:31:44 bbrinck Keep in mind the role of coercion functions is not to validate the incoming data, but rather do a best-effort attempt to get the fields into a format that will be valid according the spec
2018:09:21 19:32:25 jrychter Of course. And the suggestion about being defensive is a very good one.
2018:09:21 19:55:06 jrychter And another limitation (as I'm going through my code): if a spec is redefined under a different name, coercion needs to be redefined for that new name, too. It won't carry over.
2018:09:21 20:05:50 bbrinck Very true, it’s per name, not per spec
2018:09:21 20:06:17 bbrinck You could do some magic looking at s/form for a spec and then look up that spec … but it’d get complicated 😉
2018:09:21 18:22:27 josh.freckleton Is spec the right tool for validating data from JSON and db results (if so, any docs or blogs about it?), or should I stick with Schema?
2018:09:21 18:24:26 alexmiller spec is somewhat cumbersome right now for validating maps with unqualified keys (which tends to include cases like JSON)
2018:09:21 18:24:45 alexmiller the next round of spec changes will have some improvements in this area
2018:09:21 18:26:37 roklenarcic Is there some way to refer to args spec of a function?
2018:09:21 18:29:50 seancorfield @roklenarcic Can you provide more context? What problem are you trying to solve?
2018:09:21 18:31:26 roklenarcic I'm trying to generate arguments to a function that is specced
2018:09:21 18:31:34 roklenarcic but without calling it
2018:09:21 18:32:08 roklenarcic so I have a function and a bunch of fdefs
2018:09:21 18:32:21 roklenarcic I mean one fdef for this particular one
2018:09:21 18:32:38 roklenarcic and I want to generate vectors of arguments
2018:09:21 18:35:12 alexmiller yes, if you get a function spec, it implements key lookup so you can grab :args, :ret, :fn out of it
2018:09:21 19:33:27 ikitommi out of luck. To make a standalone transformer outside of spec, one would need to be able to parse the spec and basically rebuild the s/conform to understand what branches are being used.
2018:09:21 19:36:44 ikitommi spec-tools uses the s/conform and makes the spec do all the heavy lifting. but sadly, the leaf specs need to be wrapped into “conforming predicates”.
2018:09:21 19:39:27 ikitommi spec-coerce reads the spec forms, which works nicely for most/simple forms.
2018:09:21 19:47:06 ikitommi hopefully there will be a solution for this in the upcoming spec releases, I think it would be bad for the clojure / spec story not to support (or enable libs to fully support) coercion. one s/walkmethod on Specs for libs to use? protocol dispatch would be clean and much faster than parsing the forms or using dynamic binding like the current coercion libs do.
2018:09:21 19:56:27 jrychter @ikitommi It seems that that is exactly what I have been asking for in that thread (a way to walk a spec).
2018:09:21 19:58:30 alexmiller that is something we have talked about providing, but it’s a ways down the list
2018:09:21 19:59:23 jrychter It would seem that this is something that people with large apps with complex data structures and JSON interop encounter all the time.
2018:09:21 20:02:42 seancorfield We haven't encountered that yet 🙂
2018:09:21 20:03:28 jrychter @seancorfield You seem to be using conformers for that, judging from the code you've shown?
2018:09:21 20:04:06 seancorfield Only right at the edge -- where I think it's acceptable.
2018:09:21 20:04:42 jrychter But that proves my point. Coercion is needed.
2018:09:21 20:04:55 seancorfield Not "needed". Convenient.
2018:09:21 20:05:04 seancorfield And spec provides what we need already.
2018:09:21 20:05:57 jrychter I'd argue that it doesn't. Or my expectations are too high: I expected to be able to "spec" my data structures only once, and reuse the resulting structure for coercion. I can't do that right now.
2018:09:21 20:06:46 jrychter I can see how this could be outside the scope of spec, but this is why I'm asking (and @ikitommi seems to be, too) for a way to walk the spec data, so that we can extend it and reuse the definitions.
2018:09:21 20:23:00 ikitommi @jrychter there is CLJ-2251
2018:09:21 20:13:09 alexmiller does it seem weird to you to have a “spec” that defines your data, but then require coercion that does not conform to that spec? (it seems weird to me)
2018:09:21 20:19:08 ikitommi web is weird. we expect a valid edn data but get JSON instead ;)
2018:09:21 20:23:49 alexmiller it seems conceptually cleaner to me to separate the transformation of the crap that you get into the format you want from validating that transformed data
2018:09:21 20:24:21 alexmiller and I get that you’ve already defined the structure and it seems convenient to use that structure to learn about target expectations while doing the transformation
2018:09:21 20:24:45 jrychter @alexmiller In theory, yes. But in practice I'm dealing with data from a JSON database, nested data structures. What you are proposing means that I will be duplicating data structure definitions: once for spec, and again for coercion. Seems like a waste.
2018:09:21 20:24:57 alexmiller and so I think requests that deal with using specs as data in that way make sense
2018:09:21 20:25:25 alexmiller asking spec to do the work seems weird to me (but a good case for a lib on top)
2018:09:21 20:25:29 wilkerlucio @jrychter had you tried using coerce-structure from spec-coerce? because thats the indented feature for it, you pass it a structure and it will use the leaf attribute specs to do the coercion
2018:09:21 20:26:05 jrychter @alexmiller Actually, I do not want to involve "target expectations" in this. All I want is to reuse the structure from spec and be able to hang my own data (coercion functions in this case) on it.
2018:09:21 20:26:32 alexmiller well that would potentially be handled by meta support, which we’re in favor of (it’s just tricky to implement well)
2018:09:21 20:27:04 jrychter @wilkerlucio Yes. I am using a variant of that now, suggested by @bbrinck. But this has a number of drawbacks.
2018:09:21 20:27:05 alexmiller as a stop gap, it’s not hard to build a second registry, also keyed by spec name that had that info
2018:09:21 20:27:09 ikitommi @alexmiller agrer that it should be made on top. But we need help from spec to make this possible. E.g. the walk
2018:09:21 20:27:37 alexmiller yes
2018:09:21 20:27:37 jrychter @alexmiller That is exactly what I'm doing right now, and there are a number of limitations, especially with non-namespaced keys.
2018:09:21 20:27:52 alexmiller that may become clearer soon
2018:09:21 20:27:59 wilkerlucio spec-coerce also supports a secondary registry to specify, what I would like from spec is to be easier to traverse the spec definitions, I have to do some things that I feel are not very stable to get that
2018:09:21 20:28:16 wilkerlucio but I guess specs on the specs will solve this
2018:09:21 20:28:31 alexmiller maybe
2018:09:21 20:29:08 alexmiller this is all helpful, in particular I’m trying to clarify the problem statement so I can talk well to Rich about it
2018:09:21 20:29:52 alexmiller spec walking has come up in several contexts and is one potentially useful generic feature, meta is another
2018:09:21 20:30:06 jrychter @alexmiller It seems to me that you think much of this is unnecessary, because invalid data points to deficiencies in transport or db. That is true in general. But please consider that even using EDN you will not get sorted sets by reading. Something needs to coerce sets into sorted sets.
2018:09:21 20:30:20 alexmiller and better support for unnamespaced attributes
2018:09:21 20:30:43 alexmiller I didn’t say it was unnecessary. I’ve built my share of apps and I get it.
2018:09:21 20:31:00 alexmiller I just don’t think that spec necessarily needs to be the thing providing “coercion”
2018:09:21 20:31:26 jrychter Oh, absolutely. It's just that it makes sense to re-use the resulting structure.
2018:09:21 20:32:17 alexmiller I’m agreeing with you on that
2018:09:21 20:33:52 alexmiller using conformers for this stuff is what Rich calls the “meat grinder” approach - I see that turning the crank gets me from A to B
2018:09:21 20:34:02 ghadi Part of the challenge is that 99% of other libraries provide "coercion" of some sort (plumatic/schema and everything in Python) and there are Real Problems with that stuff. I think it has psychologically primed us to want to tangle it together... some caution is advisable and focus on the problems
2018:09:21 20:34:18 alexmiller but it misses that the grinder is not the point of it
2018:09:21 20:35:53 jrychter Here's a practical example from my screen right now:
(ns partsbox.build-quantity
(:require [clojure.spec.alpha :as s]
[partsbox.coerce :as c]))
;; A build quantity is an integer that is always greater than 0.
(s/def ::quantity pos-int?)
;; A non-empty sorted set of quantities used for build quantities and pricing quantities.
(s/def ::quantities (s/and (s/coll-of ::quantity :min-count 1) sorted? set?))
(c/defcoercion ::quantities (fn [data] (into (sorted-set) data)))
(def +default-quantities+ (s/conform ::quantities (c/coerce ::quantities [1 10 25 50 100 250 500 1000])))
This is all fine (a separate registry, another line of code for defining the coercion). I would just want to be able to write a general c/coerce that would walk the spec correctly. I can only have a deficient implementation right now.
2018:09:23 08:39:32 dottedmag I'd like to instrument a function taking an argument spec'ed in another namespace. When I do (s/fdef my-func :args (s/cat :arg1 :alias/myspec)) it fails to resolve spec via alias. (s/fdef my-func :args (s/cat :arg1 :)) works. Is it intended?
2018:09:23 11:36:00 taylor for ::alias/myspec to work I think you need to require/alias that namespace e.g. (require '[some.ns :as alias])
2018:09:23 13:52:11 dottedmag @U3DAE8HMG Yep, thanks. I was using wrong syntax (with a single :).
2018:09:23 08:40:38 dottedmag Oh, :alias/myspec vs ::alias/myspec.
2018:09:25 01:46:36 bbrinck I’m confused about the behavior of explain for keys* specs
2018:09:25 01:46:58 bbrinck For a cat spec, the in refers to the to the position of the bad data before conformance
2018:09:25 01:47:15 bbrinck but for a keys* spec, it seems to refer to the position of the bad data after conformance.
2018:09:25 01:55:05 taylor >takes the same arguments as spec/keys and returns a regex op that matches sequences of key/values, converts them into a map, and conforms that map
2018:09:25 15:29:03 bbrinck Ah, thanks! Although that explains the mechanism of why the two are different, I’m still not sure on the reasoning 😉
2018:09:25 15:31:10 bbrinck i.e. why the difference in behavior is desirable
2018:09:25 15:32:09 taylor same but that's out of my depth 🙂 I only noticed the keys* impl. would seem to cause this behavior
2018:09:25 15:33:14 bbrinck Yep, I appreciate the link 🙂 It does indeed explain the difference. Thx!
2018:09:25 15:33:56 taylor I suppose it may be just to make the final conform step easier, when the input has already been conformed to a map :man-shrugging:
2018:09:25 15:36:38 bbrinck Yeah, maybe! The downside is that when I want to locate the source of the bad data, the in value is inconsistent in it’s format. In particular, it’s hard to make a custom printer like Expound be able to locate the bad data
2018:09:25 01:47:47 bbrinck Is this a known issue? Or am I misunderstanding the intent here?
2018:09:25 19:15:21 roklenarcic Hm if I define a spec, it requires that components are already defined. So if I have mutually recursive spec, do I just define one of the components at the top with bogus spec?
2018:09:25 19:46:52 taylor I've run into this issue before with a recursive spec, and I think it involved a call to s/spec inside the recursive spec definition
2018:09:25 19:23:21 donaldball I’m not sure what you mean by it requiring that components are already defined. Certainly something recursive like this works fine:
2018:09:25 21:34:58 richiardiandrea Hello folks I am having a hard time reading this spec error, I think I need some help in understanding what is wrong:
#### [{:spec #object[cljs.spec.alpha.t_cljs$spec$alpha23581], :clojure.test.check/ret {:shrunk {:total-nodes-visited 0, :depth 0, :pass? false, :result #error {:message Call to #'lambda.response/error-map did not conform to spec:
val: "Invalid body" fails at: [:args] predicate: (cat :message string? :data (? (nilable map?)))
:cljs.spec.alpha/spec #object[cljs.spec.alpha.t_cljs$spec$alpha23551]
:cljs.spec.alpha/value "Invalid body"
:cljs.spec.alpha/args "Invalid body"
:cljs.spec.alpha/failure :instrument
, :data {:cljs.spec.alpha/problems [{:path [:args], :pred (cljs.spec.alpha/cat :message cljs.core/string? :data (cljs.spec.alpha/? (cljs.spec.alpha/nilable cljs.core/map?))), :val Invalid body, :via [], :in []}], :cljs.spec.alpha/spec #object[cljs.spec.alpha.t_cljs$spec$alpha23551], :cljs.spec.alpha/value Invalid body, :cljs.spec.alpha/args Invalid body, :cljs.spec.alpha/failure :instrument}}, :result-data {:clojure.test.check.properties/error #error {:message Call to #'lambda.response/error-map did not conform to spec:
2018:09:25 22:24:51 richiardiandrea Let me see if that's the issue
2018:09:25 22:34:54 richiardiandrea Well it seems like things are working with just a non-variadic function, however I am not sure my bug is related to the above
2018:09:25 23:16:27 richiardiandrea Yep it is relate, I can reproduce it.
2018:09:25 23:16:50 taylor what's your defn error-map arg list look like?
2018:09:25 23:37:43 richiardiandrea it was [message & [data]]
2018:09:25 23:38:26 taylor it could be related to that JIRA I mentioned above about CLJS, instrumentations, and variadic functions
2018:09:25 23:56:38 richiardiandrea Yep left a comment there, I can reproduce it here...
2018:09:26 07:21:27 steveb8n question : I’ve heard about context specific specs quite a few times i.e. same entity but different uses e.g. reading vs inserting a map from/to db. another might be reading entities in a list vs individually. I can guess how these specs might look but was hoping I could learn from looking at others. does anyone know of public codebases with examples of this technique?
2018:09:26 12:11:29 jaihindhreddy @steveb8n I would be interested to see this too.
But what you describe feels like something spec is not meant to do. Spec specs data, not entities or identities. ::entity-while-reading, ::entity-while-inserting might be better as two different specs. Maybe metosin/spec-tools can do this.
2018:09:26 12:14:25 steveb8n agreed. I think each context is probably it’s own spec.
2018:09:26 12:59:19 favila Yeh spec doesn’t support this, not sure where you heard about that. If you find out I’m interested
2018:09:26 13:03:23 favila There are two cases that hurt where I want this. One is dealing with datomic data. Because it’s a graph each ref key could be (in the widest possible case) an entity id, a string, a lookup ref, or a nested vector or map, again with unknown keys ( depends on what was pulled)
2018:09:26 13:05:00 favila At certain spots in the program I want to allow only a few of those possibilities. Eg On read I often want to guarantee that a certain set of sub-keys was pulled
2018:09:26 13:06:53 favila The other case is dealing with data where the same key is used but it may have a narrower spec depending on the “kind” of map it is in
2018:09:26 13:10:33 favila Eg a key could have values 1 2 or 3, but that key must have value 1 when some other key has a certain value
2018:09:26 13:11:14 favila Multi-spec and predicates can do this but it’s awkward and verbose and doesn’t gen well
2018:09:26 13:11:14 favila Multi-spec and predicates can do this but it’s awkward and verbose and doesn’t gen well
2018:09:26 16:59:58 pvillegas12 Interested about this, I would like to be able to express that a restriction on a composed spec. If spec1 has value :a within a set of alternatives, spec2 cannot have that value (both of spec1 and spec2 are keys of another spec)
2018:09:26 20:55:43 pvillegas12 To do this, one can s/and an arbitrary function at the compound spec level to make this happen defined as the second operand (for gen support to be there)
2018:09:26 21:17:18 favila s/and of s/keys works very badly
2018:09:26 21:17:38 favila it's unlikely a useable generator will result
2018:09:26 21:17:50 favila s/and maps to gen/such-that
2018:09:26 21:18:12 favila this is why s/merge exists
2018:09:26 21:22:28 pvillegas12 In my case I don’t want to merge two sets of specs. I have a spec that is the composition of 5 specs (with s/keys) and I wanted two ensure that two of those did not have the same value
2018:09:26 21:23:03 pvillegas12 This works with gen because the set of possibilities for those specs is 2 😛 (a really small set)
2018:09:26 21:24:23 favila oh that's not at all what I run in to
2018:09:26 21:24:32 pvillegas12 what do you run into?
2018:09:26 21:24:56 pvillegas12 (I can see how distinct-origin-target can bump gen if the resulting restriction is too big a set)
2018:09:26 21:25:38 favila something like if :m/doc-type is "x", :m/origin must be some narrower range of values that is normally allowed
2018:09:26 21:25:53 favila vs if :m/doc-type were "y" or "z"
2018:09:26 21:26:24 pvillegas12 Right, if m/origin for example is restricted to a given string matching a regex it will destroy gen’s ability to generate example data
2018:09:26 21:27:20 pvillegas12 not sure how with-gen would work with that compound spec though
2018:09:26 21:27:26 pvillegas12 (that’s how I make sure gen continues to work)
2018:09:26 21:39:24 favila s/keys+ works like s/keys, except you can add a :conf map from spec->override
2018:09:26 21:39:52 favila the spec is checked all the time
2018:09:26 21:40:12 favila but the override is also checked, and is used for gen
2018:09:26 21:44:09 favila this is really just contravariance/covariance at the end of the day. The trouble is spec doesn't have a way to structurally use the same map key and spec that key two different ways
2018:09:26 13:13:39 favila I wrote a hacky keys+ macro to try and do this. It allows an inline redef of a key’s spec+generator, but it will still conform both the “natural” and the redef-ed spec of the key
2018:09:26 17:03:32 justinlee In my reagent app on cljs, I use a spec validator on my app’s state atom. This works well, but is unfortunately a bit slow in some cases. Is there anything clever one can do to do validations in this way but skip parts of the state tree that haven’t changed? The answer might be just to use more than one atom.
2018:09:26 17:07:00 noisesmith the validation function receives the two states as args, so at least hypothetically you could check only changed subtrees using cheap identity checks
2018:09:26 17:08:04 noisesmith I'd imagine a classic recursive tree walk, either short-circuiting if identity is true, or checking if identity was false, for each branch new/old
2018:09:26 17:09:20 noisesmith then again you'd have to also walk subtrees of the spec to really make it work...
2018:09:26 17:09:26 noisesmith sounds messy
2018:09:26 17:11:55 dadair Is the state atom flat or deep? If its flat you could have a spec for each of the main subtrees, and only validate the main subtrees that have changed?
2018:09:26 17:12:10 noisesmith that is probably the best plan
2018:09:26 17:16:49 justinlee yea that’s probably the right approach. actually i already have specs for each of the main subtrees
2018:09:26 17:19:06 noisesmith and I bet using identical? instead of = performs better for checking each subtree for changes, but that should be straightforward to swap out and test
2018:09:26 17:29:00 justinlee that works beautifully. the obvious thing i didn’t realize is that the validator of course is called before the atom is changed so you can just compare it against the old atom
2018:09:26 17:32:04 pvillegas12 I would like to be able to express that a restriction on a composed spec. If spec1 has value :a within a set of alternatives, spec2 cannot have that value (both of spec1 and spec2 are keys of another spec’d map)
2018:09:26 17:32:09 pvillegas12 Is there some way to achieve this?
2018:09:26 17:36:58 noisesmith @lee.justin.m one of the args to the validator function is the old state, one is the new state
2018:09:26 17:40:09 misha assuming the state tree is a map, and if there are no cross-keys validations (if k1 is this, k2 should be that), you can top level diff new and old values, and just test it against empty (s/keys) @lee.justin.m
2018:09:26 17:42:07 misha might save you some milliseconds, if state is huge, but diff is just few keys at a time
2018:09:26 17:44:01 justinlee @noisesmith I was using https://clojuredocs.org/clojure.core/set-validator! which says `validator-fn must be nil or a
side-effect-free fn of one argument, which will be passed the intended
new state on any state change`
2018:09:26 17:44:09 justinlee is there something else I could be using?
2018:09:26 17:44:57 justinlee @misha yea there aren’t that many top level keys and most of them won’t change, so I think I can just test them with identical?
2018:09:26 17:45:47 misha that's the validator, yes. I don't know about other ones either. I think @noisesmith meant usual ref-callback with [key ref old new] signature
2018:09:26 17:47:23 dadair that’s using add-watch
2018:09:26 17:48:25 noisesmith @dadair yeah that's the one I was thinking of
2018:09:26 17:48:40 dadair it mostly depends on how you want to handle the spec failure
2018:09:26 17:48:43 noisesmith I got the two functionalities confused
2018:09:26 17:55:22 justinlee the only thing I’d really like to do is have the validator throw an error synchronously so that i get a stack trace from the offending function, though this is straying from the channel topic
2018:09:26 19:36:46 justinlee are specs queryable once they’ve been made? it’d be cool to ask for the spec corresponding to a key in a map dynamically
2018:09:26 23:18:11 seancorfield @chris547 If you call s/explain (instead of s/conform) in that second case, what does it say? I suspect you have a spec for ::email/creation-timestamp and your string is not valid for that.
2018:09:26 23:18:40 Chris It tells me that creation-timestamp doesn't match the :not-found predicate
2018:09:26 23:20:41 Chris because all it should care about is if ::email/email-address conforms, since that's the only key mentioned in the :success condition of the s/or
2018:09:27 00:01:25 seancorfield You're only reading the last part of the failure -- the first part says "2018-09-26T21:47:57.304Z" fails spec: :io.cloudrepo.signup.email/creation-timestamp
2018:09:27 00:01:53 seancorfield So you have an ::email/creation-timestamp spec and that string fails to conform to it.
2018:09:27 00:02:46 seancorfield Since that fails, the whole :success part fails and the the whole hashmap fails the :not-found part -- the val in explain is the whole hash map, not just the creation timestamp string.
2018:09:27 00:03:30 seancorfield s/keys will conform/validate any keys that are provided that have a corresponding spec, not just the listed :req keys.
2018:09:27 00:03:39 seancorfield ^ @chris547
2018:09:27 00:06:12 Chris Ah, that makes sense now
2018:09:27 00:06:36 Chris I didn't even check to see if the timestamp was okay because I didn't care about it
2018:09:27 00:06:55 Chris but I guess that's good because it will catch where you create bad data at the first possible point
2018:09:27 00:07:02 Chris rather than when you explicitly check, right?
2018:09:27 00:12:06 seancorfield Yup. The assumption is that specs should apply wherever those uniquely-name keys appear.
2018:09:27 00:12:39 seancorfield If you're using unqualified keys, you won't run into that -- since only the :req-un and :opt-un keys will map to specs.
2018:09:27 00:15:57 Chris Okay, thanks Sean - you're always super helpful!
for a data structure like [:jdbc/insert :foo {:bar "baz"}]. how would I add an optional 4th element at the end?
2018:09:27 23:07:26 seancorfield
:options (s/? ::opt-spec)
2018:09:27 23:08:05 seancorfield (assuming you wanted to optionally pass in an options map of some sort @lilactown)
2018:09:28 09:50:02 manutter51 I’m trying to run (stest/instrument) in a clojurescript repl, but I can’t even get that far:
dev:cljs.user=> (require '[cljs.spec.test.alpha :as stest])
---- Could not Analyze ----
No such namespace: clojure.test.check, could not locate clojure/test/check.cljs, clojure/test/check.cljc, or JavaScript source providing "clojure.test.check" in file file:/Users/mark/.m2/repository/org/clojure/clojurescript/1.10.339/clojurescript-1.10.339.jar!/cljs/spec/test/alpha.cljs
---- Analysis Error ----
2018:09:28 09:52:01 manutter51 Am I missing a step somewhere?
2018:09:28 09:55:38 manutter51 I just added the explicit dependency on spec.alpha 0.2.176 this morning, but it makes no difference, I get the same error either way.
2018:09:28 09:56:13 valerauko i thought you needed to depend on clojure.test for generating etc
2018:09:28 10:23:15 manutter51 You ought to be able to load the namespace though, right?
2018:09:28 10:24:10 manutter51 I’m spinning up a brand new REPL, and the first thing I type is the require, and it won’t load the namespace.
2018:09:28 11:10:05 mgrbyte I think they are the specs that have been added for clojure language itself.
2018:09:28 11:22:09 manutter51 Ah, that would make sense.
2018:09:28 11:23:15 manutter51 So is anybody else having this problem? Can’t require the cljs.spec.test.alpha ns in a freshly-started REPL?
2018:09:28 11:24:31 manutter51 It does seem suspicious that it’s looking for it specifically inside the clojurescript jar when I have a perfectly good clojure jar on the same classpath.
2018:09:28 22:51:11 manutter51 I didn’t get to it on my lunch break, but I just now added the dependency to test.check, and it works great! Thanks a ton! 😄
2018:09:28 12:01:16 manutter51 Ok, thanks, I'll try that on my lunch break; got to start day job now.
2018:09:28 12:25:54 urzds Hi! I just tried to update to Clojure 1.9 and get weird errors when building my code: :cause Call to clojure.core/refer-clojure did not conform to spec:
Sadly this error message is so convoluted that I cannot even figure out which file is causing this. Any idea what to look for?
2018:09:28 12:26:19 urzds (We do not use spec ourselves.)
2018:09:28 13:01:01 mpenet you likely have malformed ns declaration(s)
2018:09:28 13:01:59 mpenet or one of your deps
2018:09:28 13:07:34 favila Unfortunately the error will say it is coming from the requiring ns instead of the file it is in
2018:09:28 13:08:59 favila The ns form isn’t been evaluated yet (because it has an error) so the current ns ( as shown in the message) is still the requiring one
2018:09:28 15:01:32 urzds @mpenet, @favila: So the namespace that requires the broken namespace is the one mentioned in the LOAD FAILURE for <ns> line?
2018:09:28 15:01:58 favila that's been my experience, yes
2018:09:28 15:08:38 urzds Also, in the backtrace it mentions a different file from the namespace it mentions in the LOAD FAILURE for ... line.
2018:09:28 15:09:34 urzds So my guess would be that the file it mentions in the backtrace actually contains something funny. But I have no idea what I am looking for...
2018:09:28 15:14:28 dadair Looks like it is complaining about the “:as core” part
2018:09:28 15:30:44 seancorfield Yeah, the syntax is (:refer-clojure :exclude [...]) -- that :as core part is what is blowing up.
2018:09:28 15:31:25 seancorfield @urzds if you post a Gist of your dependencies and/or the full stacktrace, we can probably narrow it down for you.
2018:09:28 15:38:37 urzds Different question (I am trying to stay on 1.8 and first make sure all my deps are still working): Is it possible to hot patch a function into clojure.core? Lacinia Pedestal is broken on Clojure 1.8, because it uses a function from 1.9: https://github.com/walmartlabs/lacinia-pedestal/issues/80
2018:09:28 16:39:23 seancorfield @urzds Is there a pressing reason to stay on 1.8?
2018:09:28 16:41:47 seancorfield (I think, yes, you can define your own clojure.core/pos-int? but you'd need to do it somehow before that Lacinia namespace was even loaded so...)
2018:09:28 16:46:06 urzds @seancorfield No pressing need. But before I try to upgrade to Clojure 1.9, I want to make sure everything still works after upgrading all my deps to versions that should in theory be capable of supporting 1.9.
2018:09:28 16:52:59 seancorfield I suspect you'll be caught between a rock and a hard place there with anything that already uses clojure.spec since such code may assume a number of things present in 1.9 -- although I thought clojure-future-spec was supposed to patch those up as well?
2018:09:28 16:56:10 seancorfield Ah, I see... it expects you to (:require [clojure.future :refer :all]) into the spec-using namespace but doesn't add anything to clojure.core directly. So it only helps you use clojure.spec in your own code -- that's not going to help with any other code that already assumes 1.9+.
2018:09:28 16:57:24 urzds Ah, but that's nice. I am right now writing my own shim... 😄
2018:09:28 16:58:50 urzds So I'll just patch up lacinia-pedestal with clojure.future instead of my own shim.
2018:09:28 16:59:46 seancorfield Yeah, I think you'll have to fork lacinia-pedestal for now, patched with clojure.future, and then revert to the official version when you're ready to move to 1.9.
2018:09:28 17:00:21 seancorfield But, frankly, I would consider that a waste of effort and just go ahead with the 1.9 upgrade and do any necessary work there instead.
2018:09:28 17:00:26 urzds Yes, but they even claim to support Clojure 1.8, so it's actually a bugfix. I'll just send them a PR.
2018:09:28 17:01:22 seancorfield But on 1.8, doesn't it just not use ? That's how most of the libraries seem to handle 1.8 compatibility when they have specs?
2018:09:28 17:01:38 urzds Not this one... 😞
2018:09:28 17:02:09 seancorfield So it already assumes clojure-future-spec is being used? That's kinda odd...
2018:09:28 17:03:41 seancorfield Ah... and then clearly don't test against that setup 🙂
2018:09:28 17:05:24 seancorfield Overall tho', I still suspect you'd have a lot less pain trying Clojure 1.9 directly instead of all this working-around-spec on 1.8 🙂
2018:09:28 17:07:32 seancorfield Probably a good idea to create an issue (or send them a PR) to add a :1.8 alias to project.clj with Clojure 1.8 and clojure-future-spec so it's easy to test with in the future (and they could auto-test against both 1.8 and 1.9 on CircleCI).
2018:09:28 17:22:42 urzds @seancorfield Thanks for all your help! I don't really know what you refer to with the last line, though.
2018:09:28 17:26:15 urzds Now the spec violations vanished. That's weird...
2018:09:28 17:28:26 urzds Probably it was good that I tinkered with the different dependencies for so long. Maybe cheshire or commons-codec or commons-io or org.clojure/tools.reader were causing it, because now I :exclude them from deps that request older versions and my code suddenly compiles...
2018:09:28 17:56:41 lilactown I have a map of var-quoted functions that I'm using as a lacinia resolver map. After add fspecs for all of them and turning on instrumentation, I'm getting this error:
com.fasterxml.jackson.core.JsonGenerationException: Cannot JSON encode object of class: class clojure.spec.alpha$regex_spec_impl$reify__2436:
2018:09:30 12:11:15 misha "it" = "same keys order in map"
2018:09:30 13:59:40 bbrinck @misha I may be misunderstanding your question, but the order of a map is consistent, (i.e. for the same map m, (first m) will always return the same thing, since maps are seqable?).
2018:09:30 14:41:24 jumblemuddle Is the only difference between using spec/valid? in the pre/post map of a function and spec/fdef that spec/fdef can be specified elsewhere?
2018:09:30 14:56:44 taylor you have to instrumentfdef’d functions, and that won’t check the function return value like :post does
2018:09:30 14:57:18 taylor pre/post checks will run by default, but instrumentation is opt-in
2018:09:30 14:59:32 jumblemuddle I'm noticing that instrument-all is no longer available. Is it common to just instrument a single function while testing it nowadays?
2018:09:30 15:00:15 taylor calling instrument with no args will instrument everything that’s been loaded
2018:09:30 15:00:48 jumblemuddle Oh, nice 🙂
2018:09:30 15:00:50 jumblemuddle Thanks
2018:09:30 15:01:12 jumblemuddle fdef doesn't verify the output from the :ret parameter though?
2018:09:30 15:01:20 jumblemuddle At least while using instrument?
2018:09:30 15:01:45 taylor right, instrumented functions will only assert the fdef :args spec
2018:09:30 15:02:04 jumblemuddle Huh, is there anything that actually uses the :ret then?
2018:09:30 15:02:06 taylor there’s a lib called Orchestra with another version of instrument that adds return value checking
2018:09:30 15:02:15 jumblemuddle 👍
2018:09:30 15:02:34 jumblemuddle Thanks
2018:09:30 15:02:47 taylor yeah, if you check an fdef function it uses the :ret spec (and :fn if it exists)
2018:09:30 15:03:27 taylor check towards the bottom of the clojure.spec guide for check
2018:09:30 16:55:32 andy.fingerhut @bbrinck I do not know your use case exactly, but as long as you are only relying on that for the same identical map m, not two different maps m1 and m2 where (= m1 m2). If you try to rely on (= (first m1) (first m2)) you will often be disappointed. But yes, you can rely on (= (seq m) (seq m)) being true.
2018:09:30 17:02:04 bbrinck @andy.fingerhut good point! Yes, this is for the identical map - the “in” path returned by spec is only valid for the data you originally check, NOT other data that happens to be equal to the original data.
2018:10:01 12:55:11 jumblemuddle During development, does it makes sense to use fdef and instrument for all my functions or sprinkle my functions with assert and old enable assertions during development?
Perhaps a combination of the two?
2018:10:01 12:59:21 jumblemuddle I guess the benefit of fdef is that it can be in a separate namespace; is that recommended?
2018:10:01 13:09:38 taylor I think this depends on personal preference and whether you’re writing something intended to run on Clojure 1.8. I tend to put fdef next to the function definitions, but keep other specs in spec-specific namespaces
2018:10:01 14:50:06 bbrinck You may already know this ,but it’s perfectly fine to put the fdef definition above the function definition. I find this much more readable than putting it below the function.
2018:10:01 14:50:26 bbrinck (sometimes people don’t realize this is a valid way to organize their specs)
2018:10:01 13:04:21 jumblemuddle It seems like that would defeat some of the namespace'd keyword designs of spec though.
2018:10:01 15:08:19 mike_ananev @alexmiller hi! We found strange flawing bug for clojure.spec lib in multithreaded environment. We use latest clojure.spec versions ("0.2.176" "0.2.44") for request validation. Every http request we translate to clojure map, then we call (s/valid? spec input-request-map) for input validation. We have 300 000 sample requests (saved in edn file) for testing (constant requests). If we use one thread on JMeter to send request then we always have result true for (s/valid? spec input-request-map) for every 300 000 constant requests. if we use 2 threads or more on JMeter to send requests then sometimes we have false result. If we save false map to a file and then try to validate it (s/valid? spec input-request-map) then we see that it is always true. If we validate twice in multithreaded env (or (s/valid? spec input-request-map) (s/valid? spec input-request-map) ) then it works (but logs tell us that one of s-exps inside or sometimes false). All map is standard clojure maps (immutable). The code (or (s/valid? spec input-request-map) (s/valid? spec input-request-map) ) works for us as workaround but we can't watch on it without crying. It's Clojure, but we met undefined behaviour like in mutable world.
2018:10:01 16:34:31 alexmiller Sounds unexpected to me. :) Instead of using valid?, could you try using s/explain so you’d get a print to the console when it goes wrong? How is JMeter getting the data to send? Is it possible you have two threads reading from the same reader and thus getting something invalid? If you could isolate this down to something reproducible, would be great to have a jira for it.
2018:10:01 15:38:58 mike_ananev Any advice?
2018:10:01 17:04:58 andy.fingerhut @djtango If you are using Leiningen, the command lein deps :tree can be useful. With tools.deps there is clj -Stree. Maybe you were already aware of those and looking for something more, though.
2018:10:01 17:05:33 djtango thanks for this - wasn't really aware of using lein deps or using it in this way
2018:10:01 17:05:45 djtango at this point any and all suggestions are welcome!
2018:10:01 17:31:19 seancorfield "lein pedantic" is something to look into as it will flag the conflicts for you. Boot has a similar feature on its show task.
2018:10:01 17:31:29 seancorfield I'm not sure if there's an equivalent yet for clj...
2018:10:02 09:50:42 djtango thank you!
2018:10:01 19:14:27 jumblemuddle So, I added (clojure.spec.test.alpha/instrument) to the bottom of my core.cljs file, however I have a namespace alpha which defines a symbol (def global (beta/example ...)). The problem being that core depends on alpha which uses beta/example prior to it being instrumented. Do I have to add the instrument to every namespace to avoid this?
2018:10:01 19:32:36 taylor if you want that call to beta/example to get instrumented and you want to keep def global (instead of making it a function or wrapping it in a delay) then yeah I think you'd have to instrument it separately in this case
2018:10:01 19:34:34 jumblemuddle Makes sense 👍
In this case, it was easy enough to not define the global, though I could see scenarios where that's not really a solution.
2018:10:01 19:34:36 jumblemuddle Thanks
2018:10:01 19:15:30 jumblemuddle How do I spec and instrument functions that will be used during initialization?
2018:10:01 19:39:06 jaihindhreddy This is where I am so far.
(s/def :iso/country-code (s/and s/string? #(= 3 (count %))))
2018:10:01 19:40:41 taylor I'd probably scrape the values, put them in a set, and use that set as the spec
2018:10:01 19:41:49 bbrinck If you can get a big set of valid sample data and convert it into valid EDN, you could try using https://github.com/stathissideris/spec-provider to infer the spec
2018:10:01 19:43:15 jrychter @jaihindh.reddy in these kinds of cases I enumerate all the possibilities in a set and use that set in the spec.
2018:10:01 19:43:58 jaihindhreddy @taylor ISO charges money for downloads of data. But it is freely available on their website. Scraping that would be awkward to say the least.
2018:10:01 19:44:08 jrychter e.g. (s/and string? countries/all-country-codes) where all-country-codes is a set.
2018:10:01 19:44:14 taylor in your example all the ISO codes are on the Wiki page
2018:10:01 19:44:46 jrychter …which also makes it somewhat maintainable in the longer term.
2018:10:01 19:45:03 taylor are you concerned with keeping the list fresh as time passes?
2018:10:01 19:45:15 jaihindhreddy Ideally I would love to do something like (into #{} (slurp url-for-all-country-codes))
2018:10:01 19:45:28 taylor yeah, that's a separate problem from clojure.spec
2018:10:01 19:45:35 jaihindhreddy Country codes should be relatively stable. Dont mind manually updating it
2018:10:01 20:00:51 jrychter Well, manually is fine, but I would plan on doing it every once in a while. The list of countries is actually surprisingly unstable.
2018:10:01 21:55:56 jaihindhreddy Can I spec protocols and multimethods?
2018:10:01 22:12:35 seancorfield I think you're OK with multimethods tho' (I saw a CLJS ticket that indicated s/instrument was fixed to work with multimethods, so I assume it works in CLJ too).
2018:10:01 22:13:19 jaihindhreddy On that note, when should I use one or the other?
2018:10:01 22:14:08 jaihindhreddy Currently I think protocols should be the default, and multimethod if you think you need the arbitrary dispatch.
2018:10:03 13:54:18 fabrao use as replacement of unit test?
2018:10:03 14:02:56 taylor IMO they should supplement/augment tests, but not totally replace them
2018:10:03 13:56:25 fabrao how do you use in your workflow daily?
2018:10:03 13:56:55 taylor hey @fabrao, an example of how you might use it: have some defns you want to spec, write some specs/`fdef`s for those, then in a test namespace you might use check as part of a test
2018:10:03 13:57:28 taylor and the specs/`fdef`s can be in the same file as the functions or separate, up to you
2018:10:03 13:57:58 fabrao @taylor by the way, was you that wrote the artice?
2018:10:03 13:59:03 taylor yes, thanks for reading 🙂
2018:10:03 14:01:07 fabrao It was very clear about the propose of spec
2018:10:03 14:01:28 fabrao But you use it in your testing workflow?
2018:10:03 14:01:48 taylor yes, I tend to use instrument more than check
2018:10:03 14:02:58 fabrao but, have you already keep it on in production scenario?
2018:10:03 14:03:29 fabrao for example to validade an csv file?
2018:10:03 14:03:44 taylor no, instrument has a performance penalty, and anywhere I need to use spec "all the time" then I use the explicit conform/`valid?` kind of functions
2018:10:03 14:04:03 taylor s/explain-data etc.
2018:10:03 14:04:57 fabrao how do you use it in Exception scenarios?
2018:10:03 14:05:33 taylor like if I wanted to throw an exception because of invalid input? Probably use ex-info and maybe put some of the s/explain-data into the info map
2018:10:03 14:05:50 taylor you should also look into the expound library for spec
2018:10:03 14:06:31 taylor that can help beautify spec's description of invalid inputs
2018:10:03 14:07:12 fabrao I got it, but you mix real code with spec codes?
2018:10:03 14:07:57 taylor in some cases yeah I might call s/conform or s/explain-data or s/assert from my "normal" code
2018:10:03 14:08:13 taylor for always-on types of checks (asserts aside)
2018:10:03 14:08:47 fabrao in case of using valid?, you use it inside your functions?
2018:10:03 14:09:24 taylor probably not as often as the others, because I'm usually also interested in how/why the input was invalid
2018:10:03 14:10:25 fabrao Yes, I don´t want to polute my code with spec[ing] stuffs
2018:10:03 14:10:48 fabrao I want to be non intrusive checking
2018:10:03 14:11:09 taylor yeah, overall I don't have many places in my projects where I'm explicitly interacting with spec in my app code/logic
2018:10:03 14:12:59 fabrao yes, thank you about your help, I was afraid beeing specing everthing 🙂
2018:10:03 14:13:26 taylor no problem, one great thing about spec is you can use it as little as you like 🙂
2018:10:03 14:15:15 fabrao Yes, I saw many videos about it, but only shows something using valid? and nothing about the use like in your article
2018:10:03 14:20:04 fabrao in this way, it´s better using typed-clojure than that 🙂
2018:10:03 14:23:25 taylor I think that's fine (and it's cool that it's even possible) if that's your workflow, although personally I would probably just use regular defn and fdefs
2018:10:03 14:24:02 taylor it's really up to how you want to use spec and structure your code :man-shrugging: try some different approaches and see what you like
2018:10:03 14:31:54 fabrao Yes, I understood, thanks for your time. I´d appreciate all the subjects of your blog, all subject that I sometimes worked with
2018:10:03 14:33:00 fabrao I´m using Instaparse in production already
2018:10:03 14:33:38 fabrao that you can build your own language
2018:10:03 14:34:00 taylor I've used Instaparse in a production project too, it's great
2018:10:03 13:58:26 taylor you might choose to enable instrumentation for particular tests by calling instrument, or maybe in a dev profile so everything is instrumented at dev time
2018:10:03 17:16:30 seancorfield @fabrao The other thing to bear in mind with spec is that there are two related but separate types of spec: data specs and function specs. At work, we rely heavily on the former in production code, where we call s/conform (mostly) and s/valid? on those specs and some data. We use function specs to support testing, either with st/instrument while we're developing or as part of a "unit test", or with st/check mostly while we're developing but also in small, limited tests (generative testing can take a while so it's not really designed as part of your short, fast "unit test" cycle).
2018:10:03 17:17:21 seancorfield We've been using spec in production pretty much since the first alpha dropped, back in the early 1.9 days. We love it!
2018:10:03 17:18:32 taylor there's a Clojure 1.8 backport of clojure.spec I've used in production haha
2018:10:03 17:34:05 seancorfield That appeared long after we were already on 1.9 and using spec in production tho' 🙂
2018:10:03 17:34:18 seancorfield We're currently on 1.10 Alpha 8 in production.
2018:10:03 17:26:56 fabrao @seancorfield I saw that we can use s/conform to contruct maps from data in more easy way
2018:10:03 17:28:08 fabrao in that video explain lots of stuffs
2018:10:03 17:32:25 seancorfield @fabrao Be careful about coercions and conformers in spec tho' -- consider those "very advanced" usage until you're comfortable with the rest of it in your workflow.
2018:10:03 17:33:20 seancorfield If you bake a coercion into your spec, you are "forcing" that coercion on all clients of the spec -- and it may have consequences for generators / generative testing.
2018:10:03 17:33:34 seancorfield But, yeah, overall spec is awesome!
2018:10:03 22:07:29 Scot Where's my error? What am I doing wrong?
2018:10:03 22:08:33 Scot I'm sure I am doing something stupid, but I can't find any answers
2018:10:03 22:08:59 noisesmith to verify an fdef I think you need to use instrument to explicitly turn on verification
2018:10:03 22:09:39 noisesmith also beware of instrumenting functions that take function arguments - the spec is checked by passing various generated data to the function that was passed in
2018:10:03 22:10:18 Scot So the only way to get runtime verification is via asserts?
2018:10:03 22:10:45 noisesmith instrument turns on the validation of the function's spec, it's something you have to explicitly ask for
2018:10:03 22:11:17 Scot Thanks
2018:10:03 22:13:43 noisesmith if you want to ensure that a specific arg is always checked, you can use s/valid? as a predicate, or s/conform if you want an error for non-matching data
2018:10:04 16:48:05 noisesmith @misha thanks for the clarification
2018:10:04 16:48:59 noisesmith so instrument only validates the arglist and not the return value
2018:10:04 16:52:21 alexmiller Yes- it instruments functions to ensure you are invoking them correctly
2018:10:04 16:54:43 triss If I have a lot of functions all with the same spec how can I share it between them? I’m guessing I can use an fspec with fdef somehow? I really want these specs to show up in the doc strings
2018:10:04 16:55:55 alexmiller Sure, name the common spec and use the name
2018:10:04 16:56:27 alexmiller fdef is really a wrapper around fspec then def
2018:10:04 16:57:34 alexmiller So you can (s/def foo ::a) where ::a is an s/fspec
2018:10:04 16:57:59 alexmiller There were some bugs around this but I think those are all fixed now
2018:10:04 17:48:48 fabrao what is the right use? [clojure.spec.alpha :as s] or [clojure.spec :as s]. I tried [clojure.spec :as s] but it seems no lib in this path
2018:10:04 17:48:55 fabrao ?
2018:10:04 17:51:35 seancorfield Spec is still alpha right now.
2018:10:04 17:52:01 favila always the namespace with "alpha". Clojure 1.9-alpha had a brief period where spec was in clojure core (not a separate lib). That is why you may see non-alpha namespaces sometimes.
2018:10:04 17:52:05 seancorfield All of the Spec namespaces end in .alpha -- until it stops being alpha.
2018:10:04 17:52:34 seancorfield @favila Ah, yeah, that must have been a fairly brief period 🙂
2018:10:04 17:53:23 fabrao That´s because I was confuse
2018:10:04 17:54:09 the2bears The great thing about REPL development is you can try these things very quickly and see which one works.
2018:10:04 17:54:24 seancorfield There's a comment on that video, dated a year ago, that says "Good introduction! But something changed in the meantime: starting from 1.9.0-alpha16 you have to include clojure.spec.alpha" @fabrao
2018:10:04 17:54:42 the2bears Though Lein setup with the project.clj file can slow that down a bit.
2018:10:04 17:55:11 seancorfield @the2bears Aye, I always have a REPL open and I try stuff in it all the time.
2018:10:04 17:55:35 favila https://clojure.org/community/devchangelog clojure 1.9.0-alpha16 is when it split. So from May 24 2016 to april 26, 2017 "clojure.spec" was the correct namespace
2018:10:04 17:55:47 fabrao I left using lein for boot because of @seancorfield
2018:10:04 17:56:03 seancorfield Longer than I realized -- thanks @favila for that detective work!
2018:10:04 17:56:29 seancorfield Boot REPL is nice because you can easily add new dependencies without restarting the REPL!
2018:10:04 17:56:53 fabrao that´s the main part I changed
2018:10:04 17:57:29 fabrao C-c C-e in set-env! and that´s it
2018:10:04 17:57:46 the2bears @seancorfield my problem is I lose track of my REPL's state from time to time 🙂 . But I have 2 or 3 open at any one time, usually, main project then libraries used in it.
2018:10:04 17:58:43 fabrao I´ll do cider-connect and start it from command line
2018:10:04 17:59:53 fabrao I tried to use Proto-REPL with Atom but I can´t leave Emacs anymore
2018:10:04 19:17:42 fabrao In this case, I got Assert failed. Is there any way to custom error message?
(defn stringer-bell
"Prints a string and rings bell."
[s]
{:pre [(s/valid? (s/nilable string?) s)]}
(println s "\007"))
2018:10:04 19:21:42 fabrao but is too much code, don´t you think?
2018:10:04 19:22:39 taylor I think that reduce could be replaced with clojure.string/join maybe
2018:10:04 19:23:48 taylor also, if you think you need this type of check a lot, you could easily define a function that takes a spec, an input, and maybe an error message or explain-data-to-string function; and then wherever you call it should be pretty terse?
2018:10:04 19:24:33 taylor also check out https://github.com/bhb/expound if you haven't already, and I think there's a similar lib that's more geared towards user-facing error messages
2018:10:04 20:36:25 misha and (reduce #(str %1 %2 " ") "" campos) looks like (clojure.string/join " " campos)
2018:10:04 20:37:05 misha suddenly, not too much codeopieop
2018:10:04 23:38:26 ag if you are writing a s/fdef foo but want to place it in a separate ns (not the same where foo is), should you require the ns where the function is in ns where the spec is? But what if you need to use specs say for validation (in the ns where s/fdef is implemented) how do you avoid circular dependency?
2018:10:04 23:40:20 ag or should you move higher order specs to a separate ns, but keep s/fdef foo where foo is implemented?
2018:10:04 23:41:57 noisesmith isn't some of this simplified by the fact that the spec is looked up by a keyword? you can use a namespaced keyword before the namespace it nominally refers to exists
2018:10:04 23:42:23 noisesmith by the time the spec is actually checked, you need the ns that defined the spec for that keyword to be loaded, of course
2018:10:05 02:04:32 alexmiller you don’t have to require the namespace
2018:10:05 02:05:48 alexmiller fdef creates an entry in the spec registry keyed by the namespaced symbol, but you don’t need to load that namespace at that point
2018:10:05 19:59:19 aisamu Wrapping a cat with an and is breaking my spec in interesting ways.
What am I missing?
2018:10:05 20:02:40 taylor the s/? isn't "flattening" the regex spec (`s/cat`) because it's wrapped in s/and. I think those last tests would conform [[1 2]]
2018:10:05 20:03:26 taylor it should also conform []
2018:10:05 20:04:29 taylor (s/? ::ordered-pair) is becoming "match a sequence of zero-or-one ::ordered-pairs"
2018:10:05 20:05:05 taylor if you wanted a something-or-nothing spec you could try (s/nilable ::ordered-pair) too but I'm not sure of your use case
2018:10:05 20:09:34 aisamu Yup, both cases conform correctly! I'm mostly trying to understand the non-flattening, though. Is there an alternative construct to use the ordering function without using s/and?
2018:10:05 20:19:35 taylor hmm what would multiple pair inputs look like for you? one big sequence like [1 2 3 4] or a sequence of pair seqs like [[1 2] [3 4]]
2018:10:05 20:28:22 alexmiller s/and converts a regex spec to a non-regex. s/& is a regex version
2018:10:05 20:28:53 alexmiller will still continue operating at the same “level” of sequence
2018:10:05 20:30:03 taylor ah yeah, you can just replace s/and with s/& in ::ordered-pair if you want to conform inputs like [1 2 3 4]
2018:10:05 20:53:14 aisamu Oh,facepalm
Not only it makes perfect sense, but it's pretty obvious in retrospect!
Thanks @U064X3EF3 and @U3DAE8HMG!
2018:10:08 00:49:37 kingcode What is the fastest way to tell whether one vector is a subsequence of another?
2018:10:08 12:12:08 roklenarcic Probably the same algo than the one for substring search.
2018:10:08 12:19:33 roklenarcic Knuth-Morris-Pratt or Boyer-Moore
2018:10:08 00:50:06 kingcode sorry..wrong channel (I was thinking of using specs’ coll regex)
2018:10:08 05:18:50 tianshu Hi, how can I know if there is a spec on a given keyword or not? seems s/spec? only detect spec object.
2018:10:08 05:31:29 seancorfield (s/get-spec ::k) will return nil if no spec is registered (else will return the spec itself).
2018:10:08 06:00:38 tianshu @seancorfield thanks!
2018:10:09 16:41:30 jrychter I wish there was a book I could buy, titled "Data Modeling with clojure.spec".
2018:10:09 16:44:52 jrychter Here's an example of a practical question. Should I do (a) or (b)?
2018:10:09 16:45:56 alexmiller You should do whichever one matches your data
2018:10:09 16:46:19 jrychter Hence my wish for the book 🙂
2018:10:09 18:05:34 mattly personally I would define the spec for the id in the same ns as the id
2018:10:09 18:06:12 mattly unless the testing.spec ns is the only place you plan to refer to it
2018:10:09 18:19:39 jrychter :: is a spec for an UUID in a certain form, this gets used all over the place. Question is, do I alias it locally using ::id? In other words, who "owns" the ::id definition?
2018:10:09 18:23:00 jrychter And Alex's Zen Master answer is actually spot on. My current thinking is that I should use local ::id, since this is really a property of the data that I'm defining locally, and only coincidentally an UUID whose properties are defined elsewhere.
2018:10:09 18:23:01 alexmiller I think about it as two levels - it’s useful to have specs that capture common type/shapes/formats domain things - phone numbers, invoice numbers, etc. then there are entities that collect attributes and those may have their own names. Like you have a :foo.domain/phone and then you have :foo.company/phone
2018:10:09 18:23:19 alexmiller (and the latter would just be an alias for the former)
2018:10:09 18:24:06 alexmiller that adds weight (and there are some bugs around overriding generators for aliased specs) so it may or may not be worth it
2018:10:09 18:24:49 alexmiller phone number is probably something with extra constraints, but if you have something adequately represented as int? then I would not pull it out as a domain type as that doesn’t seem worth it
2018:10:09 18:25:15 alexmiller some day we’ll consider doing a 2nd edition of Clojure Applied and think about some of this stuff :)
2018:10:10 13:57:29 orestis I’d love a 2nd edition of Clojure Applied — or perhaps a 2nd volume? 😄
2018:10:10 14:02:22 alexmiller based on my time availability, I don’t think it’s happening anytime soon
2018:10:10 14:04:39 orestis Understandable, of course. Thanks for the first edition in any case!
2018:10:09 18:25:42 jrychter Right. This is similar to what I've been thinking, too.
2018:10:09 18:28:36 jrychter A book would be very welcome. I would really like to learn about how people approach modeling data, preferably from people who actually build real systems. Spec seems very flexible, but I found that if you deviate from a certain path, you'll be swimming against the current (for example, if you insist on using unqualified keys).
2018:10:09 18:30:27 jrychter BTW, I solved my coercion problems for the time being by judiciously removing every s/conform from my code and writing a tiny piece of code that maintains registries of coercions (multiple registries, as it turns out: one for converting to the domain model, and one for converting to db form). This seems to work very well and apart from limitations of unqualified keys in specs solves the problem for me.
2018:10:10 16:02:21 ScottStackelhouse Hi all, I don’t slack much. I have a case where I model data as a map, some keys required, some not. I already use one key as a “type” or “kind” indicator. The problem I have is I have a set of keys where all are optional, but at least one of them must be present.
2018:10:10 16:02:34 ScottStackelhouse I have not had luck figuring out how to spec that
2018:10:10 16:02:49 ScottStackelhouse Any thoughts or links to point me at?
2018:10:10 16:04:03 alexmiller use s/and to combine a predicate that checks that condition
2018:10:10 16:05:54 ScottStackelhouse Mostly the problems I run ino with spec’ing like that is it seems to imply/expect more hierarchy than o have
2018:10:10 16:06:12 ScottStackelhouse I’m sure it’s user error, but examples are hard to come by
2018:10:10 16:06:39 piotr-yuxuan Sorry for disrupting your chat. I’m wondering how to programmatically register a spec.
😞 any tiny help from you would be greatly appreciated! 💪
2018:10:10 16:16:58 piotr-yuxuan I was wondering how I could write a small library to generate specs from avro schema, so I need to call spec/def in a let and I only know the spec name at runtime.
2018:10:10 16:30:53 ScottStackelhouse @piotr2b have you looked at the macro expansions of your macro and of s/def? I did a quick try and it didn’t come out how I expected.
2018:10:10 16:31:24 alexmiller I’ve done some stuff like this elsewhere but can’t put my finger on it right now and I need to log off, sorry
2018:10:10 16:36:16 ScottStackelhouse I put the macro expansion of the s/def form into the let in place of the macro call, ie (s/def-impl ‘spec-name ‘int? int?) and it fails an assertion that “k” is a keyword...
2018:10:10 16:37:36 ScottStackelhouse Nomenclature fails me but it is like spec-name in your macro needs a double deref
2018:10:10 16:53:33 ScottStackelhouse @piotr2b If you take the expansion of s/def, and use that in your draft-make-spec macro, it works
2018:10:10 16:53:44 piotr-yuxuan wow, wait!
2018:10:10 16:53:54 piotr-yuxuan Could you show me that? 😄
2018:10:10 16:54:03 piotr-yuxuan It’s funny because we’ve swapped problems
2018:10:10 16:54:09 piotr-yuxuan I was working on solving yours
2018:10:10 16:54:10 ScottStackelhouse Yeah, on my phone tho
2018:10:10 16:54:17 ScottStackelhouse Heh
2018:10:10 16:54:29 piotr-yuxuan Here is what I’ve got so far:
2018:10:10 16:54:36 piotr-yuxuan Does it suit your need?
2018:10:10 16:55:23 ScottStackelhouse I will have to check
2018:10:10 16:55:27 piotr-yuxuan Basically I’m a bit new with macros, so if you could write down the code in addition to your previous explanations, that would be wonderful 🙂
2018:10:10 16:55:53 ScottStackelhouse Let me switch to something with a keyboard
2018:10:10 16:55:58 piotr-yuxuan ^^
2018:10:10 16:58:10 piotr-yuxuan @alexmiller how sad! if you could find it back it would be awesome. Anyway, thanks for everything you do for this amazing language 🙂
2018:10:10 16:59:33 piotr-yuxuan @scottstack, I fixed a typo, it’s better with :opt-un
2018:10:10 17:00:56 piotr-yuxuan And the last test case:
2018:10:10 17:03:46 ScottStackelhouse I am very good with macros either. I depend heavily on cider's macroexpansion
2018:10:10 17:05:21 ScottStackelhouse S/def would expand to (s/def-impl 'sname 'int? int?)
2018:10:10 17:06:02 ScottStackelhouse And that would fail an assertion that the first arg be a keyword
2018:10:10 17:06:20 ScottStackelhouse Oh
2018:10:10 17:06:37 ScottStackelhouse I see I goofed up, maybe it still doesn't work
2018:10:10 17:07:55 ScottStackelhouse Nevermind, I just confused myself momentarily... I think it is ok
2018:10:10 19:00:13 firstclassfunc Afternoon, is anyone aware of any projects to transform JSON Schema into Clojure Spec?
2018:10:10 19:22:46 lilactown I don't know of any projects for doing JSON Schema => clojure.spec
2018:10:10 19:23:01 lilactown spec-tools can go the other way: clojure.spec => JSON Schema
2018:10:10 19:24:41 firstclassfunc @lilactown Yea tks.. Do people think that would be valuable? There are lots of standards defined in terms of JSON Schema and would be beneficial to auto-generate specs so they can be consumes without the tedious nature of crafting data structures from scratch
2018:10:10 19:53:00 lilactown I think it could be pretty useful to help bootstrap some of the commonly needed specs, yeah 😄
2018:10:10 21:57:08 devn This may be a test.check question more than a spec question, but here goes:
I have a map named foo:
{:a 1 :b nil :c 32} If a is present, then b should be nil, and c should be a pos-int?.
{:a nil :b true :c nil} When a is nil, c must also be nil, and b must hold a value.
I want to be able to generate examples of both types of maps.
2018:10:10 21:57:37 devn Perhaps I should be tagging them as :type :a and :type :b and using a multi-spec?
2018:10:10 22:00:20 gfredericks As a pure t.c question it's easy - use gen/one-of; not sure about making that more spectomatic
2018:10:10 22:00:47 gfredericks multi-spec sounds plausible but I am not a dentist
2018:10:10 22:07:32 devn The way I explained it above is kind of crappy:
example:
(let [a-thing {:a 1 :b nil}
(when (and (:a a-thing) (not (:b a-thing))) (println "it's a foo")
(when (and (not (:a a-thing)) (:b a thing)) (println "it's a bar")))
I think I need some such-that magic maybe?
2018:10:10 22:10:25 devn So what I want is if I then ran (gen/sample (s/gen ::thing)), I'd never get back records which have both :a and :b populated, only one or the other.
2018:10:10 22:12:19 gfredericks Have you tried writing a map spec for each case and using s/or?
2018:10:10 22:13:34 devn no but that makes perfect sense
2018:10:10 22:13:38 devn thanks @gfredericks
2018:10:10 22:19:13 devn just to be clear, what I think you're suggesting is this:
2018:10:10 22:29:27 devn though you may be on to something with with-gen
2018:10:10 22:30:01 devn it would be nice to attach the frequency to the spec
2018:10:10 22:30:10 bbrinck untested, but then you can do something like (s/def ::either-thing (s/with-gen (s/or ::thing-a ::thing-b) #(gen/frequency [[2 (s/gen ::thing-a)] [2 (s/gen ::thing-b)]]) ))
2018:10:10 22:31:32 devn that doesn't seem to do it
2018:10:10 22:32:37 bbrinck hm, probably a bug in my code, but in general with-gen should override the default generator. remember the function needs to return a generator
2018:10:10 22:33:02 bbrinck IOW, the second arg can’t be a generator
2018:10:10 22:34:11 devn yes, which is indeed what your code does there
2018:10:10 22:34:48 devn maybe gary's previous suggestion to use one-of is worth trying here
2018:10:10 22:35:13 bbrinck what doesn’t work with frequencies?
2018:10:10 22:36:23 devn i tried changing the frequency from 2 to 0 for one of them, and it doesn't vary the samples
2018:10:10 22:36:29 bbrinck ah
2018:10:10 22:36:35 devn they all are ::thing-a-like things
2018:10:10 22:36:51 devn whereas the thing I posted above does work
2018:10:10 22:37:23 bbrinck well, if you put 0 for thing-b, then everything should be thing-a, yes?
2018:10:10 22:37:36 devn yes, but i tried it both ways, and the same result
2018:10:10 22:37:38 bbrinck ah
2018:10:10 22:39:41 devn tested again to make sure im not crazy
2018:10:11 16:35:24 domkm Has anyone written a generator for keys that always chooses to generate optional entries until *recursion-limit* is reached?
2018:10:11 18:09:06 domkm It seems like this may not be possible currently because whenever a generator-creating function is allowed (`with-gen`, (keys :gen (fn [] ...))) it is takes 0 args while the default generator-creating functions take 3 args (`[overrides path rmap]`). Why do custom generators not have access to the current length of recursion?
2018:10:11 21:36:22 basti I’ve got a pretty specific question regarding clojure spec-tools’s dataspecs -
i’ve got those two data specs, how do I “or” them together?
2018:10:12 15:13:54 basti that awesome thanks for your help --appreciated!
2018:10:12 12:14:23 Charles Fourdrignier After solving Infix Calculator on 4Clojure (http://www.4clojure.com/problem/135), I would like to write some spec for the args.
2018:10:12 12:19:01 taylor I suppose you could use number? instead of int?
2018:10:12 12:22:18 Charles Fourdrignier Of course !
2018:10:14 10:40:14 vemv Wondering if there's a precise definition of :in and :at within the context of clojure.spec failures:
...
in: [:lines 1]
at: [:lines]
...
2018:10:14 10:42:20 vemv (this comes from CIDER so perhaps an extra key is being added)
2018:10:14 17:05:51 domkm Is there a recommended way of debugging built-in spec generators? I have a large graph spec in which all leaf nodes are clojure.core functions like any?, string?, boolean?, etc. and all collections are either sequential collections or maps with only optional keys, though some keys are recursive. It halts when generating data, eventually erroring (I think it runs out of memory).
2018:10:14 18:27:12 misha > though some keys are recursive
2018:10:14 18:32:08 misha however, s/*recursion-limit* requires recursive specs to have a non-recursive branch available to be able to stop at the limit.
but you will probably get StackOverflow much earlier, if you "explicitly" omit nonrecursive branch:
2018:10:14 18:36:05 domkm @misha Hmm, I hadn't considered that it would halt before reaching the s/*recursion-limit*. I assumed that since all keys are optional it would choose a non-recursive option. I'll try messing around with s/*recursion-limit* and see what happens. Thanks
2018:10:14 18:37:17 misha you might get surprised how often limit=4 runs out of memory
2018:10:14 18:38:46 misha try limit=1 and see how deep of a structure s/exercise returns, to at least somewhat calibrate your intuition
2018:10:14 21:22:26 alza Hi there, I have a small project to transform one xml format into another (call them fmt-a and fmt-b). I thought Clojure Spec might be useful to define the shape of the data at each stage of the transformation and check the transformation works correctly for expected inputs. Then I found spec-tools and it's transformers, so I thought I'd investigate that. Although I noted that although it supports JSON it doesn't seem to support XML yet.
Anyway this was going to be my approach with spec-tools:
1. parse fmt-a-xml from xml file
2. transform fmt-a-xml-data (i.e. parsed {:tag :attributes :content structure}) to fmt-a (i.e. {:fmt-a-key some-val})
3. transform fmt-a to fmt-b (i.e. {:fmt-b-key some-val})
4. transform fmt-b to fmt-b-xml-data (i.e. {:tag :attributes :content structure})
5. format fmt-b-xml-data to xml file
Does this sound sensible? I'm not sure how to achieve all of the above with spec-tools, but I was going to start experimenting with step 3, the core data transformation.
note: there's no xml schema available for fmt-a or fmt-b, if that matters.
2018:10:16 05:59:27 ikitommi Of coercion: tested the two approaches: conforming-based (spec-tools) and form-parsing based (spec-coerce). Both have big issues. This just can’t be resolved with the current spec architecture. One CLJ-2251 please 🙂
2018:10:16 06:04:34 ikitommi will most likely merge the two approaches into spec-tools, so the st/decode will work without wrapping of specs for many simple forms, fallbacking to conforming-based approach to support all specs (regex included)
2018:10:16 17:57:40 taylor could you use a spec for the :type key like (s/def ::type #{::person})
2018:10:16 16:41:38 hmaurer or is there a better/neater way?
2018:10:16 17:03:10 jrychter Depends on what you want to do — sometimes you want a multi-spec, where you dispatch on :type.
2018:10:16 17:04:34 jrychter As a side note, I found that prefer to use boolean ::person? keys rather than a general :type. I'm converting my old code as I go.
2018:10:16 17:25:30 hmaurer @jrychter oh, why the boolean approach?
2018:10:16 17:26:07 hmaurer and under that approach, how would you suggest I handle a hierarchy like this:
https://puu.sh/BM1sE/7aaf4c8e7b.png
?
2018:10:16 17:26:21 hmaurer each of these represent maps of a certain “type” with certain properties
2018:10:16 17:26:52 jrychter Less complexity, basically. And I don't know, but I suspect that using a single data structure for all of these might not be the best approach.
2018:10:16 17:27:46 hmaurer how so?
2018:10:16 17:44:16 Nolan hey everyone, i may be missing something obvious, but is there a concise way to use spec to perform what is essentially a select-keys according to a spec? e.g.
(def composite-thing { ... })
(s/def ::some-component ...)
(s/select-keys ::some-component composite-thing)
;; => map with keys of composite-thing relevant to ::some-component
2018:10:16 17:45:49 Nolan or maybe read the relevant keys from a spec’s definition at runtime?
2018:10:16 17:53:58 taylor
(s/def ::my-map
(s/keys :req-un [::id]
:opt-un [::parent-id ::children]))
(->> (s/get-spec ::my-map)
(s/describe) ;; get abbrev. form of original spec
(rest) ;; drop `keys` symbol
(apply hash-map)) ;; put kwargs into map
=> {:req-un [:sandbox/id]
:opt-un [:sandbox/parent-id :sandbox/children]}
2018:10:16 17:59:41 seancorfield I don't think you need (s/get-spec ::my-map) there, you can just do (s/describe ::my-map) (since s/describe calls s/form).
2018:10:16 18:06:30 ikitommi remember to walk over special and and ors.
2018:10:16 19:33:36 jrychter This is something I'd really like to see. I know that Rich said that spec is not about limiting what you can do, but there are legitimate use cases where you want to enforce the set of keys and limit it only to what is in your spec (think for example API endpoints).
2018:10:16 19:38:33 seancorfield @jrychter It's easy enough (in most cases) to pull the set of possible keys out of the form for the spec and use that -- treating the spec as the "source of truth" and deriving key sets from it.
2018:10:16 21:06:55 triss I’m building my first really big interconnected system and I have to admit immutablity is driving me crazy!
2018:10:16 21:07:24 triss I just can’t used to passing paths around for my ‘objects’ all the time.
2018:10:16 21:07:41 triss What tare the other functional approaches to this?
2018:10:16 21:08:08 triss I’m really missing OO/imperative systems being happy re: passing around references to objects all the time.
2018:10:16 21:09:32 the2bears I don't miss that at all 😅
2018:10:16 21:10:10 triss I wish I didn’t!
2018:10:16 21:10:41 triss I’m passing around the paths in to a nested map all the time…
2018:10:16 21:14:48 the2bears Not sure what you're trying to do, sounds like you have a big nested state, and what about the paths you're passing around?
2018:10:16 21:15:40 triss I’m porting an old AI project to Clojure for now…. the whole system seems thoroughly object orientated. Objects sending messages to others all over the place.
2018:10:16 21:16:23 triss I’m pretty sure Core-async would make things a bit easier since I could just pass the chanell around all the time.
2018:10:16 21:16:42 triss But obviously this is a very different solution.
2018:10:16 22:04:18 jrychter It sounds like this approach deserves a good refactoring. Core.async would be just swiping the dust under the carpet. Break things down into smaller functions that do one thing only, and are testable.
2018:10:16 22:18:16 the2bears I was actually asking about your paths, that you're "passing around". I don't know what you mean by that. I can imagine a collection of functions that act upon something flowing through your program. They could be "wired" by a state machine, or a wiring of channels. Why do you think you might need to pass channels around? It's certainly possible, but it might also make sense to push data to a channel that triggers a flow through your functions and something comes out the other end, transformed.
That's what I was asking about, with the "what you're trying to do" comment. A description of your data/logic flow in a bit of high-level detail rather than just "porting an old AI project" which doesn't tell much 🙂
2018:10:16 22:18:42 the2bears ie. What does the old AI project do?
2018:10:17 12:10:44 ho0man Hi, Is there a way to make s/cat to generate vector instead of list (other than using s/with-gen) ?
2018:10:17 12:12:24 alexmiller No, and that is a known issue
2018:10:17 14:05:41 hmaurer Is there any way I can define a generator for a spec like this?
(s/def ::hello #(isa? % :im/base))
2018:10:17 14:05:55 hmaurer I would want it to pick at random from (descendants :im/base)
2018:10:17 14:06:24 hmaurer (it has to pick from the descendants at the point when the generator is called, not at the point where the spec/generator is defined)
2018:10:17 14:08:12 hmaurer nevermind, that’s just a simple application of with-gen!
2018:10:17 15:46:06 strickinato Hi - I’m pretty new to clojure, I’m playing around with spec and ran into some trouble:
I’m trying to spec a Person that has a last-contacted key and I’m using clj-time, such that I expect the field last-contacted to be a DateTimeZone. I’m running into problems though and I’m not sure how I’m supposed to be using the spec from their library (or generally the best way to proceed.
### Thought 1:
I’d like to access the ::date-time spec defined in clj-time.spec since my understanding is specs are “registered globally”
Unfortunately, it appears I can’t access it. Trying to (s/exercise ::date-time) yields and “Unable to resolve spec” error and qualifying it like (s/exercise clj-time.spec/::date-time) gives a parse error
Question: How do I use specs defined in libraries?
### Thought 2:
If the above did work, how could I even use it with spec/keys. From what I can tell, the name of the spec has to match the name of the key. eg:
(s/def ::person (s/keys :req [::name, ::date-time]))
Question: Is there syntax for doing something like the following invented code:
(s/def ::person (s/keys :req [::name, [:last-contacted ::date-time]])) where a key with a different name gets a spec run on it?
2018:10:17 15:55:03 dadair ::key is syntactic sugar that expands to :the.current.ns/key
2018:10:17 15:56:36 dadair So if you see the use of ::key in some library’s namespace “com.foo”, you need to use :com.foo/key. You will also need to require com.foo so that the ns is evaluated and the specs are registered
2018:10:17 15:48:36 taylor re: #1 you just need a slightly different syntax: :clj-time.spec/date-time, or if you've aliased clj-time.spec e.g. [clj-time.spec :as ts] then you could do ::ts/date-time
2018:10:17 15:56:40 strickinato whoa - very unexpected, but totally works!! Thank you!
2018:10:17 15:49:56 taylor re: #2 yes the name of the spec must match the name of the key, however you can alias the original spec to whatever name you like e.g. (s/def ::my-date-time :clj-time.spec/date-time)
2018:10:17 15:50:13 taylor (then use ::my-date-time in your keys spec)
2018:10:17 15:57:35 strickinato Thank you so much!! All answers I needed!
2018:10:17 16:13:33 Nolan where would be the best place to expand a bit on a question from yesterday? i have a somewhat longer scenario id like to get some input on, but dont want to spam the channel. is there an active mailing list for spec?
2018:10:17 16:25:00 taylor I wouldn't worry about spamming the channel :man-shrugging: There's clojureverse, stack overflow (code review section?), there's a google group/mailing list for Clojure but not sure if there's a spec-specific one
2018:10:17 16:33:44 Nolan much appreciated. still thinking on it a bit
2018:10:17 18:12:35 manutter51 This seems like an easy one: how do I specify the :args for this fn when I do s/fdef?
app:cljs.user=> (demo {:key/a "foo"})
#error {:message "Call to #'cljs.user/demo did not conform to spec:\nIn: [0] val: [:key/a \"foo\"] fails at: [:args :opts] predicate: map?\n:cljs.spec.alpha/spec #object[cljs.spec.alpha.t_cljs$spec$alpha38378]\n:cljs.spec.alpha/value {:key/a \"foo\"}\n:cljs.spec.alpha/args {:key/a \"foo\"}\n:cljs.spec.alpha/failure :instrument\n", :data {:cljs.spec.alpha/problems [{:path [:args :opts], :pred map?, :val [:key/a "foo"], :via [], :in [0]}], :cljs.spec.alpha/spec #object[cljs.spec.alpha.t_cljs$spec$alpha38378], :cljs.spec.alpha/value {:key/a "foo"}, :cljs.spec.alpha/args {:key/a "foo"}, :cljs.spec.alpha/failure :instrument}}
Error: Call to #'cljs.user/demo did not conform to spec:
In: [0] val: [:key/a "foo"] fails at: [:args :opts] predicate: map?
:cljs.spec.alpha/spec #object[cljs.spec.alpha.t_cljs$spec$alpha38378]
:cljs.spec.alpha/value {:key/a "foo"}
:cljs.spec.alpha/args {:key/a "foo"}
:cljs.spec.alpha/failure :instrument
2018:10:17 18:34:01 manutter51 wait, for some reason you just made me think of the caveat about nested specs, I wonder if that’s it…
2018:10:17 18:54:37 manutter51 Nope, no joy in Specville
2018:10:17 18:54:54 manutter51 I should mention this is CLJS-specific — seems to work ok in CLJ
2018:10:17 19:46:30 taylor hmm yeah I tested in CLJ and it works, and I feel like I've seen a JIRA ticket for this issue or similar for CLJS
2018:10:17 19:50:28 Nolan finally got around to posting a longer question on SO. hopefully it makes sense, but let me know how it could be improved for clarity and whatnot, or if im just missing basic concepts here. in any case its been a fun (s/)exercise
https://stackoverflow.com/questions/52862471/using-clojure-spec-to-decompose-a-map
2018:10:17 21:07:05 Nolan @taylor epic response! really appreciate it. i had played with using a separate s/def for the keys, but i kept going down a path where i was trying to use multi-specs there, too, to no avail. i think ill ultimately use something like the get-spec-keys way. the eval isn’t something i had thought about but thats a cool one too. thanks again man!
2018:10:18 17:45:00 Nolan pretty amped about how this turned out. ended up dispatching using s/or, and it will work with arbitrarily merged multi-specs (as long as the s/keys are defined by the convention you suggested, which im ok with, since this is sort of a hack anyway). but its pretty sweet. it works pretty much just like select-keys i.e.
(select-keys-spec m ::many-level-merged-spec)
2018:10:18 16:20:11 lwhorton disregard, being stupid again. 😞
2018:10:18 16:20:48 lwhorton i was getting compiler errors and it was mostly unrelated (missing spec definition further up the chain)
2018:10:18 16:21:20 favila :foo/1 won't read; using a bare number as the name part in a keyword literal is a bad idea
2018:10:18 16:21:53 favila that it works at all is an historical accident, and it wasn't fixed because it was noticed too late
2018:10:18 16:22:09 lwhorton is ::1 equally a bad idea?
2018:10:18 16:22:17 favila yes, so is :1
2018:10:18 16:23:01 favila think "would this be a readable symbol if I chopped off the initial :s?"
2018:10:18 16:24:30 lwhorton well, isn’t {:1 true} a valid map?
2018:10:18 16:24:37 favila it is
2018:10:18 16:24:45 favila I'm just talking about literals
2018:10:18 16:25:45 lwhorton hm, i have to digest this because it’s not making sense. are you saying that ::1 ultimately just becomes my.namespace/1 and thus {1 true}?
2018:10:18 16:29:06 favila I am saying clojure does not have a strict formal grammar for keywords and symbols, only human descriptions and implementations; using numbers is a greyer, edgier case that risks you falling into implementation-specific behavior
2018:10:18 16:32:25 favila I think this started it all
2018:10:18 16:33:04 alexmiller well I’m a core member and I wrote that ticket and patch
2018:10:18 16:33:28 alexmiller the original intent was that keywords follow the same constraints as symbols
2018:10:18 16:33:38 alexmiller symbols cannot start with a leading digit
2018:10:18 16:34:08 alexmiller the regex for keywords is subtly incorrect and allows keywords to have a leading digit
2018:10:18 16:34:33 alexmiller by “fixing” that we discovered that people were using such things (at the time java.jdbc made result set keywords like that for example)
2018:10:18 16:35:04 alexmiller there did not seem to be any good reason to break people’s existing working code so we rolled that back
2018:10:18 16:36:06 alexmiller so it’s possible to read and use :1 and ::1 and we have no plans to make that stop working
2018:10:18 16:36:31 favila it's deliberately not specifically allowed, however?
2018:10:18 16:36:35 favila by specifications
2018:10:18 16:36:45 alexmiller I would say that’s never been resolved
2018:10:18 16:37:20 alexmiller CLJ-1527 is the ticket to do so
2018:10:18 16:38:11 favila > Keywords - Keywords are like symbols, except:
> They can and must begin with a colon, e.g. :fred.
> They cannot contain '.' or name classes.
2018:10:18 16:38:35 alexmiller “resolving” means making that reference page and the code in sync
2018:10:18 16:38:52 alexmiller but the first question is what should be allowed
2018:10:18 16:40:10 alexmiller there are plenty of other open areas for characters that are allowed in the reader but not explicitly allowed in the reference docs
2018:10:18 16:41:41 favila two things: 1) "are like symbols" seems to cause confusion about whether what follows includes or does not include the initial colons. that was what my comment in cljs-667. 2) keywords can't have '.'? even in the ns part? (I know putting them in the name part is discouraged because that looks like class names). That's definitely wrong?
2018:10:18 16:44:09 alexmiller keywords can definitely have dots in the namespace part
2018:10:18 16:45:23 favila any way to fix those docs short of a jira patch?
2018:10:18 16:45:58 favila "They cannot contain '.' after the first '/' or name classes" suggested edit
2018:10:18 16:46:19 alexmiller It says “or in namespace names” ?
2018:10:18 16:46:55 favila "keywords are like symbols, except: They cannot contain '.' or name classes." is what it says now
2018:10:18 16:47:13 alexmiller oh, sorry I was looking at the symbol part
2018:10:18 16:48:16 favila "are like symbols" is the handwavy part that I think causes most of the confusion
2018:10:18 16:48:16 alexmiller I’ll look at an edit to that, but the intent again is to be similar to symbols and allow dots in namespaces
2018:10:18 16:49:06 alexmiller agreed - the question is whether to describe two things as similar and then indicate differences or to restate, obscuring the differences
2018:10:18 16:50:18 alexmiller I don’t feel that there is any ambiguity in intent that keyword namespaces should allow .
2018:10:18 16:50:23 favila specifically "Symbols begin with a non-numeric character" combined with "keywords are like symbols" causes one to wonder, "does a keyword begin with : or the char after that"?
2018:10:18 16:50:38 alexmiller well that is exactly the bug in the regex
2018:10:18 16:50:49 alexmiller but I would say after the colon
2018:10:18 16:51:31 favila the intent of the spec being: "expand ::, chop off the first :, and you should have a valid symbol literal"
2018:10:18 16:51:33 alexmiller if you ask for the namespace and name parts of a keyword, you don’t see a colon. the colon is syntactical
2018:10:18 16:52:08 alexmiller I don’t think that’s a semantically meaningful operation, so I wouldn’t describe it that way
2018:10:18 16:53:03 favila what I mean is the intuitive sense of a valid keyword was intended to be "does it look like a symbol if the colons that make it a keyword were removed"
2018:10:18 16:53:28 favila or at least thats how I am interpreting what you are saying.
2018:10:18 16:53:31 alexmiller I understand what you mean. I just wouldn’t describe it that way
2018:10:18 16:54:33 favila I think that's a little better than "keywords are like symbols," but still inappropriate for a capital-S "specification"
2018:10:18 16:55:10 favila but I am glad I've discovered the authorial intent behind that phrase
2018:10:18 17:58:01 borkdude hmm, no matter what options I give, custom-printer doesn’t work
2018:10:18 17:59:24 borkdude ah got it
2018:10:18 17:59:27 borkdude constantly
2018:10:18 17:59:33 bbrinck Yep 🙂
2018:10:18 17:59:53 borkdude thanks 😉
2018:10:18 18:00:04 bbrinck ha, I didn’t really do anything 😉
2018:10:18 18:00:16 bbrinck Just keep in mind that alter-var-root will appear to not work if you are within a clojure REPL (you’ll want set! in that case)
2018:10:18 18:05:51 borkdude it appears that expound is working, but I don’t see my function being used in the output:
2018:10:18 18:20:31 borkdude @bbrinck in my custom function I want to say: if this is a com.stuartsierra component map, I want to print only the keys (else I’m flooded with pages of information), else I want to do whatever is the default in expound
2018:10:18 18:20:50 borkdude @bbrinck can I do the else branch in a nice way?
2018:10:18 18:31:35 bbrinck @borkdude Hm, unfortunately it’s not super easy to do the else part right now, but if you create an issue I’ll think more about how to make it nicer. Right now, I think the best you could do is to use the private function value-in-context or grab the value of the private var *value-str-fn* and call that. https://github.com/bhb/expound/blob/master/src/expound/alpha.cljc#L85
2018:10:18 18:31:41 bbrinck Not exactly pretty 😞
2018:10:18 18:32:32 borkdude What’s the best way to make clojure.spec.alpha not print the entire component system map. We are using (s/assert ::system system). Even when making a custom expound printing function, clojure.spec still barfs my entire console with output.
2018:10:18 18:34:15 bbrinck I’m not sure exactly what you mean. Can you provide an example of the output?
2018:10:18 18:40:08 bbrinck I’m curious to know which part of the output is showing the entire map - I would think if you provide a custom printer, you could in principle hide it entirely (or show as much as you like), but maybe it’s printing somewhere else?
2018:10:18 18:41:08 borkdude until line 124 it’s ok, there I only print the keys of the system map. afterwards, still the whole system is printed.
2018:10:18 18:44:03 bbrinck @borkdude Ah, right, so this isn’t actually spec - the issue is that the program is printing out all the exception data from the assertion
2018:10:18 18:44:44 borkdude right
2018:10:18 18:44:57 bbrinck Is this running in boot? I wonder if the error printer is configurable? I can see why this is the default (you want to see the data for an assertion exception)
2018:10:18 18:45:14 bbrinck but in the context of spec, it will put all of the detailed explain-data into the exception
2018:10:18 18:45:14 borkdude yes, it’s boot
2018:10:18 18:46:39 bbrinck I can see why that’s how boot works and I can see why spec adds a lot of info to the exception (might be useful), but when they work together, it’s definitely a lot of output.
2018:10:18 18:47:26 borkdude I also tried this:
(try (rr/reset)
(catch Exception e
(throw (component/ex-without-components e))))
2018:10:18 18:47:55 borkdude but that doesn’t work, because the assert exception already has the entire system in it :-s
2018:10:18 18:48:46 bbrinck hm, not sure what you mean. Why can’t you alter the exception data in ex-without-components?
2018:10:18 18:48:53 bbrinck is it already in the string message?
2018:10:18 18:49:13 bbrinck if it’s just in the data, I presume you could just dissoc it
2018:10:18 18:50:05 bbrinck or just construct a new error that has less info, using the message from the exception
2018:10:18 18:50:28 borkdude hmm, I have to inspect the exception in the REPL to see what to dissoc, but good idea
2018:10:18 19:40:06 borkdude @bbrinck so what I wanted was actually not a custom :value-str-fn function, but a way to transform my value before it’s printed by expound. would it make sense to have a :transform (fn [v] ...) option in expound on the custom printer?
2018:10:18 19:46:46 bbrinck Hmm, good question.
2018:10:18 19:47:24 bbrinck Can you talk more about why a transform fn would be preferable to customizing the way it prints?
2018:10:18 19:48:42 bbrinck Maybe we should move this to #expound since we’re talking about potential features
2018:10:20 10:28:03 slipset This is probably working as expected, but why?
2018:10:20 10:29:41 slipset Then, when I ask if a given fn is valid:
2018:10:20 10:30:01 slipset
(s/valid? ::reducing-fn (fn [x y]) ;; takes forever and ends with a out of heap memory
2018:10:20 10:31:13 slipset So it seems (because test.check needs to be a dep for this to run) that spec is using generative testing to verify if the spec is valid.
2018:10:20 10:31:49 slipset Hmm, do I actually have a question? No, probably not, I just wanted to state that I was surprised by this, but it’s probably just the way it has to be.
2018:10:20 10:33:52 hmaurer Is there a spec which forces a certain value? same as #(= % some-value) but with a built-in generator, etc
2018:10:20 10:43:17 taylor You can use a set with the value in it
2018:10:21 22:00:31 alexmiller That’s the compiler hook under fn
2018:10:21 22:01:38 alexmiller Certainly not the preferred entry point
2018:10:21 22:06:09 borkdude We’re writing some specs for clojure.core functions. I notice things can get a bit slower on big codebases with instrumenting all core functions. Is there a way to let spec know to not check when a function is called from clojure itself? so only external calls. I hoped direct linking would solve this, but I don’t think it is now
2018:10:21 23:08:27 alexmiller I would not expect calls between core functions to get checked
2018:10:21 23:18:12 andy.fingerhut I would be very surprised, if you are compiling Clojure with the default option of direct linking enabled, that calls between Clojure functions within Clojure are being checked, if you are using instrument. If you believe you are seeing otherwise, I would love to see an example of that.
2018:10:21 23:20:33 andy.fingerhut instrument works by re-def'ing Vars. direct linking means that the current value of the Var is ignored, instead being embedded in the calling function at compile time.
2018:10:21 23:22:35 andy.fingerhut FYI, there is one function in clojure.data/diff's call tree that calls a function in clojure.set (I think clojure.set/difference) with non-set arguments. I have tried writing a spec for clojure.set/difference that causes errors if you call it with anything other than a set, and the only way I could make the error be caught is to disable direct linking when compiling Clojure.
2018:10:21 23:26:01 andy.fingerhut And if you want to try compiling Clojure with direct linking disabled, and thus likely see instrumented runs get even slower, there are some instructions near the bottom of this page, with the heading "Building Clojure without direct linking": https://dev.clojure.org/display/community/Developing+Patches
2018:10:22 06:51:10 borkdude Thanks. Then my understanding of direct linking is correct and I need to do more research to see how to speed up things.
2018:10:21 23:05:27 hmaurer is there a shortcut for a Spec s/or which only contain spec keywords and want to tag them with the same keyword? i.e. (s/or ::foo ::foo ::bar ::bar ...)
2018:10:21 23:07:34 alexmiller Not in spec, no
2018:10:21 23:09:49 hmaurer Thanks @alexmiller. Other question: I read that specs are discouraged to be used as runtime because they can be slow. With that in mind, would it still make sense to use specs at runtime to branch on code based on the shape of data. i.e. check if data conforms to some spec to know if it’s of some “kind”, branch on that, etc
2018:10:21 23:10:03 hmaurer use it as duck typing for dynamic dispatch
2018:10:21 23:11:16 hmaurer i.e. use s/valid? and s/conform pretty extensively in runtime code
2018:10:21 23:11:24 alexmiller Only you can answer whether it’s fast enough for you
2018:10:21 23:11:58 hmaurer It’s definitely fast enough for me; I’m just wondering if it goes against some deeper philosophy that I’m missing and could come bite me in the back down the line (beyond performance)
2018:10:21 23:44:53 hmaurer I’ve another question on Clojure spec for whomever is around: how do you avoid circular namespace dependencies? It seems preferable to put every spec in a namespace mapping to the namespace part of the spec keyword to be able to then require and alias that namespace wherever the spec is used. i.e.
2018:10:21 23:47:05 hmaurer The only solution I can think of it to still create the my-app.schema.person namespace but to move the attribute specs for person in the my-app.schema namespace, like this:
2018:10:21 23:48:00 hmaurer actually I could still use the namespace alias here, but I would have to define the attribute specs in my-app.schema namespace and essentially just create an almost blank file for the my-app.schema.person namespace
2018:10:21 23:54:31 misha > It seems preferable to put every spec in a namespace mapping to the namespace part of the spec keyword
false opieop
actually, "it really depends..."
2018:10:21 23:56:46 misha imagine random map flowing through your app code. Do you really want to look for :my-app.schema.person/first-name in it? or is something more readable would be better and enough? (and I bet :my-app is much longer, and you'll end up with keys violating PEP8's 80 chars length in no time)
2018:10:22 00:00:56 misha also, I feel like you split namespaces too much (with regards to spec, at least.)
2018:10:22 00:01:17 misha @hmaurer
2018:10:22 00:03:56 hmaurer @misha can you make it much shorter though? you could skip the my-app.schema part, sure
2018:10:22 00:06:30 misha the spec kw length you have in example - is ok as is I think, but: I'd used that length for something more descriptive/convenient, so you would not feel the nee to alias it away, and avoid entire issue with dependencies.
2018:10:22 00:08:58 misha and when you don't need to use aliases, you just need to make sure all specs are :required at the app entry points. no need to import those all over the place, as those are accessible from global registry, which you can get away initing just once.
2018:10:22 00:11:53 misha it is helpful to think about spec-kw namespaces as db table names, not as code namespaces those happen to be defined in. Otherwise specs would not survive first mild refactoring
2018:10:22 00:13:02 hmaurer @misha that makes sense; thanks a lot 🙂
2018:10:22 00:13:42 hmaurer although I was going to put those spec in a schema namespace, not next to code, so there isn’t a big difference there between that and the “table names” approach you’re suggesting
2018:10:22 00:13:51 hmaurer beyond the extra verbosity of the prefix
2018:10:22 00:14:53 misha you receive map as rest response. it contains "...schema.../..." keys. what a waste of electricity opieop
2018:10:22 00:15:12 hmaurer 😄
2018:10:22 00:16:12 misha then, imagine, you now need to move them around in the code. – you either loose all your aliases, or break backward compatibility. ouch
2018:10:22 00:17:10 andy.fingerhut I mean, isn't one of the proposals of namespaces in keys that you don't rename them, unless you are planning to make a breaking change in a published API, and you should weigh consequences of that pretty seriously?
2018:10:22 00:17:11 hmaurer so I guess it would still be fine to namespace-qualify collection specs (i.e. (s/def ::person (s/keys ...), but you would avoid such long namespace qualification for key specs
2018:10:22 00:17:26 hmaurer ?
2018:10:22 00:17:55 andy.fingerhut And gzip and things like Transit reduce the waste of electricity somewhat.
2018:10:22 00:18:02 misha there is an ugly workaround, (create-ns 'foo.bar.baz) (alias 'fbb 'foo.bar.baz), but it is meh.
2018:10:22 00:18:12 hmaurer because collection specs are essentially a codebase-level notion, whereas keys may be sent over the network, stored, etc
2018:10:22 00:18:21 hmaurer I think?
2018:10:22 00:18:59 misha @andy.fingerhut imagine "...schema..." suffix in every sql column or table name
2018:10:22 00:19:28 andy.fingerhut My mind isn't reeling yet, but I haven't lived and breathed it day in and out, either. Six of one, half dozen of another.
2018:10:22 00:19:51 misha and yes, you should not rename those, if they are public outside the app code. therefore if you suddenly need to move definitions elsewhere - file ns and spec ns do not match anymore.
2018:10:22 00:21:06 misha @hmaurer nested maps is the collection spec over the wire right there.
2018:10:22 00:22:01 hmaurer mmh yeah
2018:10:22 00:22:24 hmaurer but the keyword does not appear on the wire
2018:10:22 00:22:35 misha anyway. the best way to appreciate this - is to experience pain yourself kappa
2018:10:22 00:24:53 misha try to model your app your way, and give it week or two
2018:10:22 00:27:16 hmaurer ah yep nevermind, you’re right
2018:10:22 00:27:21 misha also, if you don't preserve ns for js clients - you are missing out as well: "my-app.schema.person/first-name"
e.g. cheshire prints qualified keywords this way by default
2018:10:22 00:27:49 misha and in js/python/java/xpath/etc. you can just obj["my-app.schema.person/first-name"]
2018:10:22 00:28:38 hmaurer so, i.e., in this case, would you name things as:
2018:10:22 00:30:44 misha well, if you are confident you will not have external person - it might be ok. otherwise, prepending with a short project codename would not hurt.
but I'd think twice before adding anything between codename and entity name
2018:10:22 00:32:49 misha if you are in a corporate setting, you are probably required to add at least full org name instead or in addition to project name: :com.cognitect/person:com.cognitect.project/person
2018:10:22 07:34:33 borkdude Is it possible to have clojure.spec not throw exceptions on fdef violations, but only print “fdef error, line so and so” and then continue as always?
2018:10:22 13:08:35 alexmiller no
2018:10:22 13:31:30 hmaurer is there a way to conform a value to a spec only one level deep?
2018:10:22 14:04:52 alexmiller no
2018:10:22 22:50:21 hmaurer thanks
2018:10:22 22:45:43 hmaurer https://github.com/funcool/cats/blob/master/src/cats/monad/either.cljc#L48
Is there anyway I can spec such a type (`Either left right`) on a per-function basis? So for example, I might want to add a spec to a function saying it returns either a string? or a number?, etc
2018:10:22 22:53:23 alexmiller Always good to remember that specs are not types. You can use s/or to spec an alternative but that’s not going to be generic.
2018:10:22 22:53:25 hmaurer (I realise this sounds similar to what the s/or spec does, but I need to work with records here, as defined in cats)
2018:10:22 22:53:38 hmaurer ah
2018:10:22 22:53:50 hmaurer is there any way I can do something generic in this case @alexmiller?
2018:10:22 22:54:49 alexmiller You can treat records as maps with unqualified keys and use :req-un to specify specs on keys
2018:10:22 22:55:00 alexmiller But, not generically
2018:10:22 22:56:06 alexmiller For specific functions, you could define a :fn spec that checks a relationship between args and ret
2018:10:22 22:56:28 hmaurer @alexmiller but then I’ll have to make qualified keys for every possible way there is to use the Either record. As far as I know there is no way to directly define an unqualified map spec, right? i.e. nothing like (s/foo :left string? :right number?) which would accept a map with keys :left and :right matching the specs
2018:10:22 23:24:53 misha or, keep lifting separate from business-logic
2018:10:22 23:30:19 favila @hmaurer I had this problem continually, where I have a spec key but I need to narrow it further in specific contexts without changing the key name
2018:10:22 23:30:28 favila I ended up making a terrible macro, keys+ to do it
2018:10:22 23:31:47 favila (s/def :either/either (s/keys ::req-un [:either/left :either/right])) is the basic notion of "either"
conformed values are garbage though, but exercise works just fine
2018:10:22 23:42:30 hmaurer thanks @favila! I’ll read this up. Does this spec give nice errors / generators / conforming?
2018:10:22 23:45:19 favila The only thing you are allowed to do is use :conf to override the normal spec/predicate/generator for that key
2018:10:22 23:45:52 favila “Override” is maybe too strong because the registered spec is checked too
2018:10:22 23:57:25 hmaurer Ah right, great. Out of curiosity do you use Either / cats in your codebase @favila? Or did you write this for another use-case?
2018:10:23 00:18:18 favila Other use case
2018:10:23 00:18:25 favila But it’s the same basic problem
2018:10:23 00:19:02 favila There’s a general structure and spec, but also more specific specs which must nevertheless maintain the structure
2018:10:23 00:19:48 favila Using predicates all the time became unbearable
2018:10:23 13:39:25 j0ni Hey folks, I'm having a tough time trying to figure out how multi-specs interact with unqualified keys - here's some example code which is a trivial version of what I'm trying to achieve:
any guidance as to how I square this apparent circle?
2018:10:23 14:01:40 seancorfield @j0ni Your defmulti and s/multi-spec should have :mode, not ::mode. You want a function that operates on your data and returns the discriminant. That's the keyword :mode, not the spec ::mode.
2018:10:23 14:09:04 j0ni wow, thanks @seancorfield! I had been through this permutation, but apparently evaluating the ns is insufficient, I needed to reload the namespace completely
2018:10:23 14:10:11 seancorfield A trick with defmulti is to put (def animode nil) above it and then it should work with just a re-eval of the ns.
2018:10:23 14:10:26 j0ni ooh nice 🙂
2018:10:23 14:10:57 j0ni well, useful, if actually a little gross 🙂 but I'll take it
2018:10:23 15:41:05 adamfrey I was surprised by this behavior in spec. Is there a reason that I shouldn't expect this to work?
the spec just calls the set as a function and (#{nil} nil) => nil in clojure and spec is looking for the function truthiness. The only way to make this work would be to change the way spec deals with sets as specs where spec could call them with (contains? #{nil} nil) instead. But that sort of special casing might be undesirable.
2018:10:23 16:42:47 bmaddy I'm trying to come up with a spec for nested associative structures:
2018:10:24 02:26:37 taylor @bmaddy I’m able to get that to reliably generate with s/*recursion-limit* 1 by pulling the recursive s/or branch into its own spec:
but I’m not sure why that is, maybe it’s the way that recursive depth is tracked internally and the “inline” recursive s/or specs lose some context on recursion?
2018:10:24 03:03:40 misha this is the first thing I did, and it still failed for me numerous times, @taylor
2018:10:24 03:53:40 taylor :man-shrugging: works on my machine
2018:10:24 04:16:50 bmaddy Oh, I get it now. I'd tried setting s/*recursion-limit* to a low number before to help avoid StackOverflow issues, but that's actually the problem here. When the recursion limit is hit, it must throw that error. Thanks @misha and @taylor!
2018:10:24 15:40:50 basti Is there a strict validation mode in spec?
Whereas if you provided more data then fail?
2018:10:24 15:52:35 dadair No; closed specs are generally discouraged. You can do your own strictness using s/and, however.
2018:10:24 19:31:30 bbrinck @basti spell-spec gives you options for either keeping the map open (but reporting likely misspelled keys) or complete closed maps (not recommended) https://github.com/bhauman/spell-spec
2018:10:25 00:23:49 jrychter I really do not like this canned answer ("closed specs are discouraged"). It gets repeated a lot, and yet there are valid scenarios where you do want to restrict the data to exactly what is specified in the spec without manually duplicating lists of keys. Think about API endpoints, or anything that receives data from an untrusted source in general.
2018:10:25 00:34:44 seancorfield If you ignore the additional parameters, why do you care whether they are passed in?
2018:10:25 00:36:32 seancorfield One valid case I can see for closed specs is when you're about to store data into a database (and you'll get an error if you try to store columns/keys that do not exist in the table). At that point tho', you can extract the list of valid keys from the spec itself and use select-keys to get just the keys you care about. Again, you don't necessarily care that more data was present -- you just need to ensure you don't send it to JDBC.
2018:10:25 00:37:04 seancorfield (and of course you could use select-keys on the API input data too, if you wanted)
2018:10:25 00:37:37 seancorfield The reality is that most APIs out there silently ignore extra form and query parameters.
2018:10:25 00:38:03 seancorfield (which is all a long way of saying I think the canned answer is correct)
2018:10:25 00:59:03 taylor I agree there are cases were you need to restrict inputs but I think those are usually near the boundaries of apps, and spec doesn’t prevent this. Other than that I don’t think (in Clojure) your code should “break” if passed a map with some data it doesn’t care about, and I think most projects have much more code on that side of the fence than the other
2018:10:25 00:59:33 jrychter If your database is a document database (like mine) and stores your data pretty much in model form (like mine does), and your API serves your ClojureScript app which keeps the same data model, then you very much care about additional parameters. You can pass data pretty much intact between client code and the database, but you do need to make sure that you don't store any extra data that just happens to arrive over the network.
2018:10:25 01:01:24 jrychter The manual select-keys solution seems obvious and results in error-prone code (been there done that). A more reasonable approach is to get the list of allowed keys from the spec.
2018:10:25 03:30:41 seancorfield When I do select-keys I derive the list of keys to be selected directly from the spec.
2018:10:25 03:41:56 jrychter …and that is exactly what I'll be doing.
2018:10:25 01:02:38 jrychter I'd also submit that "most APIs" describes "most traditional APIs". My APIs speak EDN and receive Clojure data structures which I verify using spec.
2018:10:25 01:04:43 csm possibly s/conform should remove keys not in the spec? I’m not sure of the implications of that, though.
2018:10:25 01:05:44 taylor there’s no need to change spec, you can easily get this restrictive behavior if you need it
2018:10:25 01:07:16 bbrinck You don’t need to duplicate keys w/ spell-spec
2018:10:25 01:09:08 csm maybe I should just ask, though: why doesn’t conform remove keys not in a map spec?
2018:10:25 01:09:28 jrychter spell-spec solves a different problem and has a different goal. But I am dealing with the problem myself, I just wanted to point out that whenever people repeat what Rich said about specs intended to be open, there should be a footnote saying "except in cases where you do need to restrict your data, such as in APIs".
2018:10:25 01:11:38 bbrinck @csm AIUI, conform is useful for destructuring, so presumably there isn’t a need to remove keys.
2018:10:25 01:12:02 bbrinck (because you would usually know the keys you want to grab as part of destructuring)
2018:10:25 01:12:47 csm 👍 I’m just thinking out loud if there are approaches that would satisfy use cases like the above, but without making specs closed
2018:10:25 01:13:04 bbrinck Ah, I see
2018:10:25 01:14:10 bbrinck remember that conform also changes the shape of data, so may not apply here if you’re trying to write the input to, say, a document DB
2018:10:25 01:14:57 bbrinck (I suppose you could conform/unform if it did remove unknown keys, but that gets a little strange 🙂 )
2018:10:25 01:15:26 csm yeah, I thought about that; maybe a fn that does “check this spec, then select-keys” would be an OK addition
2018:10:25 01:15:39 andy.fingerhut Doesn't the word "discouraged" in "closed specs are discouraged", vs. e.g. "closed specs are prohibited", convey to you that there might be occasional valid use cases for them?
2018:10:25 01:15:40 bbrinck anyway, I am curious why strict-keys from spell-spec doesn’t work here
2018:10:25 01:17:03 andy.fingerhut And sure, people can repeat things without knowing the reasons for them, nor the subtler points of when they are good ideas, and when they are not. That is a more general habit of some people to be second-handed in what they say, rather than understanding themselves.
2018:10:25 01:17:16 andy.fingerhut not restricted to computing at all.
2018:10:25 01:17:37 bbrinck Yeah, I think “discouraged” is an accurate term, especially since I’ve seen a few people jump to use closed specs by default. It turned out that open specs were a better fit.
2018:10:25 01:17:53 bbrinck But yes, there are exceptions
2018:10:25 01:18:21 csm I, for one, wish spec was mature before I started using schema instead. I’m not a fan of closed specs for APIs
2018:10:25 01:25:04 jrychter I do agree that open specs are a good idea.
2018:10:25 01:27:25 andy.fingerhut @jrychter I am not trying to jump down your throat. I am asking because I would actually like to learn more about your use case where you want to disallow unrecognized keys. You say you may get these extraneous keys over the network at some document database service, and want to detect this as an error?
2018:10:25 01:28:44 jrychter @andy.fingerhut my API receives EDN data, in model form. I want to pass this data to my model code (where extra keys might not matter), and often directly to a document database (RethinkDB in this case). I do not want to store extra keys, for obvious reasons.
2018:10:25 01:29:26 jrychter I know that if one uses an SQL database, there is code that needs to re-pack the data, but in my case there is often no such code.
2018:10:25 01:31:08 andy.fingerhut Bear with me if I ask something foolish for lack of experience in these kinds of applications, but if the data model is extended in the future, a reason commonly given for open specs is that unless you can somehow arrange for both participants to be upgraded simultaneously (e.g. by shutting down the service while everyone upgrades), then you can't upgrade without lots of errors flying around during the partially updated state, can you?
2018:10:25 01:32:13 jrychter An invariant in my case is that the spec describes the data model.
2018:10:25 01:32:52 andy.fingerhut In all processes on all systems where the code is running, all of the time, even during updates to the data model?
2018:10:25 01:33:52 andy.fingerhut If yes, then it sounds like partially updated systems are never an issue in your use case. That certainly simplifies implementing such things, if so.
2018:10:25 01:36:53 jrychter At the moment I mostly use atomic "stop-the-world" migrations, but in general the assumption has been that the spec needs to accomodate partially updated systems. But again: I do agree that open specs are a good idea, I just think that people take Rich's "discouragement" a bit too far and see closed specs as "a bad idea". They are not a bad idea.
2018:10:25 01:38:06 andy.fingerhut If open specs are a good idea, then there must be some situations where closed specs are a bad idea.
2018:10:25 01:39:00 andy.fingerhut It sounds like Rich & team's experience in consulting is that they have seen multiple scenarios where people wanted to close their specs, and later regretted it.
2018:10:25 01:42:07 andy.fingerhut I mean, I know people can read/hear that and think "Rich means that for 100% of all software everywhere", but I doubt that. But he is speaking in context of an industry where I think there is an over-tendency to make brittle inextensible systems.
2018:10:25 01:42:23 taylor spec isn’t particularly unusual in encouraging openness/forward-compatibility either, it’s a concern that has to be addressed in things like protobuf, thrift, etc. too
2018:10:25 01:44:10 andy.fingerhut I know I'm in the minority working at Cisco in embedded software that is far too much C code for my taste, but these systems are incredibly brittle. Changing an int to a long is often developer-weeks worth of effort, if it is in a message spread over 3 processes.
2018:10:25 03:11:11 the2bears This current discussion on closed specs sounds a little too much like using or relying on spec to clean your data. To be fair, though, I don't have any use cases for a closed spec myself and might be missing the real picture. Just seems better to clean the data with a different set of functions and tools.
2018:10:25 03:14:20 misha how about not using tool-designed-not-to-solve-A to solve A?
2018:10:25 03:15:33 misha if it is such a bummer, just implement closed keys with 1-2 macros yourself, or take one of many available in the libs already.
2018:10:25 03:16:56 misha implementing select-spec-keys is trivial, especially when you know what specifically you prefer to do if extra stuff present – ignore, warn, blow up, etc.
2018:10:25 06:10:27 orestis I implement this select-spec-keys just yesterday. It’s very straightforward to write and I can deal with assumptions about namespaces, blow up or warm etc etc.
2018:10:25 13:25:17 jaihindhreddy How can I add test.check to my deps.edn?
2018:10:25 13:29:55 jaihindhreddy Because it doesn't have any dependencies, will this work?
2018:10:25 13:38:00 jaihindhreddy Ah, so because test.check publishes artifacts (on maven) I don't need to worry about the fact that it doesn't have a deps.edn
2018:10:25 13:41:41 gfredericks Right
I guess I should add an empty one
2018:10:25 13:41:56 gfredericks I'll do that soon
2018:10:25 14:49:41 alexmiller you can force the dep type without having a deps.edn
2018:10:25 14:53:19 alexmiller that overrides the manifest detection and forces it to be a deps.edn project, which is tolerant to a missing deps.edn
2018:10:25 14:53:37 alexmiller and just treats it as a project with no deps
2018:10:25 16:15:09 seancorfield A useful trick -- is that documented in the CLI/tools.deps Reference? (specifically that it can be used to treat an arbitrary git-based project "as a project with no deps")
2018:10:25 16:37:41 alexmiller Not currently
2018:10:26 00:38:12 bbrinck If you’ve run into “Cannot convert path” errors with Expound (which happens if you use a spec with a conformer), please try out 0.7.2-SNAPSHOT and let me know what you think
2018:10:26 02:02:56 richiardiandrea Will try that out thanks a lot for the fix!
2018:10:26 08:55:37 borkdude why does the assert not throw there?
2018:10:26 09:12:42 mpenet you prolly need to enable asserts
2018:10:26 09:14:52 mpenet (s/check-asserts true)
2018:10:26 09:15:07 mpenet by default they are compiled in but not enabled
2018:10:26 09:18:39 mpenet for debugging (and because I am lazy) I sometimes use s/assert*, it bypasses the various flags
2018:10:26 09:18:50 mpenet but that's hairy territory
2018:10:26 11:27:06 gfredericks why does stest/check take symbols instead of vars? it doesn't require code for you does it?
2018:10:26 23:50:28 asilverman Hi there clojurians, I am looking for some help regarding the use of clojure.spec.alpha. Basically I would like to verify that a map of key-value pairs i receive as an argument is a sub set of a well defined group. For example let the group of valid keys be :key1 :key2. I would like my spec to conform if I get a map with en empty amount of keys, or with either :key1 exclusively or :key2 exclusively or both :key1 and :key2 exclusively. I’m confused about how to specify this
2018:10:27 01:15:06 misha @ariel.silverman what does "exclusively" mean here exactly? only those keys, and no others?
are those qualified keys? do you have value specs too, or is it just keys?
2018:10:29 16:13:03 asilverman Thank you misha for the very thorough explanation. The code snipped was extremely helpful, as @U04V70XH6 mentioned, I also asked this question in the #beginners forum. The idea was to limit an API endpoint query parameters to a known set and error out when anything but a valid sub-set of them is passed down. Thank you again!
2018:10:27 01:22:02 misha
2018:10:27 01:24:33 misha and a due disclaimer "clojure.spec is explicitly designed to discourage closed maps ("no keys except these")"
2018:10:27 01:43:32 seancorfield (it was answered in #beginners )
2018:10:27 13:59:03 souenzzo #datomic query returns a java.util.HashSet, that prints like a set but it's not a coll?. So I cant use s/coll-of on it.
I changed my spec from (s/coll-of ::stuff) to seqable?
Should spec have a s/seq-of that use seqable? or somethink like that?
2018:10:27 16:12:41 dangercoder Is running (spec/explain ::foo data-structure) in the beginning of a function bad practise?
2018:10:27 16:16:11 dangercoder I'm using spec/fdef right now and turning on instrument in dev-environment.
2018:10:27 16:36:40 borkdude Nothing’s bad, just depends what you want
2018:10:27 16:51:13 dangercoder True, I was thinking if there can be some kind of noticeable performance loss if having instrument on in production, but i'll have to measure and see later on.
2018:10:27 17:22:08 borkdude yes, this definitely affects performance
2018:10:27 17:48:14 andy.fingerhut I don't have any measurements of a real application with instrument on vs. off, but I do have measurements for clojure.set functions with instrumentation on vs. off, for a simple spec that just checks whether the args satisfy set? or not. It is partway down the README on this page: https://github.com/jafingerhut/funjible#wait-isnt-this-what-clojurespec-is-for
2018:10:27 17:49:12 borkdude I had enabled speculative that has some amount of core specs. My backend + frontend became significantly slower.
2018:10:27 17:49:17 andy.fingerhut instrument in that case is significantly slower than the original functions when their run time is short. The percentage overhead obviously gets smaller as the run time of the instrumented function increases.
2018:10:27 17:51:05 borkdude I think how may want to use “instrument all” is when I have some weird bug whose stack trace I find uninformative. Then instrument all, run again, find the problem, turn it off.
2018:10:27 17:51:55 dangercoder @borkdude sounds like a solid approach
2018:10:27 17:52:40 borkdude some specs I do want on all the time in dev, especially at boundaries like DB insert queries
2018:10:27 17:53:02 borkdude it’s a tweaking process I think
2018:10:27 18:57:43 seancorfield As another data point for the overhead of instrument: clojure.java.jdbc has its specs in a separate namespace and the time difference between running the tests with and without instrumentation -- just on the public API functions -- is huge. I would be extremely careful about turning on instrumentation in production.
2018:10:27 19:00:35 seancorfield Another thing to watch for: if you have a higher-order function instrumented, you'll invoke generative testing on the (function) argument being passed in -- which you almost certainly don't want in production (you probably wouldn't even want org.clojure/test.check included as a dependency in production?).
2018:10:27 19:13:10 dominicm We've been using s/assert to have dev and production operate differently.
2018:10:28 20:27:23 alexmiller Undetermined whether it will remain in the api
2018:10:28 20:55:55 domkm @alexmiller I'm finding it to be extremely useful, for what it's worth
2018:10:28 20:59:49 alexmiller Are you using it for anything other than or?
2018:10:28 21:00:32 alexmiller I think it’s definitely useful, but wonder whether just having a nonconforming flag on or would solve the 95% case
2018:10:28 21:11:37 domkm @alexmiller It does look like I am only using it for or currently. I still think having general nonconforming is useful and shouldn't be removed. We don't know what third-party specs we might want to utilize without conforming.
2018:10:28 21:12:57 alexmiller So far, I don’t think I’ve ever seen anyone use it for anything but s/or
2018:10:28 21:00:33 borkdude speaking of undocumented APIs, is there a way to ask spec what functions are currently instrumented? not which functions are instrumentable?
2018:10:28 21:01:38 borkdude alternatively, asking wether a function is instrumented would also be fine
2018:10:28 21:01:50 alexmiller Ask the registry
2018:10:28 21:02:28 alexmiller Well I guess that’s not telling you what’s instrumented
2018:10:28 21:02:39 alexmiller I’d look at the impl of unstrument
2018:10:28 21:02:42 borkdude I’ll tell you the use case.
2018:10:28 21:05:31 borkdude the with-unstrumented macro is problematic in clojurescript though, because part of instrumenting/unstrumenting happens at macro compilation time and it uses two separate atoms to bookkeep instrumented functions…
2018:10:28 21:38:57 borkdude if (stest/instrument 'clojure.core/reduce) gave back [cljs.core/reduce] only when it wasn’t already instrumented, I could leverage that.
2018:10:28 21:40:09 borkdude but of course I could call (s/unstrument 'clojure.core/reduce) first, to check if it was already instrumented…
2018:10:28 21:40:32 borkdude this way I wouldn’t have to use something private
2018:10:29 16:33:56 jaihindhreddy Alex Miller commented on a podcast about making specs more data oriented. Can't wait to see what's cookin' at Cognitect!
2018:10:29 13:03:15 domkm Has there been any thought/discussion about clojure.spec for directed graph validation? The problem we're running into is that some edges are required and that causes validation to never terminate due to cyclic validations. It would be interesting if there were a way to make required validation respect s/*recursion-limit*.
2018:10:30 13:17:16 borkdude I have an interesting case where I call stest/check from the REPL (lein + cider nrepl) and it doesn’t finish. Anyone familiar with that one?
2018:10:30 13:18:28 borkdude it does terminate when I run the tests from the command line
2018:10:30 13:19:18 alexmiller how do you know it won’t finish? did you solve the halting problem!?
2018:10:30 13:26:51 borkdude I’m a human, I use heuristics.
2018:10:30 13:28:55 borkdude seems to work with clj repl
2018:10:30 13:33:10 gfredericks Have you tried at least a few times from the command line?
Tests that probabilistically take a Long Time can be misleading like that with just a few data points
2018:10:30 15:05:34 borkdude @gfredericks trying.
hmm, now clj lingers after printing the result of stests:
2018:10:30 15:21:24 gfredericks so that code launches a future somehow
2018:10:30 15:22:14 gfredericks I think shutdown-agents might not be synchronous in its effect
2018:10:30 15:22:32 gfredericks so the fact that it waits longer when there's a future in the background that hasn't completed isn't surprising
2018:10:30 15:22:37 gfredericks but why there's a future in this case I don't know
2018:10:30 15:23:01 gfredericks (I haven't used clojure.spec.test.alpha much)
2018:10:30 15:26:43 taylor
at clojure.spec.test.alpha$quick_check.invokeStatic(alpha.clj:310)
at clojure.spec.test.alpha$quick_check.invoke(alpha.clj:303)
at clojure.spec.test.alpha$check_1.invokeStatic(alpha.clj:336)
at clojure.spec.test.alpha$check_1.invoke(alpha.clj:324)
at clojure.spec.test.alpha$check$fn__3088.invoke(alpha.clj:412)
at clojure.core$pmap$fn__8342$fn__8343.invoke(core.clj:6994)
2018:10:30 15:27:44 gfredericks ah well there you go then
2018:10:30 15:28:00 gfredericks gotta spend 15 seconds generating a few last giant piles of garbage in case somebody wants them
2018:10:30 15:30:23 gfredericks @borkdude the most recent alpha of test.check might be a bit better about this. I can't remember, and also don't know what version you get by specifying RELEASE
2018:10:30 15:31:04 taylor looks like org/clojure/test.check/0.10.0-alpha3
2018:10:30 15:31:21 gfredericks oh; nevermind then
2018:10:30 15:31:46 gfredericks for this kind of test it's probably best to use a smaller max-size for the gen/any
2018:11:02 12:58:30 gfredericks I wouldn't have expected that, but assuming you're right and you can't override builtins, another option would be having your own (def ::any any?) that you can override
2018:11:02 12:58:46 gfredericks unfortunate if you have to do that, but should work at least
2018:10:30 15:46:58 borkdude @alexmiller if you want I can post that script in an issue if you think it’s something that should be improved over time
2018:10:30 15:51:01 alexmiller I don’t understand what we’re talking about
2018:10:30 15:51:35 alexmiller check returns a lazy seq - you need to consume it
2018:10:30 15:52:20 alexmiller As noted in the doc string
2018:10:30 15:52:53 alexmiller So wrap a doall around the call to check
2018:10:30 15:52:55 borkdude aah, wrapping it in a doall works
2018:10:30 15:53:27 borkdude thank you
2018:10:30 15:53:08 borkdude issue solved (see above thread).
2018:10:30 15:54:22 borkdude didn’t realize that (pun intended)
2018:10:30 21:15:11 andy.fingerhut I slightly envy your pun-creation skills.
2018:10:30 18:00:47 martinklepsch I'm not sure if this is a bad idea or something but I'd love to always have stest/instrument switched on during development.
Is there some way to keep functions instrumented after re-defining them in the REPL?
2018:10:30 18:09:29 taylor I see there's (defonce ^:private instrumented-vars (atom {})) in spec.alpha, but I don't see it publicly exposed. Something that came to mind was using that to re-instrument things that were previously instrumented?
2018:10:30 20:39:58 dadair If you are using something like Component or Integrant you can wrap their system reset functions with calls to instrument
2018:10:30 20:40:15 dadair I’ve done that with our duct-based project at work
2018:10:30 22:52:48 martinklepsch I’ve considered that but often I’m just working in one namespace in the REPL and restarting the system seems like overkill in a way
2018:10:30 22:53:07 martinklepsch I guess I should just bind stest/Instrument to a hotkey
2018:10:30 18:45:27 borkdude @martinklepsch I hook up stest/instrument with component. So when I restart the system, my new fns are instrumented.
2018:10:30 23:10:21 borkdude somehow this also lingers after the test has executed, even with doall:
(ns spec-keys-test
(:require
[clojure.test :as t :refer [deftest testing is]]
[clojure.spec.alpha :as s]
[clojure.spec.test.alpha :as stest]))
(defn foo [n] n)
(s/fdef foo
:args (s/cat :n number?)
:ret number?)
(deftest repro-test
(testing "output of check"
(let [r (doall (stest/check `foo))
c (count r)]
(println "count" c)
(is (= c (count (keep :spec r))))
(is (= c (count (keep :sym r))))
(is (= c (count (keep :clojure.spec.test.check/ret r)))))))
(t/run-tests)
2018:10:30 23:56:42 andy.fingerhut I think shutdown-agents was mentioned before, but wasn't sure whether you had tried it. If your code calls future directly (or indirectly through one of several other functions), it can cause a 60-second pause at the end before the JVM exits: http://clojuredocs.org/clojure.core/future
2018:10:31 08:18:43 borkdude Right, that’s it. So doall + shutdown-agents it is.
2018:10:30 23:56:55 andy.fingerhut e.g. pmapclojure.java.shell/sh
2018:10:31 00:45:16 misha @domkm I think you can use your own predicate in root graph spec (s/and asyclic? (s/keys ...)), which internally uses s/*recursion-limit*, but that would traverse graph twice
2018:10:31 00:47:22 domkm @U051HUZLD Thanks for the suggestion. I think I wasn't clear. The graph is cyclic. I just only want to validate to N levels and stop validating after that point.
2018:10:31 00:52:54 misha well, I got nothing then, except writing your own validator, and forfeiting all the for-free generators, etc.
2018:10:31 00:53:20 misha or, use 1 spec to generate, and custom walking predicate to validate
2018:10:31 00:54:58 misha predicate can somewhat reuse specs for nodes and edges, for keys, but it is still not pretty
2018:10:31 09:18:42 magnusdk Has anyone found a nice workaround for this: https://dev.clojure.org/jira/browse/CLJ-2067?
[spec] (s/def ::a ::b) throws unable to resolve error if ::b is not defined
2018:10:31 09:19:10 magnusdk We hack-fixed it by defining this macro:
(defmacro def*
[name spec]
`(do (when (and (qualified-keyword? ~spec)
(not (#'s/maybe-spec ~spec)))
(s/def ~spec
#(throw (ex-info "Spec is declared, but not defined!"
{:spec-name ~name
:spec-alias ~spec
:args %&}))))
(s/def ~name ~spec)))
(def* ::foo ::NaS) ;;=> :my.spec/abc
(s/valid? ::foo "something") ;;=> clojure.lang.ExceptionInfo: Spec is declared, but not defined!
(def* ::NaS string?) ;;=> :my.spec/NaS
(s/valid? ::foo "something") ;;=> true
And doing something similar with s/fdef, but this broke down when we started using orchestra for instrumentation.
2018:10:31 13:44:39 domkm If you can't rearrange the order of your s/defs, perhaps you can (s/def ::b any?) first and then redef it later?
2018:10:31 15:27:01 magnusdk edit: just some more context
Using a similar macro as the one above we specced a function’s :ret using a spec that hadn’t yet been defined, but we knew that it would be defined after all the namespaces had been loaded. However, when we started instrumenting the function (using orchestra) the spec was still throwing our «declared, but not defined» exception. It seems that the spec wasn’t overwritten, but describing the specs in the REPL displayed the expected results.
2018:10:31 15:34:54 magnusdk Your suggestion gets rid of the exceptions, but the spec is still not redefed 🤔 The function can now return anything because the any? spec still stands
2018:10:31 17:32:51 bmaddy This seems like a silly question, but is there a standard way of checking specs in tests? I suspect it uses (-> (stest/enumerate-namespace 'user) stest/check) and looks something like this:
2018:10:31 17:53:13 seancorfield @bmaddy Generative tests can be long-running so having them in "unit tests" that you run all the time is a bit of an anti-pattern.
2018:10:31 17:57:51 jaihindhreddy How do you guys do unit testing? Just what percentage of tests are manually written example based ones?
2018:10:31 18:13:51 bmaddy We mostly do classic unit tests. We have one file where we use clojure.test.check.clojure-test/defspec, but I think that was before spec came out. We're about 97% unit tests.
2018:10:31 17:58:06 bmaddy Hmm, so do people just remember to run a stest/check function that's in a comment after the function every time they update it? That's what I've been doing so far, but I figured there was a better way.
2018:10:31 18:10:53 bbrinck I think check can be useful when writing the function/spec, but for tests, I tend to use test.chuck and write my properties in the test
2018:10:31 18:11:52 bbrinck then I can use chuck/times to control the number of generative tests (low number during development, higher number on CI)
2018:10:31 18:13:01 bbrinck but I don’t tend to write a lot of functions that have super interesting :fn specs, I just check :args and :ret specs in tests
2018:10:31 18:13:45 bbrinck My tests will instrument my functions, then for any input I can just use s/gen to get the generator, use that with test.chuck
2018:10:31 18:22:55 jaihindhreddy I thought test.chuck was a typo. 😂
2018:10:31 18:23:27 bbrinck heh, no, it’s a really useful lib for generative testing 🙂
2018:10:31 18:27:53 seancorfield And, yeah, big +1 for test.chuck -- we love the regex string generator in it!
2018:10:31 18:30:14 seancorfield @bmaddy I think a good option is to have some scripts that run stest/check and summarize/assert the results are good, and then run those scripts directly as part of either periodic manual or full-suite testing (rather than automated/unit test level stuff).
2018:10:31 18:32:47 seancorfield At work we have a few generative tests that run in our "mainline" automated/unit tests, and then we have some more extensive generative/property tests that are in (comment ...) forms in various files that we currently run manually, from time-to-time.
2018:10:31 21:38:07 borkdude maybe changing the clojure version of the alias stc to clojure.test.check makes more sense than the in-ns thing in clojure.spec.test.alpha
2018:10:31 22:02:57 borkdude Ah I see why that isn’t the case, because clojure.test.check is lazily loaded (optional dep).
2018:10:31 22:04:35 borkdude Would it be OK if clojure.spec.alpha created an empty ns in a file, instead of the in-ns?
Then both requires for cljs and clj could look the same. This would be nice in .cljc.
2018:10:31 22:23:25 borkdude Posted an issue about this at JIRA
2018:11:01 15:22:16 borkdude Is nil considered valid input for Clojure set functions? 0 arity returns empty set, but 1 arity with nil returns nil
2018:11:01 15:22:44 borkdude In other words should a spec for those functions account for nil input or throw
2018:11:01 15:45:23 seancorfield @borkdude Could you show some code? I'm not sure what you're asking.
2018:11:01 15:50:32 borkdude (set/union) vs (set/union nil), they differ in return type
2018:11:01 15:55:35 seancorfield set/union is only defined for arguments that are sets tho', right?
2018:11:01 15:56:54 seancorfield (and nil is not a set -- so it's "garbage-in, garbage-out" here?)
2018:11:01 15:58:02 seancorfield So I guess the answer to your question is "No, a spec for those functions should not allow nil as input".
2018:11:01 16:16:55 borkdude If we can agree for union etc. that 1) :ret should be set? then we must either 2) reject nil inputs in the :args spec, or 3) assert that the implementation is wrong
2018:11:01 16:18:53 borkdude maybe @alexmiller can say something on this
2018:11:01 16:54:49 alexmiller don’t know
2018:11:01 16:55:39 borkdude would we find it useful for fdefs detect nil (as an instance of non-sets) inputs for set fns?
2018:11:01 16:57:31 borkdude or can we say this is undefined territory and assume the simpler spec
2018:11:01 16:57:36 alexmiller I think right now I would treat both inputs and output as nilable
2018:11:01 16:57:48 alexmiller as existing code may be relying on the behavior of those things
2018:11:01 16:58:54 borkdude if we don’t allow nil, we will find out 🙂
2018:11:01 16:59:12 alexmiller if your spec fails on working existing code, I think your spec is wrong
2018:11:01 16:59:32 borkdude true, but I mean, we can actually discover if people use this in the wild and then adapt the spec accordingly
2018:11:01 17:00:50 alexmiller well, disagree
2018:11:01 17:01:13 borkdude yes…?
2018:11:01 17:01:47 alexmiller I don’t know how else to say it
2018:11:01 17:02:41 alexmiller you asked for my opinion. My opinion is that you should spec the inputs and output as nilable.
2018:11:01 17:03:14 borkdude yes, I wondered why you disagree with adapting the spec gradually when we discover that people actually use those fns like that
2018:11:01 17:03:33 borkdude but maybe we can rightaway assume people do this
2018:11:01 17:04:29 alexmiller nils are often used interchangeably with empty collections. it seems unlikely to me that there is not some code relying on this either for input or output
2018:11:01 17:04:59 borkdude right.
2018:11:01 17:08:39 borkdude next case…
(clojure.set/union 3)
2018:11:01 17:09:00 borkdude the identity function….
2018:11:01 17:09:49 borkdude I think it’s safe to assume people do not use the 1-arity as the identity function 😉
2018:11:01 17:09:52 alexmiller I think you can figure that one out
2018:11:01 17:11:07 borkdude thanks for the replies
2018:11:01 20:39:28 danielcompton Is there a recommended way of running stest/check in a deftest? I looked around and couldn't see anything official, just lots of different ways people did it
2018:11:01 20:40:58 borkdude @danielcompton if there is I’d like to know, because I’m doing that right nw
2018:11:01 20:48:08 borkdude @danielcompton I’m trying to write tests for clojure, cljs and self-hosted cljs all within one .cljc file. Guess what, all three environment expect and return different keys for the clojure.test.check opts/rets.
2018:11:01 20:48:28 danielcompton Ouch
2018:11:01 20:53:21 bmaddy @borkdude, @danielcompton There was a short discussion about that yesterday. People suggested using test.chuck (not a typo) or separating your property based tests out and run them separately.
2018:11:01 20:53:52 bmaddy This is mainly because I'm curious, but does anyone now why the args passed to :fn in s/fdef are conformed values? I'd like to use the actual value in there, but dealing with the conformed value is tricky.
2018:11:01 20:54:09 borkdude @bmaddy does test.chuck support .cljc?
2018:11:01 20:54:53 bmaddy I've never used it, but the readme says this in the Acknowledgements section: @nberger for adapting to cljc format for ClojureScript, and general maintenance help
2018:11:01 20:55:59 andy.fingerhut @borkdude Fine detail nit on your earlier questions -- in the clojure.data/diff implementation there are calls at least clojure.set/union, and perhaps a couple of other clojure.set functions, with sequences as arguments (the return value of the clojure.core/keys function IIRC), so not a set and not nil. I believe the implementation of clojure.set/union for the ways they are called from always return correct results (i.e. duplicates in the return value are only harmful for performance, not correctness of the clojure.data/diff results). Not sure if that is considered a bug or something that an 'official' spec for clojure.set/union should allow. https://dev.clojure.org/jira/browse/CLJ-1087 I am an interested observer of such detailed questions, too, not a policy maker.
2018:11:04 15:09:57 roklenarcic I'm writing a macro where I want to turn the passed in symbol/keyword into a spec. How do I do that?
2018:11:04 15:09:57 roklenarcic I'm writing a macro where I want to turn the passed in symbol/keyword into a spec. How do I do that?
2018:11:04 15:11:02 taylor s/get-spec if you’re trying to look up an existing spec by symbol/keyword
2018:11:04 15:12:07 roklenarcic yeah but that doesn't work for predicates
2018:11:04 15:12:14 roklenarcic like someone passes in string?
2018:11:04 15:12:40 roklenarcic s/get-spec of course says that no such spec has been registered
2018:11:04 15:14:19 borkdude Does creating a ::string spec work? @U08EKSQMS just made a PR to speculative for a lot of those
2018:11:04 15:17:12 taylor it sounds like you want to check if the symbol resolves to a function (predicate), and then make a spec out of that function?
2018:11:04 15:17:29 roklenarcic The thing is that it's a library so I don't know what people will be passing in.
2018:11:04 15:17:34 roklenarcic Yes something like that
2018:11:04 15:18:18 roklenarcic creating spec out of a symbol has the benefit of having that symbol show up in description of the spec failure
2018:11:04 15:33:37 taylor this doesn’t really help, just an observation re: names/descriptions in failures:
2018:11:04 15:43:05 taylor maybe something like that would work
2018:11:04 15:43:27 roklenarcic cool I'll check it out
2018:11:04 17:52:32 misha why do you need it in the first place? to have named things in explain's out? @U66G3SGP5
2018:11:04 18:30:50 roklenarcic yes 🙂
2018:11:04 23:47:05 didibus I'm confused, sounds like you just want to call (s/def). It takes a keyword or symbol and registers a spec for it.
2018:11:06 11:38:11 roklenarcic is there a way to spec map's key+value, but the key isn't specific
2018:11:06 11:38:54 roklenarcic like how do I say: map key is symbol or keyword, if key is symbol then value must be string but if key is keyword then value must be integer
2018:11:06 11:39:48 roklenarcic I tried s/coll-of :kind map? but that only conforms value
2018:11:06 12:46:15 roklenarcic see... now I lost a term
2018:11:06 12:46:40 roklenarcic input map has 3 key-values
2018:11:06 12:46:46 roklenarcic result has two items
2018:11:06 12:46:47 borkdude ah I see
2018:11:06 12:47:25 borkdude so maybe every will help there?
2018:11:06 12:48:02 roklenarcic nah, I'll just switch back
2018:11:06 12:48:05 roklenarcic to :kind map?
2018:11:06 12:48:10 roklenarcic and take vals
2018:11:06 12:48:21 roklenarcic problem solved
2018:11:06 12:48:25 borkdude if I had time I would help you figure it out, but I have to go now
2018:11:06 12:48:49 roklenarcic don't worry, I have a solution, I was just looking for something more elegant
2018:11:06 12:50:35 borkdude yes, (s/conform (s/every (s/or :s1 (s/tuple symbol? string?) :s2 (s/tuple keyword? int?))) '{a "A" :b 3 c "B"}) works
2018:11:06 12:51:00 roklenarcic every doesn't do any conforming at all FYI
2018:11:06 12:51:05 roklenarcic at least I think so
2018:11:06 12:51:07 borkdude oh you want conforming
2018:11:06 12:51:12 borkdude sorry 🙂
2018:11:06 12:51:35 roklenarcic (s/coll-of x :kind map?), conforms key+value into value
2018:11:06 12:52:04 roklenarcic so you just have to take (vals ) of value in the map then
2018:11:06 12:52:46 borkdude @roklenarcic
(s/conform (s/cat :map-entries (s/* (s/alt :s1 (s/tuple symbol? string?) :s2 (s/tuple keyword? int?)))) '{a "A" :b 3 c "B"})
2018:11:06 14:00:12 alexmiller I don’t think this approach will work on latest spec - regex specs now only work on sequential collections (too confusingly dangerous to do sequential spec’ing on unordered collections)
2018:11:06 14:02:20 alexmiller (s/conform (s/coll-of (s/or :s1 (s/tuple symbol? string?) :s2 (s/tuple keyword? int?))) '{a "A" :b 3 c "B"})
2018:11:06 14:02:24 alexmiller would be better
2018:11:06 14:04:27 borkdude that was already proposed by me, but that didn’t work, because the result would be: {:s1 [c "B"], :s2 [:b 3]} and you lose map-entries
2018:11:06 14:04:48 borkdude I guess you can call seq on the map first if it doesn’t work with the newest spec
2018:11:06 14:07:33 borkdude every also works, but that doesn’t conform and the OP wanted that
I would expect the additional cat to require additional wrapping but it appears that ::entrty-x and (spec/cat :entry ::entry-x) are identical for some reason.
2018:11:06 13:46:27 borkdude I guess you can nest s/cat?
2018:11:06 16:07:40 ghadi it is, thanks @U06BE1L6T
2018:11:06 18:06:03 alexmiller yep, thx
2018:11:06 17:27:17 eoliphant Hi, I'm trying to get a feel for where to 'draw the line' on more dynamic behavior in specs. I've created an EDN based data description language for one of my projects. I have it spec'ed out pretty thoroughly, but I have a situation where I have say :data/type that could be one of a fixed set of types (:string :long, etc) or a type that's defined elsewhere in the larger data structure I'm validating. my code is already doing some additional validation based on core.logic etc once the definitions are loaded. So just wondering if it's better to defer this type of check to that point
2018:11:06 18:20:20 mathpunk Is there an equivalent of ns-unmap for a symbol you used spec/def on? (I suspect this question will reveal an error in my mental model of how symbols and specs relate)
2018:11:06 18:21:34 didibus Just s/def it to nil
2018:11:06 18:21:51 mathpunk 🆒 thanks!
2018:11:06 18:21:59 didibus If I remember correctly
2018:11:06 19:15:26 alexmiller yes
2018:11:07 13:38:34 danielstockton Not sure how to achieve something. If I want to check a map for :key, but define the spec with ::some-type-of-key (different spec name) in (s/keys :req-un [::key])?
2018:11:07 13:39:20 danielstockton More concretely, I have a :type key on multiple things, and the spec is slightly different for each.
2018:11:07 13:39:20 danielstockton More concretely, I have a :type key on multiple things, and the spec is slightly different for each.
2018:11:07 13:46:07 jumar You should probably use namespace-qualified keys in that case (that is :req)
You can also use a level of indirection and define specs like this:
(s/def :a/type int?)
(s/def ::a-type (s/keys :req-un [:a/type]))
(s/def :b/type string?)
(s/def ::b-type (s/keys :req-un [:b/type]))
(def int {:type 10})
(def b {:type "number"})
(s/valid? ::a-type a)
;; => true
(s/valid? ::a-type b)
;; => false
(s/valid? ::b-type b)
;; => true
(s/valid? ::b-type a)
;; => false
2018:11:07 19:35:24 shaun-mahood @alexmiller: I'm doing my first really rigorous spec of something with a lot of data, and this morning I'm on my 4th or 5th time understanding why a certain design decision makes more sense than I realized with either smaller data or less complete specs. It's pretty great to be using something with so much depth of thought and experience behind it!
2018:11:08 09:51:57 danielstockton Can anyone suggest how i might spec that a vector should contain maps in a particular sort order? (based on a key)
2018:11:08 09:53:27 danielstockton It might contain any number of maps, but they should be in a particular order based on a :type key.
2018:11:08 10:37:47 borkdude @danielcompton use this predicate?
Does it matter if the returned value is simply truthy rather than true?
2018:11:08 11:00:30 borkdude depends what you want. it’s a convention to return booleans from ? functions.
2018:11:08 11:01:07 borkdude note that you’re using > which doesn’t work for the case when you have equal elements with regards to k
2018:11:08 11:01:34 borkdude stylistically: correctly- is a bit redundant, it’s either sorted, or not
2018:11:08 11:03:06 danielstockton Thanks, you're right on all fronts..
2018:11:08 11:04:01 borkdude haha, didn’t mean to be pedantic, but couldn’t help 😉
2018:11:08 11:04:14 danielstockton Problem is sorted? is a core function
2018:11:08 11:04:32 borkdude sorted-by? isn’t
2018:11:08 11:05:02 borkdude but this is just naming, choose whatever you want 😉
2018:11:08 11:05:22 danielstockton sorted-accordingly?😛sorted-by? is just fine
2018:11:08 11:18:30 Matt Butler Hi, I'm trying to spec a list of different maps, based on the docs I think I'm supposed to use a multi-spec, which i've done in the past without trouble. However I want to dispatch on more than one key/value, which I haven't seen an example of. Is this supported?
I'm doing this, and valid? works correctly (returns true).
But explain returns something unexpected so wanted to check im not doing something wrong
I suspect its because I've just put something random :foo/type in the retag arg.
2018:11:08 12:36:50 Matt Butler So it appears that multispec doesn't conform the set to a spec for you, which was causing the inconsistency.
Returning (s/spec #{{:bar :a :baz :b :biff :c}}) from the methods, now keeps both explain and valid happy.
2018:11:08 11:22:45 Matt Butler On a slightly unrelated note, is it considered back practice to transform data before checking it against a spec?
I have the same data but in 2 different formats and i'd prefer to not to spec each one individually.
I think I can write some fairly simple code to transform 1 into the other and then check it against a single spec. [a b c] => {:foo a :bar b :baz c}
2018:11:08 13:16:18 alexmiller Doesn’t seem inherently bad to me
2018:11:09 17:22:27 Lyn Headley Is it the case that cljs.spec.gen.alpha does not support recursive-gen? I am seeing: ^--- Use of undeclared Var cljs.spec.gen.alpha/recursive-gen
2018:11:09 20:50:50 hiredman recursive-gen is a thing that test.check has, but I don't think spec exposes a wrapped version of it
2018:11:09 20:51:31 hiredman (in cljs or clj)
2018:11:10 06:39:43 tianshu I have a question, is it s/conformer not work on s/map-of? is it designed?
2018:11:10 16:57:32 schmee I’m very confused by the behavior of instrument and fspec
2018:11:10 16:57:57 schmee long story short: how do I turn off generative checking of fspec?
2018:11:10 16:58:46 schmee *fspec-iterations* doesn’t cut the mustard since it will try to create generators even though you set it to 0
2018:11:10 16:59:39 schmee my workaround now is to copy-paste the spec in question and replace the fspecs with any? and pass that to the :spec arg to instrument, but that is not a viable solution
2018:11:10 23:52:46 didibus Are you saying a call to instrument forces the gen to be generated?
2018:11:10 23:55:46 schmee that seems to be the case
2018:11:10 23:57:21 didibus Hum, let me check if I have that behavior
2018:11:11 00:04:46 didibus I don't get that behavior
2018:11:11 00:05:14 didibus
2018:11:11 00:06:12 didibus This works fine for example. Even though if you run:
(sgen/generate (s/gen `add))
you get ExceptionInfo Unable to construct gen at: [:a] for: (= (type %) java.lang.Long) clojure.core/ex-info (core.clj:4739)
2018:11:11 00:13:58 alexmiller when an instrumented fdef with an fspec is checked, the :args spec of the fspec is generated and then the fspec function is invoked to verify it conforms to its spec
2018:11:11 00:14:17 alexmiller many people find this surprising and I expect we will ultimately change it
2018:11:11 00:19:44 schmee what do you think of my suggestion above as a temporary measure (to not call validate-fn when *fspec-iterations* is 0 or nil)?
2018:11:11 00:20:37 schmee also, I appreciate that you are considering options for this 🙂
2018:11:11 00:23:39 didibus Oh I see, you mean if you spec a higher order function on an fdef. Interesting, I don't think I ever specced one before.
2018:11:12 19:23:28 asilverman #clojure-spec - hi all, could you help me understand why
(inst? (t/date-time 1986 10 14))
=> false
2018:11:12 19:24:20 schmee I guess t/ is clj-time/?
2018:11:12 19:26:34 ghadi @ariel.silverman (class (t/date-time 1986 10 14)) would be some third-party Joda time thing, which isn't in scope of the inst? predicate
2018:11:12 19:38:46 asilverman @ferossgp @schmee - fair enough. How do I sample a joda time thing then?
2018:11:12 19:54:26 asilverman #clojure-spec - I was hoping to ask for some help generating samples for clj-time/date-time since I am currently interested in testing my spec and using the spec as a generator for unit tests. Can anyone spare a couple minutes to help or point me out to some resource that can assist me in this goal?
2018:11:12 20:02:23 schmee something like this should get you going:
2018:11:12 20:35:29 asilverman I was able to require the namespace clj-time.spec and now I am trying to figure out how to do the dynamic binding of *period*
2018:11:12 19:59:46 schmee @ariel.silverman I have a one around for that, sec…
2018:11:13 20:29:23 Dormo Do people tend to keep spec instrumentation on in testing environments? I don't mean automated testing, but environments for manual/internal testing before being deployed to production
2018:11:13 20:30:07 Dormo i guess "staging environments" would be a good term
2018:11:13 21:56:18 souenzzo I use just on automated test (lein test)
2018:11:13 21:56:49 souenzzo The "develop" jar should behave the same as "prod" jar.
2018:11:15 11:01:16 stijn How can I create a generator that either generates something of a type or nil? (e.g. (spec/gen (spec/inst-in start end)) or nil)
2018:11:15 11:04:29 borkdude @stijn the either part you can do with s/or or s/alt
2018:11:15 11:04:47 borkdude or use nilable
2018:11:15 11:05:04 stijn oh, of course 🙂
2018:11:15 11:05:11 stijn thanks!
2018:11:15 14:43:58 borkdude what’s the minimum clojure version in which spec can be used, 1.9.0?
2018:11:15 14:51:38 mpenet 1.8 if you use clojure-future-spec
2018:11:15 14:52:01 mpenet prolly lower too, but we ran on 1.8 + clojure-future-spec smoothly for a while on some services
2018:11:15 15:06:30 borkdude ah yes, now that you remind me, we did too
2018:11:15 16:19:02 Wilson Velez Hi all, Where can I start to learn spec?
2018:11:15 16:57:42 borkdude wonder what the current state of that is
2018:11:15 16:59:28 eoliphant thanks
2018:11:15 17:00:39 eoliphant yeah to your point it’s a tuple [:odd 1]
2018:11:15 18:12:03 jaihindhreddy Well, you could flip the order. (s/def ::small-even-odd (s/and #(< % 100) ::even-or-odd))
2018:11:15 18:37:55 alexmiller won’t gen then
2018:11:15 18:38:11 alexmiller actually, I guess it won’t gen regardless
2018:11:15 18:38:25 alexmiller you need an int? at the beginning or something
2018:11:15 19:31:05 devth if i have two functions in the same namespace that i want to spec with req-un with the same named arg e.g. match but different shapes, must I put those specs in other namespaces in order to specify the different shapes?
2018:11:15 19:34:31 devth since the spec must be named match and I can't specify two specs with the same name, e.g.:
2018:11:15 19:35:44 noisesmith if they are req-un why not just give them different namespaces?
2018:11:15 19:36:01 devth instead of :: you mean?
2018:11:15 19:36:15 devth like :fun1/match:fun2/match?
2018:11:15 19:36:33 noisesmith right - :: is a shorthand for the current namespace
2018:11:15 19:37:06 devth ok. guess i wasn't sure if it's appropriate to abandon the :: convention since all my other specs in that namespace are using it 🤔
2018:11:15 19:37:44 devth almost seems like that makes them private/internal specs
2018:11:15 19:38:32 alexmiller it’s ok and fine to do that
2018:11:15 19:38:46 devth got it. thanks :+1:
2018:11:15 19:38:51 alexmiller fyi, changes coming in this area for next version of spec that should make it less weird
2018:11:15 19:38:57 noisesmith I've never seen any assumption or convention that specs that don't use the current namespace are private
2018:11:15 19:42:23 borkdude another way to mark them private is to move them to an foo/impl.clj namespace 🙂
2018:11:15 19:42:43 devth is that a java thing? 🙂
2018:11:15 19:43:01 borkdude pretty general convention
2018:11:15 19:43:17 noisesmith it's a convention in many clojure projects, if the ns has impl as part of its path you know you aren't expected to rely on it as a consumer
2018:11:15 19:43:27 alexmiller core.async is a good example
2018:11:15 19:43:32 devth ah, cool
2018:11:15 19:44:52 seancorfield @alexmiller Are you aware that clojure.spec-alpha2.test/check is "broken" at the moment?
2018:11:15 19:45:10 alexmiller yes
2018:11:15 19:45:25 alexmiller and I will fix it as soon as I finish getting the next Clojure released :)
2018:11:15 19:45:30 alexmiller beta that is
2018:11:15 19:45:46 seancorfield 'kthx
2018:11:15 19:46:16 seancorfield I tried to fix it locally but I haven't quiet gotten my head around the path through the code to figure out exactly what needs changing... yet...
2018:11:15 19:47:18 alexmiller I did not intentionally change anything around this, so it’s something I (unintentionally) broke, but haven’t looked yet
2018:11:15 19:50:48 seancorfield I think it's because stest/check still, ultimately, calls s/spec (a macro) and this line in check-1 no longer behaves the same as before:
specd (s/spec spec)]
2018:11:15 19:51:26 seancorfield (I just haven't quite figured out the correct invocation to replace that with)
2018:11:15 20:05:48 alexmiller prob a bug in the s/spec impl
2018:11:15 21:20:57 seancorfield FYI, I tried switching our code over to clojure.spec-alpha2 because, why not? I ran into two speed bumps (so far)...
2018:11:15 21:21:47 seancorfield 1. we reuse :clojure.core.specs.alpha/binding in one of our specs (I got around that by using eval to s/def that spec into the new spec's registry).
2018:11:15 21:22:38 seancorfield 2. we use A spec that uses test.chuck fails with
Caused by: java.lang.IllegalArgumentException: No method in multimethod 'create-spec' for dispatch value: ws.domain.member/bounded-string
2018:11:15 21:31:48 seancorfield Not yet sure what causes that.
(defn bounded-string
"Given min/max lengths, return a spec for a string within
those lengths (inclusive).)"
[from to]
(s/and string? #(<= from (count %) to)))
2018:11:15 21:36:02 seancorfield I have a feeling this is related to the stest/check failure since it seems to be due to an assumption that a form like (a-fn ...) can be dispatched on the first argument, rather than evaluating it and dispatching on the result?
2018:11:15 21:38:03 alexmiller yeah, might be missing a case there
2018:11:15 21:55:07 seancorfield If it's any help, I tried this
(defn bounded-string
"Given min/max lengths, return a spec for a string within
those lengths (inclusive).)"
[from to]
(s/and string? #(<= from (count %) to)))
(def ^:private email-bounded-string? (bounded-string 5 128))
and it failed with
Exception in thread "main" Syntax error compiling at (ws/domain/member.clj:87:1).
Unable to resolve symbol: from in this context
so at this point I'll give up 🙂
2018:11:16 13:51:24 stijn is there a way to do something like this (s/def ::string-or-number (apply s/or [:string string? :number number?])) or is it macros to the rescue for e.g. creating a combined spec from a list of regexes?
2018:11:16 13:55:06 mpenet you can do this with eval
2018:11:16 13:56:22 mpenet
(eval `(s/or
2018:11:16 13:56:24 mpenet etc
2018:11:16 13:57:16 stijn @mpenet thanks!
2018:11:16 13:59:52 borkdude @stijn apparently (s/def ::string-or-number (s/or # also works
2018:11:16 14:08:20 alexmiller this stuff is better in spec-alpha2
2018:11:16 14:08:36 alexmiller but don’t have time to talk about it
2018:11:16 14:09:05 mpenet yep hairy
2018:11:16 14:09:11 mpenet eager to see the new stuff
2018:11:16 14:25:16 mhuebert When using fdef, are the arguments validated by :args supposed to be affected by the destructuring forms present in the function argslist?
2018:11:16 14:26:29 taylor no
2018:11:16 14:26:21 mhuebert eg. in the following, the fact that I am using & [doc] to read the 2nd arg seems to affect how instrument works
2018:11:16 14:28:47 mhuebert I think I found that same example in my searching
2018:11:16 14:29:18 taylor I think your example specs a fixed, 1- or 2-arity function instead of a variadic one that just destructures the first add'l arg
2018:11:16 14:30:30 taylor which is why it works when you rewrite the function with fixed 1- & 2-arity
2018:11:16 14:33:43 mhuebert instrument also fails when I try to use it with your slarp example
2018:11:16 14:41:04 taylor FWIW I'd go with the fixed-arity version regardless of CLJS spec bug :man-shrugging:
2018:11:16 14:41:17 mhuebert why?
2018:11:16 14:41:39 taylor it's less ambiguous about how many args the caller can pass
2018:11:16 14:42:20 mhuebert the actual use case is more like (defn register-key [key & [doc options data]) where both doc and options are optional
2018:11:16 14:43:14 mhuebert i’d like to have them show up in argslist (for eldoc etc) but in smaller arities it isn’t known whether the 2nd option would be doc, options, or data
2018:11:16 14:43:46 mhuebert but i just started writing in this way recently and wasn’t sure if it is a great idea
2018:11:16 14:44:03 taylor I might also consider kwargs-style variadic function in that case too
2018:11:16 15:42:07 borkdude @pablore for now we use ifn? in speculative, it’s a little (too) general though…
2018:11:16 15:44:07 pablore Can I still specify args and ret?
2018:11:16 16:50:19 jimbob do i need to require the namespace of the specs im using?
2018:11:16 16:50:30 jimbob example if in my ns i have :blah.spec/reward
2018:11:16 16:50:37 jimbob do i need to require blah.spec?
2018:11:16 16:50:42 jimbob in the ns that calls it?
2018:11:16 16:53:16 borkdude @ben.borders no, but you need to call the ns in which the spec was defined to be able to use it
2018:11:16 17:00:02 jimbob thanks!
2018:11:16 17:08:05 jaihindhreddy I want to study the spec source and understand it. Should I study spec.alpha or spec-alpha2?
2018:11:16 17:16:43 seancorfield @jaihindh.reddy The only difference between those right now is the latter has the macro implementations refactored out as functions, that the macros then call.
2018:11:16 17:17:53 seancorfield There are several bugs in spec-alpha2 right now preventing it from actually being used but the code itself is fine to read for learning purposes... although I'm not quite sure what you'll really get from the source, as opposed to the guide and reference material...?
2018:11:16 17:18:53 seancorfield (like many parts of Clojure itself, the source is kinda gnarly and does not represent "typical" Clojure code nor, often, "best practice"... and that's fairly typical of a compiler and its runtime/standard library)
2018:11:16 17:28:52 jaihindhreddy I've recently started to read the Clojure source. Wanted to do that for spec too.
Gotta say, it is pretty gnarly.
2018:11:16 17:41:47 jimbob strange
2018:11:16 17:41:51 jimbob this doesnt make sense to me
2018:11:16 17:41:58 jimbob got a
java.lang.Exception: Unable to resolve spec: :facts.rewards.specs.reward-award/award
2018:11:16 17:42:09 jimbob but then i explicitly required the namespace
2018:11:16 17:42:15 jimbob
[facts.rewards.specs.reward-award]
2018:11:16 17:42:17 jimbob and now it works
2018:11:16 17:42:30 jimbob but other specs do not require the spec namespace to be required?
2018:11:16 17:44:13 schmee which specs are you referring to as “other specs”?
2018:11:16 17:48:36 jimbob the other specs i have defined.. i dont want to enumerate all of them, but they dont require the namespace of the spec they are calling to be explitly required
2018:11:16 17:54:55 schmee maybe those namespaces are required somewhere else in the code?
2018:11:16 17:55:17 schmee specs use a global registry, so as soon as a namespace is required in on place it will populate the registry with all its specs
2018:11:16 17:58:48 jimbob ah so it needs to be required somewhere?
2018:11:16 17:59:01 jimbob otherwise its not populated in the registry
2018:11:16 17:59:19 jimbob so i must not be requiring the ns then in src code
2018:11:17 07:02:12 ikitommi Is there any high-level backlog or set of goals for the spec-alpha2?
2018:11:17 22:47:32 seancorfield Based on the changes so far (compared to spec.alpha), it adds a function-based interface to allow programmatic construction of specs. Which people have asked for a lot.
2018:11:19 09:45:52 stijn how do people practically deal with entity maps that have required keys in one situation, but don't in another situation?
2018:11:19 09:48:03 didibus Multi-spec
2018:11:19 09:46:48 stijn I understand you'll probably have to create a separate spec, but that might require duplicating the entire tree of references to other entities, no?
2018:11:19 09:50:37 borkdude @stijn best to come up with an example
2018:11:19 10:17:34 pyr Hi, I'm trying to spec a map (over which I dont' have control) which uses an inconvenient idiom: for a number of members it accepts either a :membername or :memberid key, but at least one must be present. If there's a single member like this, spec/or is convenient.
I naively started producing a number of spec/or specs (of the sort: (spec/def :member (spec/or :by-id (spec/keys :req-un [:member/memberid]) :by-name (spec/keys :req-un [:member/membername])))) but if I merge those with spec/merge or spec/and validation becomes impossible
2018:11:19 10:19:21 pyr Let's say that I have (spec/merge ::member1 ::member2) The input for validation of member2 seems to be the output of conform for member1, which becomes a tuple [:by-id val] instead of plain val. Beyond doing some sort of pre processing of the map, any idea of how to address this from spec?
2018:11:19 10:23:58 mpenet can't you use (s/keys :req-un [(or ::membername ::memberid)]) ?
2018:11:19 10:24:48 pyr or as in spec/or?
2018:11:19 10:24:58 pyr if so, I didn't try, let me try it
2018:11:19 10:25:03 mpenet as in or within keys
2018:11:19 10:25:12 mpenet it supports that
2018:11:19 10:31:21 mpenet it also supports and, kind of a cool feature
2018:11:19 10:31:29 mpenet from the docstring: (s/keys :req [::x ::y (or ::secret (and ::user ::pwd))] :opt [::z])
2018:11:19 10:49:13 pyr @mpenet thanks, never knew about that
2018:11:19 10:58:48 borkdude I see the docstring, so that’s definitely supported
2018:11:19 10:59:01 mpenet I had typos in my example, I am fairly sure it works with both req-un or req (not in front of laptop atm)
2018:11:19 10:59:21 pyr ah got it
2018:11:19 10:59:31 pyr I left one in opt-*
2018:11:19 11:00:52 pyr perfect, thanks @mpenet 🙂
2018:11:19 12:52:32 stijn @borkdude example would be: a Person entity in one case has required fields :person/name:person/birth-date:person/address, and Address has required fields :address/street:address/city, but in another case I want a person with required keys :person/social-security-number and :person/address, but the Address has no required fields. How do you model that? I can define a spec for Person type 1 and one for Person type 2, but both will have a required key :person/address, but the spec for :person/address is different in these two entity map specs.
2018:11:19 15:20:24 alexmiller I think it’s not expensive in general, it’s expensive to have spec for things used by Clojure to do everything else
2018:11:19 15:20:35 alexmiller because the validation is then also checked during validation
2018:11:19 15:21:01 borkdude to be clear, I spec’ed a function like assoc, not assoc itself
2018:11:19 15:21:26 borkdude so I made sure that it wouldn’t be used internally
2018:11:19 15:21:45 borkdude (def my-assoc assoc) and then spec’ed my-assoc
2018:11:19 15:23:04 alexmiller given that you’re doing these a million times each, they seem pretty fast to me? you’re talking about microseconds per?
2018:11:19 15:23:32 alexmiller if you could provide a narrative rather than just a wall of data, that would help
2018:11:19 15:24:06 borkdude right. I instrumented an Advent of Code puzzle and the time went from 3s to 15 minutes. That seemed a little but much for instrumentation overhead.
2018:11:19 15:24:46 borkdude but maybe that’s just the way it is
2018:11:19 15:24:51 alexmiller well, it depends what it’s doing and what you spec’ed
2018:11:19 15:25:03 alexmiller if you instrument something in a hot loop…. then that will be slow
2018:11:19 15:25:13 alexmiller was an effort to compare different fws
2018:11:19 15:25:36 alexmiller at a micro level and spec did pretty well there
2018:11:20 06:14:47 didibus @borkdude 's benchmark is showing that instrument is much slower then valid? The benchmark you linked used valid?
Have an idea why instrument would be slower then valid?
2018:11:20 06:30:43 borkdude Not much slower. It does a little bit more.
2018:11:20 06:49:12 didibus Right, from the impl, it does seem it uses conform to validate. Even though it doesn't seem to do anything with the conformed value. I wonder why its not just using valid?
2018:11:20 06:53:33 didibus Oh, valid also uses conform. Ok, well I'm learning a few things looking at the impl 😛
2018:11:19 15:26:09 rickmoynihan What are the big changes in spec-alpha2? I’m assuming there are breaking changes, given the artifact has changed names.
2018:11:19 15:27:01 alexmiller at the moment, the public api is the same
2018:11:19 15:27:14 alexmiller there are some additions and likely breaking changes down the line
2018:11:19 15:28:16 rickmoynihan ok so the repackaging is mostly precautionary? i.e. in case you were to push a breaking change?
2018:11:19 15:29:33 alexmiller it is currently a work in progress, we’ll have more to talk about it as we get to a release point
2018:11:19 15:30:09 rickmoynihan Understood.
2018:11:19 15:26:44 borkdude cool, thanks. yes, the code was a hot loop. I also tested with spec-alpha2 and saw about the same numbers.
2018:11:19 15:27:53 borkdude is spec itself also compiled with direct linking?
2018:11:19 15:28:01 alexmiller no
2018:11:19 15:28:04 alexmiller it’s not compiled
2018:11:19 15:28:10 borkdude ah, that explains something
2018:11:19 15:29:02 alexmiller there’s some trickiness in the circularity between clojure and spec.alpha
2018:11:19 15:29:14 alexmiller and compilation makes that trickier than it’s worth
2018:11:19 15:29:26 borkdude no problem. thanks for the info
2018:11:20 00:36:51 lwhorton is it possible to express “a map with either key A or key B, but not neither”?
2018:11:20 00:38:37 lwhorton (without unioning the value of the key under something like s/keys :req [:a/foo] where s/def :a/foo (s/or :b :a/bar :baz :a/baz) (since i want the key’s value to be either :bar or :bar and not :foo)
2018:11:20 08:41:48 roklenarcic Is there any difference between: (s/& (s/cat .... and (s/and (s/cat ...
2018:11:20 12:38:54 alexmiller & is a regex op that nests with other regex ops
2018:11:20 08:47:43 Cheng I read some spec impl code, I see it using a global store for these specs. I think spec is more of meta data of data, why it cannot implemented via meta data? Make it a meta system?
2018:11:20 09:01:00 roklenarcic metadata of what?
2018:11:20 09:02:25 roklenarcic When I define a spec for a keyword, where exactly would you attach the metadata?
2018:11:20 09:06:51 schmee @roklenarcic s/& is for spec regexes while s/and is for general predicates
2018:11:20 09:08:21 didibus Well, specs are not tied to a var, so meta wouldn't really make sense. They are tied to values. The keyword or symbol is just a name for the spec
2018:11:20 09:09:01 schmee fspecs are tied to vars
2018:11:20 09:09:22 didibus No they're not
2018:11:20 09:09:24 schmee or, it can be at least 🙂
2018:11:20 09:09:46 didibus Well, you can relate them
2018:11:20 09:12:26 didibus That's why check-fn takes a function and a spec seperate. Because you could use any fspec to spec to check any function
2018:11:20 09:14:10 schmee yeah, I was thinking of s/fdef
2018:11:20 09:15:19 didibus Ya, I mean, in practice, for functions, its convenient to use the same symbol to point to your function and its spec.
2018:11:20 09:15:42 didibus So in that case, you could argue Specs could have just been meta on the function
2018:11:20 09:15:53 didibus Or its var
2018:11:20 09:16:05 didibus But, when your speccing data, that wouldn't have made sense
2018:11:20 09:16:32 didibus So that's why a registry is more logical
2018:11:20 09:17:04 didibus Also, now you can define specs before you define functions.
2018:11:20 09:43:03 Cheng @schmee i am in china, now i cannot open that url, i will check that article later, thanks for the information.
2018:11:20 09:43:34 Cheng i think spec is meta data in essence, whatever it currently implemented via meta mechanical or not.
2018:11:20 09:45:21 Cheng i think the problem is keyword should support meta data as we are associating meta data to it.
2018:11:20 12:41:07 alexmiller Keywords are interned and reused so cant support metadata
2018:11:20 09:46:44 Cheng i think now it’s a workaround to impl meta data in global store, is is really a good practice to do so?
2018:11:20 09:49:01 didibus It depends what the meta is about. Like if you want to specify additional things about a Var then you can add that to the Var meta. Same for anything that supports meta.
2018:11:20 09:49:06 didibus But Spec is meta about values
2018:11:20 09:49:24 didibus And values can not have meta attached to them directly. A value can be an int for example
2018:11:20 09:50:38 didibus So for example, this is a spec #(> % 10)
2018:11:20 09:51:29 didibus It doesn't need to be attached to anything to be one
2018:11:20 09:52:03 didibus So say you want to give that spec a name, you can do s/def ::greater-then-ten #(> % 10)
2018:11:20 09:52:41 didibus You did not add meta to the keyword ::greater-then-ten here. This spec is not metadata about the keyword.
2018:11:20 09:53:18 didibus It is just a specification of values that you named ::greater-then-ten
2018:11:20 09:53:55 didibus Now you can do: s/valid? ::greater-then-ten 15) for example
2018:11:20 09:55:33 didibus So I'm not sure you should think of Specs as meta data. It isn't really meta data in the normal sense. Like, yes, in this case, you can say that 15 conforms to the ::greater-then-ten spec. And so, ::greater-then-ten can be seen as metadata about the value 15. But again, meta about values is not really a concrete thing.
2018:11:20 10:01:45 Cheng @didibus thx for the explanation
2018:11:20 10:06:34 Cheng in my opinion there are plenty of problems about current spec impl, for example: global store for spec, use keyword as spec name, mix keyword with spec, instrument is a state changer...
2018:11:20 10:07:07 Cheng overall, it’s not FP style
2018:11:20 10:07:28 Cheng it’s OO style
2018:11:20 10:08:16 mpenet for sure there are issues but it's not related to OO or fp at all
2018:11:20 10:09:41 mpenet vars are "global", shoving everything in them would be the same ultimately, you just move the bloat somewhere else. A separate registry is a good thing
2018:11:20 10:10:33 mpenet ask youself: where is the info about the Types "stored" (if that even makes sense) in haskell: it's not a first class thing
2018:11:20 10:10:55 borkdude even in Haskell programs they use global mutable cells, IORef, MVar, TVar, etc.
2018:11:20 10:12:50 mpenet spec-alpha2 is supposed to fix quite a bit of the common complains, so I guess "wait and see"
2018:11:20 10:20:22 didibus • The global store has advantages too. It is simpler to manage a single global store, and namespaces should avoid clashes.
• You don't have to use keywords for spec names, you can also use symbols. But I think keywords was chosen as convention specifically to make it visually separate from Vars
2018:11:20 10:21:35 didibus * Instrument is used for testing only. And, it shouldn't be altering state. Think of it as aspect oriented programming if you know about that.
2018:11:20 10:28:42 Cheng instrument directly changed the function via byte code operate, i hope instrument can return a new function not directly change on existing function
2018:11:20 10:29:39 Cheng i didn’t aware we can use symbol instead of keyword in a/def
2018:11:20 10:30:08 borkdude instrument doesn’t change the original function, it wraps it in a new function that’s stored in the root of the var
2018:11:20 10:30:20 borkdude on unstrument this is reverted
2018:11:20 10:31:23 Cheng i don’t see it’s more easy to maintain a global store than separate vars
2018:11:20 10:32:46 borkdude an alternative could maybe have been via metadata on the var: instrumented? true but that would incur performance overhead when not using spec, because when calling a function via a var you would have to do this check
2018:11:20 10:34:29 didibus Ya, instrument and unstrument also take a lock on the Var before changing it
2018:11:20 10:35:18 didibus But it does mean you can not have an instrumented fn in one thread, and an uninstrumented one in another. Though that almost sounds like a positive in this case.
2018:11:20 10:37:22 borkdude you can have with-instrument-disabled per thread maybe, since it’s a dynamic var?
2018:11:20 10:40:38 didibus Oh, that's true, I just saw the instrumented code checks for a dynamic var to decide to validate or not. So I guess you could in fact do that.
2018:11:20 10:42:12 didibus Hum, that gives me an idea for when I alter-var-root things in my tests. I could instrument things to be "mockable", and then have a dynamic binding pointing to a mock, and this different mocks per thread and my tests would work in parallel.
2018:11:20 10:38:38 didibus @xfcjscn909 If you wanted non global stores, you'd have to first create a spec registry and put it somewhere in scope. And if it wasn't in scope anymore, it would stop working. And then as soon as you have more than one, you might forget which registry holds the spec you are looking for. All spec functions would take an additional arg for the registry to use. I mean, it would not be terrible, but I feel it would be a bit more inconvenient. And in practice, I never felt the need for separate registry. That's what namesapces are for on symbols and keywords.
2018:11:20 10:39:32 didibus In effect, you would be using a Var as a namespace mechanism, instead of just using the existing namespace mechanism of keyword and symbols I feel.
2018:11:20 10:39:36 borkdude Clojure apps pretty much converged to “one global atom” instead of “many smaller atoms”, e.g. re-frame
2018:11:20 10:39:40 borkdude it’s much easier to reason about
2018:11:20 10:44:06 didibus Also I'd say, if you don't find that top level defs are a problem for defining Vars, in practice, Specs are defined in a similar fashion, so it'll be a similar experience.
2018:11:20 10:53:53 borkdude the thing that would maybe problematic is if multiple libraries define specs for the same thing. e.g. lib1 defines a spec for clojure.core/merge and lib2 does too. currently you have no way to say: I want to exclude loading this fdef when requiring a namespace
2018:11:20 10:54:19 borkdude last one wins
2018:11:20 11:06:36 didibus Hum, right. I think I tend to use Spec much more around my data, and a lot less to spec functions. So I haven't faced these kind of issues very much.
2018:11:20 11:08:44 didibus Not yet at least
2018:11:20 11:09:54 borkdude it hasn’t been a major problem, but it would be useful to me to see from which namespace a spec comes from and to conditionally instrument depending on the namespace where it came from.
2018:11:20 11:12:35 didibus Hum, interesting. fdef could probably capture that
2018:11:20 11:13:07 didibus Probably was not a feature they thought of
2018:11:20 11:13:11 borkdude yeah and it could store it in the registry
2018:11:20 11:15:47 borkdude but maybe in a future version of spec specs will have some form of metadata, so it could also happen there
2018:11:20 11:16:06 didibus Ya, meta on spec would be great.
2018:11:20 12:42:55 alexmiller It’s on the list
2018:11:20 15:39:37 martinklepsch Is there a way to define a spec that matches the structure of a conformed value?
2018:11:20 15:41:02 martinklepsch E.g. (s/cat :a string? :b string?) would conform to {:a "a" :b "b"}. Now I want to instrument a function so that it only takes values having :a and :b with values matching string?
2018:11:20 15:47:25 alexmiller automatically, no
2018:11:20 16:00:00 martinklepsch ok cool, thanks. Is the general expectation that once values are conformed they are no longer passed to functions that have spec'ed arguments? (I guess in most cases conformed values won't match the original spec)
2018:11:20 16:02:08 alexmiller correct
2018:11:20 22:56:11 Dormo Is there a way to make a spec to check if a vector has either 1 or 2 values, where the first value is a keyword and the second value is "anything or nothing"?
I was thinking this would work, but it does not: (s/def ::event (s/tuple keyword? (s/nilable any?)))
2018:11:20 22:57:36 alexmiller tuples are fixed size
2018:11:20 23:00:02 Dormo The explicit key names are actually a bonus as well.
2018:11:20 23:01:11 alexmiller the regex ops cat, alt, +, *, ?, etc are regex ops used to describe the structure of a sequential collection
2018:11:23 00:29:22 rgm is there a way to fspec that, say, 250+ functions all conform to the same shape? (Apologies if this is in the docs; my scanning and google-fu don’t quite know how to phrase my question quite right).
2018:11:23 00:29:54 rgm I guess I want something like (s/fdef-and-register-a-name ::function-shape ,,,)
2018:11:23 00:30:32 rgm and then loop over all my function names as (s/fdef-by-name my-var ::function-shape)
2018:11:23 00:31:47 rgm or maybe it’s just (map #(apply s/fdef % common-shape) ('my 'coll 'of 'function 'names))
2018:11:23 00:31:56 rgm (nvm, I think I just rolled my own)
2018:11:23 00:32:31 rgm (assuming common-shape is just '(:args ,,, :ret ,,, :fn ,,,) )
2018:11:23 00:34:18 rgm (argh, s/fdef is a macro not a fn)
2018:11:23 01:58:47 taylor @rgm maybe something like this would work:
2018:11:27 00:49:41 richiardiandrea hello folks, it there a predicate for non-nil? I want to make sure that the key is there but it is not nil and I don't care about the value - can't use any? basically
2018:11:27 00:50:27 taylor does some? work?
2018:11:27 00:50:51 richiardiandrea right 😄
2018:11:27 00:51:11 richiardiandrea thank you @taylor forgot about that one 🙂
2018:11:27 00:53:38 taylor just checking it out locally b/c I’ve never tried using some? for a spec:
2018:11:27 15:25:50 urzds Is there a way to separate the spec of the structure of a data structure from the spec of the data, and allow different "models" to be used in different circumstances? I.e. have a :usergroup {::meetingpoint {...} ::users [{::address ... ::name ...} ...]} (with (s/def ::users (s/* ::user)) for the geo routing part of my application, but ::user {::email ... ::name ...} for the messaging part of my application, but let both parts share the non-structural specs of ::user's data? I.e. can I tell spec "validate this map with structural information X ... switch to a different component of my app ... now validate the same map with structural information Y", with both X and Y using the same leafs and the map being the same, using the same keywords as keys, just the structure being different? (Following up on @stijn's question from earlier and my own discussion with @orestis in #graphql.)
2018:11:28 04:48:00 valerauko i'd split the specs
2018:11:28 04:48:32 valerauko in the geo-relevant ns check with a ::geo-users and in the other with ::email-users
2018:11:28 10:44:48 misha @urzds and "structure" means "value of the keyword"? so you want shape (keys set) of the maps to be the same, but content (vals) be different?
e.g. {::foo 1} and {::foo "str"}?
2018:11:28 10:51:21 misha If yes, and if you are limiting yourself to :: - just define everything (key specs and map specs) in 2 different namespaces. You'd have to define map specs twice (for your X and Y cases), since ::name will resolve to different keywords in each namespace: :my.ns.x/name and :may.ns.y/name, so your map specs would look similar, but resolve/expand into different ones, depending on namespace they are in: (s/keys :req [::name]) -> (s/keys :req [:my.ns.x/name]) and (s/keys :req [:my.ns.y/name]).
I would not chase the goal to save few lines of text here, and just define it "twice". At least you'll keep "go to spec definition" feature or your IDE working.
2018:11:28 12:17:46 urzds @misha The other way round: The values should have the same requirements, but the structure / shape / keys required may be different.
2018:11:28 12:19:25 urzds I always thought that if I (s/def ::foo ...), and validate a map that contains this exact fully qualified keyword, then spec would pick up the s/def automatically. And that this would stop working if the namespace part of the keyword would be different.
2018:11:28 12:21:25 urzds That's why I assumed that I can only define one map structure (at least using spec how I understood it), because eventually spec will validate the keys inside that map using the global definition for that key, instead of using the local structure that I would like to use in that specific place.
2018:11:28 12:26:14 urzds Or will that start to work if I use :req-un / :opt-un instead of :req and :opt? I.e. can I define multiple structures / shapes using the -un variants, because the match in the map only uses the keyword name, but the structural / shape validation happens using the namespace+name? So a structure / shape that is defined using :req and :opt is enforced everywhere and defines the complete data structure, but if it is defined :req-un and :opt-un, other definitions might exist in other places that have different structural / shape requirements?
2018:11:28 14:42:44 favila @urzds Not sure I follow the convo here, but s/keys is about validating the presence and absence of keys on a map
2018:11:28 14:44:05 favila the values of the keys are validated by the spec attached to that keyword
2018:11:28 14:45:31 favila s/keys does not require that every keyword mentioned have a spec
2018:11:28 14:45:42 favila so you can require a key but not spec its value for e.g.
2018:11:28 14:46:19 favila also :req supports and and or for grouping requirements
2018:11:28 14:46:29 favila (and :req-un)
2018:11:28 14:47:48 urzds Given this data: {::foo {::bar 1 ::baz 2}}, how would you validate it two times using spec, such that for the first time {::foo {::bar ...}} is required and the 2nd time {::foo {::baz ...}} is required?
2018:11:28 14:48:16 favila you cannot without a predicate
2018:11:28 14:48:36 favila ::foo must always have the same spec
2018:11:28 14:48:37 urzds But in both cases, ::bar and ::baz should be int?, if they are present.
2018:11:28 14:48:53 favila what determines whether ::bar or ::baz is required?
2018:11:28 14:49:16 urzds Me, depending on the context.
2018:11:28 14:50:29 mpenet spec of value of foo can be a multi-spec or a s/or
2018:11:28 14:50:47 favila you either have to give up and get a least-common-denominator spec for ::foo (e.g. (s/keys :req [(or ::bar ::baz)]) or (s/keys :req [::bar ::baz])
2018:11:28 14:51:16 favila or you have to assert your different requirements on the spec for the outer map using predicates that look inside ::foo
2018:11:28 14:53:09 favila spec has ironclad devotion to the principle that the keyword of an item in a map is its spec
2018:11:28 14:53:25 favila so you can only spec ::foo once
2018:11:28 14:54:14 favila there are a few ways to do this, but in the end you must do it in such a way that ::foo has only one spec
2018:11:28 14:54:45 favila (another possibility is making it a multispec and putting something on ::foo's map itself that determines if ::bar or ::baz is required)
2018:11:28 14:55:33 urzds So even if I spec (s/def ::foo-with-bar (s/keys :req-un [::bar] ::opt-un [::baz]) and (s/def ::foo-with-baz (s/keys :req-un [::baz] ::opt-un [::bar])) and use the appropriate one in the given context, that will not work?
2018:11:28 14:55:53 urzds multi-spec is if (s/def ... ...) has a 3rd argument?
2018:11:28 14:56:05 favila no multi-spec is a different concept
2018:11:28 14:56:12 favila dynamic dispatch to spec based on value
2018:11:28 14:56:46 urzds Do you have a link to the library or docs on multi-spec?
2018:11:28 14:57:28 favila if you give your two foo uses different keywords your outer spec is just (s/keys :req [::foo-with-bar]) or (s/keys :req [::foo-with-baz]) and everything is fine
2018:11:28 14:57:54 favila the problem is you want those two different specs, BUT ALSO as the value of a map whose key is ::foo
2018:11:28 14:58:03 favila spec is steadfastly opposed to that
2018:11:28 15:06:34 favila unnamespacing the key is another possible workaround
2018:11:28 15:06:50 urzds OK, seems they strongly disagree with me:
> Defining specifications of every subset/union/intersection, and then redundantly stating the semantic of each key is both an antipattern and unworkable in the most dynamic cases.
2018:11:28 15:07:54 urzds > unnamespacing the key is another possible workaround
I thought I read somewhere that specs are forbidden from having non-namespaced keys.
2018:11:28 15:08:12 favila no I mean the key in the value
2018:11:28 15:08:21 favila I am saying exactly what you did is a possible workaround
2018:11:28 15:08:37 favila but your maps can only look like {:foo ...}
2018:11:28 15:08:56 favila you have deliberately introduced ambiguity about what the spec of :foo is
2018:11:28 15:10:04 favila (also you don't need to introduce ns forms when specing; maybe you are just doing to save typing but it isn't strictly necessary)
2018:11:28 15:10:32 urzds How else would you do it, without the ns forms?
2018:11:28 15:10:47 favila :with-baz/map-with-baz for eg
2018:11:28 15:11:08 favila :: is just sugar for "keyword in current namespace"
2018:11:28 15:11:22 favila it's not spec-specific
2018:11:28 15:11:52 favila ::alias/sym is another thing that exists
2018:11:28 15:12:40 favila if you (:require {my.long.ns :as n]), you can use ::n/x to mean :my.long.ns/x
2018:11:28 15:13:08 favila these expansions occur at read-time
2018:11:28 15:13:13 favila (so very early)
2018:11:28 15:16:18 favila anyway, to deal with this exact problem you have I wrote a spec called keys+ which is a compromise: in a particular map, a particular key can get an additional (not replacement) spec to validate against and to use for generators
2018:11:28 15:16:46 favila so spec ::foo with the widest spec
2018:11:28 15:17:32 favila and contextually in certain s/keys specs, say it must conform to ::foo-with-bar or ::foo-with-baz also
2018:11:28 15:17:53 favila I was dealing with data whose shape I could not control, and the alternative was renaming keys everywhere
2018:11:28 15:17:57 favila
2018:11:28 15:18:51 favila so your example would look like (keys+ :req [::foo] :conf {::foo ::foo-with-bar})
2018:11:28 15:19:11 favila the value in ::foo must validate against both ::foo and ::foo-with-bar
2018:11:28 15:19:33 favila so ::foo is (s/keys :opt [::bar ::baz])
2018:11:28 15:19:55 favila but you can see from the gnarliness of this code that spec really doesn't even want this to be allowed
2018:11:28 15:20:10 favila I don't even necessarily recommend that you use this
2018:11:28 15:36:21 eoliphant hi i’m working on an fspec for a macro. The macro itself, takes an argument that will become a spec basically something like (s/def spec-key spec-pred)` in the macro body. Trying to come up with a valid spec for :spec-pred since it could be a function or say (s/or … etc. etc.) . So far I have this (s/or :fn fn? :spec s/spec?)
2018:11:28 15:42:15 favila you might need ifn? too (to catch stuff like #{1 2 3}
2018:11:28 15:49:01 alexmiller You might want to look at CLJ-2112
2018:11:28 15:49:13 alexmiller Specifically the patch in there
2018:11:28 15:56:31 urzds @favila Thanks!
2018:11:28 16:01:23 eoliphant thanks guys
2018:11:28 18:25:25 eoliphant having a peculiar issue . I’m working on a spec for a variadic function, the ‘rest’ should be members of a set. In my case the set members are maps, and the test always fails. replacing them with say keywords works fine.
2018:11:28 19:53:14 taylor (test-mac "a" {:a :a}) works, for example
2018:11:29 15:12:16 eoliphant ah.. crap.. right.. any ideas on how to perhaps work around this?
2018:11:29 15:13:50 taylor could maybe move some functionality out of the macro into a function (and spec that), or reconsider whether you need a macro at all
2018:11:29 16:12:16 eoliphant yeah it’s a bit of a chicken/egg problem. I’m working on a little DSL to create data to define a domain, so want to be able to assert stuff about say :user/name including it’s spec. so I need to pass in the spec, as well as other meta data so needed to use macros
2018:11:29 16:12:22 eoliphant but going to play around with it
2018:11:30 13:59:34 benzap So Rich said they're releasing a new clojure.spec, is it going to be radically different, or backwards compatible?
2018:11:30 14:00:51 mpenet he mentioned being careful about backward compat in the talk. Then again it's an alpha, so breakage might very well happen
2018:11:30 14:34:28 alexmiller The plan is that it will be as backwards-compatible as possible
2018:11:30 14:34:58 alexmiller Things like s/keys may continue to exist but be deprecated for example
2018:11:30 14:35:19 gklijs From what I got, instead of having to split into required and optional argument sin the spec. it's moved to something different, and you can specify on the function witch fields you need/expect
2018:11:30 14:36:29 gklijs It it already in spec alpha-2, or do we need to wait a bit?
2018:11:30 14:45:31 alexmiller No, it’s not available yet
2018:11:30 14:47:08 alexmiller We are doing some reorg of the impl in spec-alpha2 at the moment
2018:11:30 14:47:31 alexmiller And then we will be diving into the new stuff
2018:11:30 14:49:24 benzap just finishing up the talk, so spec stuff will be baked into defn, or is that pseudo-code?
2018:11:30 14:49:40 benzap or rather spec/defn
2018:11:30 14:52:31 benzap man, i'm really excited for these changes
2018:11:30 14:53:18 benzap I was hesitant when using spec in my work's project, but I feel like it would really benefit from these changes, especially when defining specs between different data layers
2018:11:30 15:02:11 orestis Just a few days ago I suggested to someone to just use optional keys for his specs and validate presence of keys in some other way. (GraphQL context). Glad to hear there’s going to be a better way.
2018:11:30 16:02:29 alexmiller @benzap it’s possible it will get integrated more into defn - currently TBD but Rich is thinking about it
2018:11:30 16:03:39 benzap Interesting! I can definitely see the tradeoffs on both sides
2018:11:30 16:04:05 alexmiller that of course wouldn’t happen till Clojure 1.11, if at all
2018:11:30 16:05:32 benzap That's good to know, i'm thinking i'll use the current clojure.spec until then
2018:12:02 06:23:52 fossifoo nice talk/announcement. really looking forward to it, especially the "Better programmatic manipulation of specs" part 🙂
2018:12:02 20:57:31 djtango Does anyone know why this is crazy slow / never terminates?
2018:12:02 20:57:42 djtango ^ (you should be able to copy paste that straight into your REPL)
2018:12:02 20:58:36 djtango oddly - if I swap out (s/coll-of any?) for coll? I see the same behaviour but if I use (s/coll-of integer?) it terminates nearly instantly (and successfully)
2018:12:02 20:59:01 lilactown it looks like there are many cases to generate 🙂
2018:12:02 20:59:58 lilactown I didn’t try it, but I can tell that generating cases for (s/coll-of any?) is going to cause test.check to really go crazy
2018:12:02 21:00:04 djtango ah I see
2018:12:02 21:00:05 jaihindhreddy And is generally not a good fit for generative testing
2018:12:02 21:00:28 lilactown you can specify an upper limit to generation
2018:12:02 21:00:28 djtango have I unwittingly stumbled onto a bad example of how to use spec / gen testing?
2018:12:02 21:01:10 jaihindhreddy You might wanna customize the generator so the tests will be quick without changing the semantics of validation that your spec gives
2018:12:02 21:01:12 djtango reverse felt like a nice simple function for illustrating fn specs
2018:12:02 21:02:12 jaihindhreddy It is, and I think you wanna keep any? in your spec but customize the generator to improve the testing
2018:12:02 21:02:35 lilactown clojure.spec/coll-of takes a :gen-max argument
2018:12:02 21:03:00 lilactown you could limit it to like 5 or 100 or something that at least finishes 😛
2018:12:02 21:07:05 jaihindhreddy That is an option too.
2018:12:02 21:08:04 jaihindhreddy This is what I was talking about: (s/def ::reverse-ret (s/with-gen (s/coll-of any?) #(s/gen (s/coll-of int?))))
2018:12:03 03:19:32 kvlt Does anyone know of a tool where I can give it a datastructure, and it will generate a spec for me based upon that datastructure?
When I test this function with stest/check I’m seeing values generated via string? being validated against :ss/diagram-yaml-str — is that what’s supposed to happen? If so… why? (This is probably just PEBKAC)
2018:12:03 18:24:47 seancorfield What's the generator for :ss/diagram-yaml-str look like?
2018:12:03 18:25:08 seancorfield (I would expect that to also have string? in it based on its name)
2018:12:03 18:29:55 avi It’s a little crazy
2018:12:03 18:30:46 avi The thing is, whenever I exercise that spec it succeeds just fine. And… hmm, well, I hope that exercise does validate the generated values against the spec 😅
(defmacro sometimes [body]
`(when (< (rand) 0.5)
~body))
2018:12:03 18:38:22 taylor FYI you’ll probably want to replace that sometimes with one of test.check’s generators, otherwise it’s an unbound/uncontrolled source of randomness to test.check
2018:12:03 18:39:18 avi Great tip — thanks so much!
2018:12:03 18:39:54 taylor maybe something like (gen/frequency [[1 (gen/return "thing")] [1 (gen/return nil)]])
2018:12:03 18:40:23 avi ah that’d work? equal frequencies means equally likely?
2018:12:03 18:40:36 avi (I didn’t look at the docstring for frequency yet)
2018:12:03 18:41:03 taylor that’s my untested assumption 🙂 and there might be an even more appropriate function in test.check I’m forgetting/don’t know about
2018:12:03 18:41:05 avi based on the link you provided, looks like I should use (gen/return nil) rather than just nil
2018:12:03 18:41:39 taylor oh yeah, sorry… my example had concrete values instead of generators, fixing
2018:12:03 18:42:07 avi 😎
2018:12:03 18:43:19 taylor I suppose you could use gen/elements too, if your values are static/not-generated. I suppose elements given a two-element coll would choose between them equally?
2018:12:03 18:44:01 avi I dunno what the distribution is
2018:12:03 18:44:18 avi IIRC some test.check generators don’t use even distributions
2018:12:03 18:48:38 avi I wonder if this is on the right track:
2018:12:03 18:59:12 taylor looks pretty reasonable to me. I still wonder if elements would be better suited in this case b/c it'd definitely be more concise
2018:12:03 18:59:33 avi probably
2018:12:03 18:59:45 avi thanks for helping me improve my code!
2018:12:03 18:35:32 avi Ah yes, peeked at the source of exercise and it calls conform on all generated values. (It’s mentioned in the docstring too.)
2018:12:03 18:57:25 avi I think this is starting to dawn on me… I think maybe (I’d like to think because I’ve been rusty) I’ve just been not quite grokking s/or … when a value is conformed against an s/or spec it will quite naturally be conformed against each scenario in order. And after a value is generated via a generator, I guess there’s no mechanism at that point to conform it straight off against its original scenario spec… in other words, s/or isn’t like pattern matching, which I guess was for some reason how I was trying to use it.
2018:12:03 19:03:38 taylor I think it's pretty conceptually similar to pattern matching, but I'm not sure I follow this issue:
>And after a value is generated via a generator, I guess there’s no mechanism at that point to conform it straight off against its original scenario spec
2018:12:03 19:06:40 taylor BTW one way I think s/or differs from pattern matching like in ML languages is that s/or clauses don't have to be exhaustive by default. Don't have to be non-overlapping either
2018:12:03 19:35:58 avi er, yeah, sorry. I think it was just my rusty thinking.
• An or spec is a spec, and (I suppose) most functions that work with specs will treat/use an or spec as an opaque spec, like any other spec.
• Accordingly, the workflows of conform and stest/check have no affordances for taking values generated from the or spec and influencing how those values are conformed against the or spec and its internal scenarios
• Accordingly, values generated by any given scenario of an or spec might be conformed against any other given scenario of an or spec, because that’s how an or spec works — values are conformed against its scenarios one by one in order
• The only reason I got caught up on this is because one of my specs would sometimes throw an exception when passed an invalid value
2018:12:03 19:36:05 avi I hope that might begin to approach making some kind of sense
2018:12:03 19:37:56 taylor Yeah I’ve been bitten using a predicate in an or that wasn’t nil safe. Order matters in some cases
2018:12:03 19:38:11 avi yes, exactly
2018:12:03 19:38:20 avi my spec calls a function that throws if passed an empty string
2018:12:03 19:38:23 avi related to a regex search
2018:12:03 18:57:26 avi Maybe
2018:12:03 18:57:45 avi I feel like I need a couch to lie down on and someone with a pipe to nod and doodle on a notepad
2018:12:03 18:57:53 avi sorry for the noise!
2018:12:03 20:31:02 prozz hmm, have a strange question: how to actually spec that something is a sentence? or write a generator for a good sentence? (let's say lorem ipsum language with limited number of words is enough)
2018:12:03 20:33:27 prozz just realized im specing stuff as string? but would like my generators to be clever and generate usernames as random strings but description as some sort of lorem ipsum with spaces. if this question is too beginner like for this channel, let me know pls.
2018:12:03 20:44:41 andy.fingerhut If you mean actual grammatically correct English, or some other natural language, then you need some kind of NLP library or a product of machine learning for that. If by "sentence" you mean something else, please clarify.
2018:12:03 20:45:45 noisesmith and spec explicitly isn't a string parsing library - anything for differentiating one string from another will be wrapped up and just be another function as far as spec is concerned
2018:12:03 20:46:00 andy.fingerhut Sorry, you did mention "some sort of lorem ipsum with spaces", but still might need some clarification of exactly what kind of strings you want to include vs. exclude. A regular expression could be written over strings that matches most lorem ipsum like text, but rejects things that contain weird special characters.
2018:12:03 20:47:14 andy.fingerhut If you did that, and then wanted to generate random examples for testing, you would likely only be able to do so by writing a custom string generator, because otherwise the default test.check string generator would, with very high probability, generate strings that do not satisfy the spec.
2018:12:03 20:57:52 gfredericks test.chuck (different lib) has a regex -> generator thingamabob
2018:12:03 20:58:50 andy.fingerhut cool. I probably saw that mentioned before and forgot about it. Such a thingamabob sounds pretty non-trivial, unless you found a library that transforms regexes into state machines for you.
2018:12:03 20:59:03 gfredericks it was nontrivial
2018:12:03 20:59:38 gfredericks the most nontrivial parts were punted on (esp. lookahead, lookbehind)
2018:12:03 21:00:13 andy.fingerhut Makes sense. Common regex libraries include so many features like that, that complicate analyzing them (and IMO, understanding what they do)
2018:12:03 21:01:25 gfredericks one big hard part was parsing the regexes correctly, and the other one was handling characters correctly
2018:12:03 21:06:01 andy.fingerhut I saw a quote similar to the following (paraphrasing from memory) in the early 1990s in someone's signature on a public Internet forum and loved it: "Ain't nothin's easy when you're doing it fer real! -- a gunnery sergeant in the U.S. Army"
2018:12:03 21:08:04 gfredericks well some things are, but they're not very interesting to talk about
2018:12:03 21:33:07 prozz thanks for explanation, you all above confirmed what i thought. need to dive deeper into generators, check possibilities and decide what i really wanna do 🙂 basically i want a fixtures generator for particular domain. hence need for sentences.
2018:12:03 23:19:10 prozz is there an existing generator for normal distribution from given range?
2018:12:04 00:14:47 alexmiller No
2018:12:04 00:14:54 alexmiller Well, not in spec
2018:12:04 00:15:05 alexmiller Maybe there is in test.check
2018:12:04 00:15:20 gfredericks Nope
2018:12:04 00:16:17 gfredericks Don't have any continuous stat distributions
2018:12:04 00:16:43 gfredericks You could make them using fmap from something we do have
2018:12:04 00:16:53 prozz yep, there is frequencies
2018:12:04 00:17:07 gfredericks Unclear what kind of shrinking and growth you'd want in many cases
2018:12:04 02:51:43 andy.fingerhut I haven't checked out their implementation but kixi stats lib claims to implement this and many other distributions: https://github.com/MastodonC/kixi.stats
2018:12:04 16:52:53 jaihindhreddy Will the decomplecting of schema/structure from selections/context be there in 1.11, or will it get in earlier?
2018:12:04 16:55:52 seancorfield @jaihindh.reddy The whole point of clojure.spec being developed and delivered as a separate library is that it is no longer tied to specific Clojure versions.
2018:12:04 16:58:07 jaihindhreddy While that is true, spec does come in the clojure jar if I'm not mistaken, hence my doubt.
2018:12:04 17:10:32 alexmiller no, it does not
2018:12:04 17:10:46 alexmiller it’s a dependency, which you can independently update
2018:12:04 17:10:59 alexmiller we’ve updated spec two or three times since 1.9 for example
2018:12:04 17:11:20 alexmiller I expect that the new work will be available before 1.11
2018:12:04 17:11:59 alexmiller but it’s likely to be in a different namespace so will be a little trickier to use - we’ve spent some time thinking through that, and probably will require some more
2018:12:04 21:23:00 urzds Is there some valid-or-explain function that I can easily use in my function's {:pre [(valid-or-explain ::spec %)]} context?
2018:12:04 21:23:34 urzds Having s/valid? in there as suggested in the guide is nice, but if I cannot see what it is complaining about, it does not help much... 😞
2018:12:04 21:37:34 seancorfield s/assert
2018:12:04 21:39:16 taylor gotta be careful using s/assert in pre/post-conditions because if a nil input is valid, it'll return it (nil), causing the pre/post-condition to fail
2018:12:04 21:39:39 taylor (because nil ≈ false)
2018:12:04 21:40:26 taylor and also have to be mindful of s/*compile-asserts*
2018:12:04 21:41:52 taylor @urzds if you want this behavior in pre/post-conditions it should be very easy to write a small function that does it
2018:12:04 21:41:58 seancorfield Well :pre / :post only take effect if *assert* is true...
2018:12:04 21:42:35 taylor *assert* is unrelated to s/*compile-asserts* AFAICT
2018:12:04 21:43:26 seancorfield Yes, but the same caveat essentially applies -- you don't get the check in all cases.
2018:12:04 21:45:31 seancorfield Your point about nil is a good one -- I hadn't thought about that.
2018:12:04 21:53:33 taylor it was a learned lesson 🙂
2018:12:04 21:48:13 seancorfield If you want the check always performed, write it as code in the body of the function. If s/assert works for you (not nil values, (s/check-asserts true) has been run), that's probably a reasonable approach -- otherwise you'll have to write your own variant of s/assert. Or perhaps write an fspec for the function and instrument it?
2018:12:05 08:14:55 jumar You can also set it via system property clojure.spec.check-asserts
2018:12:04 21:53:46 urzds Thanks!
2018:12:04 21:59:39 urzds What would be a good way to check whether a collection contains certain elements? I.e. I have [{:name "a"}{:name "b"}{:name "c"}] and I want to ensure that the collection always contains at least one element with :name "a" and one with :name "b".
2018:12:04 22:00:01 urzds Additionally, I would like to ensure that each :name only occurs once.
2018:12:04 22:01:11 taylor you could use s/and with an extra predicate for those conditions
2018:12:04 22:01:55 urzds (Also note: The element's maps contain more keys than just :name, so I cannot use :distinct.)
2018:12:04 22:03:31 dchelimsky Would it be reasonable to change the structure to a map where name moves up to keys?
2018:12:04 22:04:01 dchelimsky I don’t assume it is reasonable. Just asking if it is.
2018:12:04 22:04:21 urzds Was thinking about that initially, but I want to expose the data structure using GraphQL, and that can afaik only query for predefined keys.
2018:12:04 22:05:00 dchelimsky I’ve run into that recently :)
2018:12:04 22:05:31 taylor maybe something like this for your uniqueness constraint, and easy to add your other constraint
here's another way, not necessarily any better though
2018:12:04 22:09:54 urzds At least it becomes more clear what we're trying to do.
2018:12:04 22:10:26 taylor and you may prefer to break each condition into its own predicate like your first example
2018:12:04 22:10:43 taylor as opposed to packing multiple conditions into one predicate
2018:12:04 22:11:31 urzds So I can combine the ::specs instead of having the big inline function?
2018:12:04 22:11:50 taylor yep
2018:12:04 22:12:51 taylor the explain output for invalid inputs will likely be easier to understand if the predicates are separated
2018:12:04 22:14:35 taylor i.e. if they're all combined the explain output isn't going to tell you exactly which condition failed
2018:12:04 22:16:46 urzds Thanks!
2018:12:04 22:17:03 urzds I guess there is no way to reuse names in both specs?
2018:12:04 22:07:31 urzds Looks horribly complicated, I guess I'm missing the easy syntax...
2018:12:04 22:49:28 alexmiller coll-of has a :distinct option fyi
2018:12:04 22:49:55 alexmiller oh, I guess you’re a level off of that
2018:12:05 14:27:29 domkm A :distinct-by option would be helpful. For coll-of, I've rarely needed :distinct but often needed :distinct-by.
2018:12:05 21:19:23 lilactown sorry if this is a tired question, but is there any upcoming plans for supporting dynamic creation of specs other than eval-ing the macros? e.g. I want to stick a spec in a DB and use it to check some data later
2018:12:05 21:43:23 lilactown thx! I'll wait for docs - it's not clear me to from the commits what it's taking shape as yet.
2018:12:06 17:16:21 borkdude what would be a good name for a lib with some testing utils around specs? mostly things like instrumenting and unstrumenting in the scope of a body kind of macros, cross platform. my gut feeling says that spec-test will be too confusing
2018:12:06 17:22:59 shaun-mahood Pretty sure checkulative should be a word
2018:12:06 17:27:04 borkdude haha
2018:12:06 21:36:14 Marc O'Morain chickity check yo self
2018:12:06 19:15:55 lilactown I'm having a silly thought: clojure-spec-as-a-service.
We have a few different clients in our system, some of which are Clojure(Script) and some aren't. For those that aren't, exposing common specs for our domain is a bit more troublesome then distributing a library.
One solution might be to create a service that accepts (data, spec-name) and responds with true/`false` if it conforms. Could also do generation of fake data that conforms.
Anyone done this before? I'm not sure if this is a good idea yet, architecturally
2018:12:06 19:17:34 noisesmith @lilactown I'd be tempted to copy the way kafka works with avro - the schema is versioned and serialized and there's a rest endpoint to look up the schema, then you can deserialize, cache and reuse it
2018:12:06 19:18:58 noisesmith and naturally you can use post to add a new spec or new version of a spec, with backward compat enforcement
2018:12:06 19:21:48 lilactown > look up the schema, then you can deserialize, cache and reuse it
practically this means that I need to have a common language and engine for working with those schemas in all my clients
2018:12:06 19:22:26 lilactown I think the problem I have right now, is "I have all these specs; how do I reuse them across my JS web & mobile clients?"
2018:12:06 19:23:27 lilactown do I build a language on top of spec and build an engine for each platform? Do I adopt something else other than clojure.spec that I more of my clients will understand?
2018:12:06 19:24:01 rapskalian @lilactown that's an interesting idea...would part of the spec endpoint also do format conversion? For example, would it be necessary to validate JSON data (convert it to EDN, run through spec/valid?, return result)?
2018:12:06 19:24:46 lilactown yes, I am thinking that JSON with be the format it primarily accepts
2018:12:06 19:24:57 lilactown so additional complexity
2018:12:06 19:33:01 rapskalian One issue I can think of would be semantic parity between environments...JSON is sort of inherently less expressive then, say, EDN, so you'd have to be careful that they were validating "apples to apples" so to speak.
2018:12:06 19:34:37 noisesmith transit could be useful for this - it's expressly designed for interchange (unlike EDN) and it has implementations for the bigger languages, and it can handle all the standard clojure data types
2018:12:06 19:38:01 rapskalian Spec reuse is definitely the right idea. It seems like there'd be quite a few "translation layers" no matter how you tackled it. For example, I've spoken with people who are using Datascript as a sort of "lingua franca" of domain modeling, and then generating specs from that representation. I wonder if "specs" could be generated for other runtimes in their respective native form.
2018:12:06 19:38:47 lilactown a lot of the data we'd be verifying would be JSON in the first place - e.g. we get a response from our GraphQL API, and then want to see if it conforms to certain rules: "Does this user data have <product> associated with them?" "Are they allowed to use <capability>?"
2018:12:06 19:40:03 lilactown but there could be other context, unrelated to user data, of course
2018:12:06 19:41:01 noisesmith oh - also if you are doing permission checks there's actually an advantage to checking on the central server instead of telling the client how to validate
2018:12:06 19:41:29 noisesmith (eg. keeping changes in sync without pushing...)
2018:12:06 19:43:01 lilactown yeah permissions isn't primarily what it would be used for but I feel it could grow out of this service organically
2018:12:07 00:21:19 ro6 I'm starting to try using ::keyword style keywords more, but running into a limitation and wondering if it's a misconception on my part. I tried using the namespace as a prefix on a longer keyword, eg ::more.prefix/name, but that's an "Invalid token". Is there something about the Clojure impl (eg around namespaces or interning?) that would break if that were allowed? Is there a "good idea" that's being enforced here to save me from a "bad idea"? I was hoping it would expand to :the.ns.more.prefix/name.
2018:12:07 00:22:24 seancorfield ::foo/bar will expand the alias foo into the full namespace.
2018:12:07 00:23:56 seancorfield (that's essentially the same as (require '[long.ns.foo.bar :as foo.bar])
2018:12:07 00:24:05 ro6 right, at the usage site. I'm inside the defining namespace and conceptually wanting to "group" some things without creating a whole new ns
2018:12:07 00:24:23 seancorfield :: can only be used with an alias.
2018:12:07 00:24:39 seancorfield (or a bare keyword -- the "alias" is implicitly the current namespace)
2018:12:07 00:24:45 ro6 right
2018:12:07 00:27:32 ro6 I'm just wondering if there's a deeper "why" for that.
2018:12:07 00:28:01 seancorfield I'm not sure what your question is...?
2018:12:07 00:28:08 noisesmith :: is a keyword reading feature that works with aliases
2018:12:07 00:28:45 seancorfield ::more.prefix/name means "look up more.prefix as an alias and resolve it (to some namespace) and then construct a keyword with that namespace and /name"
2018:12:07 00:28:52 seancorfield That is the semantics of ::
2018:12:07 00:31:42 ro6 I think what's going on in my mind is that as I imagine these ns-qualified keywords escaping my Clojure system, I want to narrow them to remove ambiguity. In other words, I'm trying to name things so they still make sense with fewer contextual assumptions.
2018:12:07 00:32:48 seancorfield Not much outside Clojure is going to be able to accept qualified keywords... Or am I misunderstanding you?
2018:12:07 00:34:45 ro6 No, I think you're right on.
2018:12:07 00:35:29 ro6 I guess this line of thought is more about self-documenting keywords for human consumption than machine disambiguation.
2018:12:07 00:35:31 seancorfield If you want an alias, to avoid typing, you can always do this inside your the.ns namespace:
2018:12:07 00:37:11 ro6 interesting compromise. That would let me organize my code into files independently of how I'm organizing my names/concepts.
2018:12:07 00:37:32 seancorfield (there's talk of an easier way to create aliases being added to Clojure at some point but that's what we have right now)
2018:12:07 00:37:55 seancorfield There's no need for the "namespace" portion of a qualified keyword to be an actual namespace with code.
2018:12:07 00:38:12 seancorfield We have several qualified keywords in our code base that do not map to namespaces.
2018:12:07 00:39:43 ro6 ok. Thanks for the info.
2018:12:07 00:40:56 ro6 I also just realized that dot-separated segments after the / are possible too
2018:12:07 00:46:12 ro6 I guess at runtime what separates keywords (ns-qualified or not) from strings is mostly faster equality checks due to interning.
2018:12:07 00:46:13 alexmiller But not recommended
2018:12:07 00:46:25 ro6 haha, thanks.
2018:12:07 07:34:05 mac Does anyone know of a (stateful) generator of locally unique integers for use with spec?
2018:12:07 16:27:15 gfredericks what does "locally unique" mean?
2018:12:07 16:27:34 gfredericks @mac ^^
2018:12:07 17:11:11 ro6 (guessing) Non-repeating relative to some local scope (eg the generator itself, the running process, ....)?
2018:12:07 17:11:39 ro6 Not sure about the "stateful" part
2018:12:07 19:09:28 mac @gfredericks Just means that for the run of the test no two ints produced will be the same.
2018:12:07 19:12:59 gfredericks @mac depends what "for the run of the test" means in this context; is this the generative test that spec does with spec/check or whatever it is?
2018:12:07 19:34:14 taylor maybe (gen/vector-distinct gen/int) would be useful... somewhere
2018:12:07 20:45:11 mac @gfredericks Yes it is "the generative test that spec does with spec/check"
2018:12:07 20:46:32 mac @robert.mather.rmm Stateful because I cannot imagine being able to guarantee uniqueness without maintaining state.
2018:12:07 20:47:06 gfredericks the standard way to do it would be to write your own generator at the lowest level that encompasses the whole relevant scope of uniqueness
2018:12:07 20:47:24 gfredericks you could take the default generator and gen/fmap it to something that uniqueifies the ints
2018:12:07 20:47:45 gfredericks an alternative, if you don't care how big the ints are, is to just generate really big integers and not allow them to shrink
2018:12:07 20:51:32 gfredericks (you'd need to take care to make the generator something like uniform-distribution, so it doesn't start small either)
2018:12:07 20:52:42 mac @gfredericks thanks
2018:12:07 20:56:11 ro6 I don't know if it can be done lazily/immutably, but if you're worried about performance and you know how many random+unique ints you need up front, there's a cool algorithm for that.
2018:12:12 20:09:50 domkm Apologies if this has been asked previously but why are the Alpha1 and Alpha2 namespaces inconsistent? I would expect clojure.spec.alpha2 instead of clojure.spec-alpha2?
2018:12:12 20:12:11 alexmiller because we found having all the namespaces end in “alpha” was annoying
2018:12:12 20:43:17 domkm I see
2018:12:12 22:35:08 seancorfield @alexmiller On that subject, should I open JIRA issues for the bugs I ran into with spec-alpha2 or isn't it time yet?
2018:12:12 22:56:37 alexmiller I would wait right now
2018:12:12 22:56:48 alexmiller I have another giant refactor pending
2018:12:13 05:43:35 noprompt There’s also the greek letter beta. troll
2018:12:13 05:44:55 noprompt Why should beta be reserved only for use-at-your-own-risk software I ask? 😄
2018:12:13 10:29:13 jumar Let's say I have following config spec:
I used explain-data and returned :path key to indicate which keys are in the config are invalid/missing.
What I didn't realize is that when the key is missing then the path is empty (instead of pointing to the particular key).
Is there a nice way how to get the name of a missing required key?
2018:12:13 13:41:22 seancorfield Not nice but you can pattern match :pred with destructuring since it has that particular shape and extract the missing field name from the contains? call.
2018:12:13 13:56:22 jumar Thanks for the idea!
I'm sure there's a better way but here's my quick solution:
2018:12:13 13:42:01 seancorfield (And it might change in a future version of spec)
2018:12:13 23:42:19 thedavidmeister @simon223
2018:12:14 00:27:12 aengelberg why doesn't clojure.spec.gen.alpha have recursive-gen?
2018:12:20 20:21:27 wilkerlucio this seems more a generation concern than a spec concern, so I think makes sense been part of test.check / test.chuck or something else
2018:12:18 19:17:40 rapskalian Cool! Thanks for sharing. It looks like the double generator is a bit "quantum biased", for lack of a better word.
2018:12:19 01:48:01 bbrinck If you previously couldn’t use Expound because your specs used a custom conformer, try Expound 0.7.2 - it may not be able to give a precise error, but at least it won’t throw a “Cannot convert path” error
2018:12:19 19:56:55 ro6 Question about evolving specs: I realize the whole idea with specs is "don't change them in breaking ways", so I'm thinking about how to plan for that.
If I define a spec with a single definition, then later realize there's another valid representation, so I want to change it to (s/or ....), can that be a non-breaking change to consumers?
2018:12:19 19:58:35 ro6 I guess if the original definition was one of the options under the s/or it would still validate the same, but the conformed value would change right? I guess generally the conformed value is more likely to be depended-upon by internal code rather than API consumers...
2018:12:19 20:15:25 alexmiller adding additional valid things means old stuff should continue to be valid
2018:12:19 20:15:51 alexmiller I don’t know that we would expect forward-consistent stuff out of conform though
2018:12:20 17:38:19 jaihindhreddy Will there be a way to s/select into a subset of dispatch keys of a s/multispec?
2018:12:20 17:44:29 jaihindhreddy Say I have a polymorphic map called ::payment-info which is polymorphic, like this: (s/def ::payment-info (s/multi-spec pi ::type)) where (s/def ::type #{::credit-card ::debit-card ::net-banking ::bitcoin})
Because cards are processed one way, and each of the other types another way,
is s/selecting into ::credit-card and ::debit-card a good idea?
Was just wondering about this, s/select isn't even a thing yet so these are potential future questions 🙂
2018:12:20 18:12:35 alexmiller it’s not real enough yet that I can’t answer those questions
2018:12:20 18:55:53 ShaneLester How would I write a spec to say- in this map I expect to have X Y and Z keys (which have their own specs) and then I don’t care about the rest of the map? Does specifying those 3 in the :req of the keys function assume that I don’t care about the rest?
2018:12:20 18:56:20 taylor yeah the s/keys specs are "open" by default
2018:12:20 18:57:04 ShaneLester Okay cool, thanks
2018:12:20 20:53:48 ShaneLester So I have these 3 specs here… I am trying to say, hey there is going to be any number of inputs here in this input-type. But apparently something like {:inputs [:email {:value nil}]} doesn’t meet this spec? It’s saying it should satisfy map? but it seems to me that that IS a map. Anyone have a clue what I’m doing wrong here?
2018:12:20 21:10:40 taylor ::inputs is expecting a collection of ::input-types which are maps/`s/keys` specs, but your input is a single map
2018:12:20 21:11:03 ShaneLester ohhh I see. thank you
2018:12:20 21:12:56 taylor [{:value nil}] would conform to your existing spec
2018:12:20 21:13:49 taylor you may have some other issues judging from the vector [:email {:value nil}], with the spec as it is
2018:12:20 21:14:13 ShaneLester Right… yeah I may be doing some things in not the best way 🙂 time to re-evaluate
2018:12:20 21:15:46 taylor feel free to post some other sample inputs in here and I could help write some sample specs for them
2018:12:20 21:17:31 ShaneLester Cool, I appreciate that
2018:12:20 22:10:24 ag can someone throw an example of recursive spec… e.g. directory structure?
2018:12:20 22:16:45 ag right.. right… I vaguely remembered, but I couldn’t figure it out… Thanks!
2018:12:20 22:30:54 ag now I wonder if there is a way to make
binding [clojure.spec.alpha/*recursion-limit* 2]
part of (s/with-gen?
2018:12:20 22:31:18 ag so it never generates more than 2 levels deep?
2018:12:21 00:19:53 ag is it possible to run count of (s/def #{"a" "b" "c"}) ?
2018:12:21 01:59:38 taylor is that s/def missing a name?
2018:12:21 01:59:56 taylor e.g. (s/def ::foo #{1 2 3})
2018:12:21 02:03:01 taylor anyway if your spec is a literal set, you could do
(s/def ::foo #{1 2 3})
(s/form ::foo)
=> #{1 3 2}
2018:12:21 02:03:25 taylor or you could define the set separately, and reference it from the s/def and wherever you need to count it
2018:12:21 00:20:10 ag how can I find out how many elements there?
2018:12:21 01:57:35 ro6 Is there a way to abstract over whether a key in a map is namespaced or not? In a certain context, as long as the value conforms, I want to accept both with the same spec.
2018:12:21 02:48:29 alexmiller If you’re using s/keys, the :req-un and :opt-un will do that
2018:12:21 17:08:21 urzds Can I spec record methods just like any ordinary function?
2018:12:21 17:09:57 urzds The guide does not really go into details on records, except that one can spec their fields/attributes: https://clojure.org/guides/spec
2018:12:21 17:15:18 alexmiller records don’t really have methods
2018:12:21 17:15:33 alexmiller they implement interfaces/protocols which define methods
2018:12:21 17:15:47 alexmiller currently, you can’t spec protocols or interface methods
2018:12:21 17:16:27 alexmiller and because of the calling implementation it’s not possible to instrument them (and probably not something we’re ever going to really do because of that)
2018:12:21 17:16:52 alexmiller but I wouldn’t completely rule it out
2018:12:21 17:21:25 urzds @alexmiller So I can spec neither the protocol for all its implementations, nor every record implementation individually?
2018:12:21 17:23:28 alexmiller no
2018:12:21 17:23:28 urzds But I could place the implementation in a function, which I could spec, and then from the record implementation just call that function? I.e. insert one step of indirection?
2018:12:21 17:23:41 alexmiller yes, but I wouldn’t do that just to be able to spec it
2018:12:21 17:24:46 urzds Well, right now my implementations look like (s/assert ...args...) (s/assert ...body...), which is hardly better...
2018:12:21 17:25:06 alexmiller this is a case where the impl of protocols (designed to tap into the highly optimized java/jvm calling semantics) is at odds with the dynamic indirection possible in Clojure via vars
2018:12:21 17:26:04 alexmiller I guess maybe there’s some possible future where newer Java capabilities like method handles could be used to implement this kind of thing
2018:12:21 17:26:48 alexmiller retaining most of the speed but also giving you the dev time instrumentation
2018:12:21 17:33:53 urzds So what do you do when during development you want to ensure that the maps returned from a method implementation of the record conform to a spec? And that all callers call it properly? Insert (s/assert arg-spec arg) (s/assert ...body...), like I did?
2018:12:21 17:36:36 urzds I don't really care if the call slows down by an order of magnitude, if it helps me in debugging where that borked data came from.
2018:12:21 17:37:12 urzds Or do you just split up the code in such small pieces that the method implementation in the record does not really do any significant work anymore, so you have enough functions that you can properly instrument?
2018:12:21 17:38:08 alexmiller I don’t think there is one canonical answer to that question, it depends on the code
2018:12:21 17:39:03 alexmiller splitting up code into smaller pieces is usually a good idea though
2018:12:21 17:39:55 urzds Thanks, I'll start with that then, maybe it already alleviates the problem.
2018:12:21 22:06:45 dominicm Is there any work yet on determining whether a spec is a subset or a superset of another? I think Rich has hinted at this a lot.
2018:12:21 22:34:51 alexmiller No
2018:12:21 23:27:43 ro6 is anyone using (s/cat ...) to spec strings? I realize I could just delegate to a traditional string regex with (re-matches ...), but then I give up on high-fidelity explain results no?
2018:12:22 15:39:28 dustingetz How do I spec inside a reagent reaction or other container type that isn't a seq
2018:12:22 15:41:32 dustingetz Has this been debated before?
2018:12:22 16:23:20 didibus Well, you're not supposed to spec everything
2018:12:22 16:25:36 didibus The same way you don't necessarily put comments on everything
2018:12:22 16:26:49 didibus You need to ask, is this hard to understand, do people wonder what the shape and structure of this thing is, if so, a spec can be a great way to make that easier
2018:12:22 16:28:40 didibus Or, if you specifically want to perform generative tests on a particular function that would benefit from it
2018:12:22 16:31:30 didibus Or if you need to validate user input, or validate data you're about to persist, etc.
2018:12:22 16:33:09 didibus But if its obvious what the things are, you don't need a spec.
2018:12:22 16:34:07 didibus Now to spec an atom, I mean, specs are just predicates. You can easilly do: #(s/int? @%)
2018:12:22 16:35:01 didibus Now you do have to make sure that the deref won't trigger unintended effects
2018:12:22 16:35:49 didibus Like in the case of a future or delay
2018:12:22 16:37:11 didibus And for atom, the other challenge is mutation. Your spec needs to either be the union of all possible valid values the atom will ever contain, throughout its many mutations
2018:12:22 16:38:13 didibus Or it has to be more generic, like validates that you expect the thing to be an atom and that's all
2018:12:22 16:38:49 didibus Or you need a way to know the particular context and time and what in that context and time the atom is supposed to contain
2018:12:22 16:39:07 didibus And spec relative to that
2018:12:22 16:40:48 didibus The validator idea is good also. Validators were designed to validate the data being set on an atom. So you can leverage spec to make the validation.
2018:12:22 16:40:59 didibus Since spec is pretty much a DSL for easy validation
2018:12:22 17:58:06 dustingetz It's the ratom which is the incidental complexity, it is spiritually just data. For better or worse, Reagent UIs pass things that are data as reactions and it is what it is. But I was unable to get it to work in a nontrivial case
2018:12:22 23:33:02 didibus Can you show an example? Also, the ratom isn't a normal atom, and its reactive nature has a different hook for when the data would be updated and need to be validated. At least from what I know, I actually havn't used reagent
2018:12:22 21:41:19 tedcushman I have encountered errors when trying to use s/and with s/or as the first argument:
=> (s/explain
(s/and (s/or :int int? :double double?)
pos?)
3)
ClassCastException clojure.lang.MapEntry cannot be cast to java.base/java.lang.Number clojure.lang.Numbers.isPos (Numbers.java:96)
2018:12:22 21:43:26 tedcushman Using s/or after the first argument appears to work though:
2018:12:22 21:44:06 tedcushman Unfortunately, if you are trying to write (s/and (s/or ...) (s/or ...)) there doesn’t seem to be a workaround.
2018:12:22 21:58:41 alexmiller s/and will flow the conformed result so the and will be receiving a value like [:int pos?]
2018:12:22 21:59:15 alexmiller prob the best option here is to (s/or :int (s/and int? pos?) :double (s/and double? pos?))
2018:12:22 22:59:41 tedcushman ok, now that I read the doc, I see the part about “successively conformed values”.
2018:12:27 18:33:37 thomas Hi All, I have been trying to write a spec recently and I can't quite describe what I want...
I have a map where two sets of keys depend on each other like this:
2018:12:27 18:35:14 thomas
{:a "string"
:b :value-b
:c :value-c}
or
{:a "string"
:b :value-not-b
:c :value-not-c}
and the :b and :c values depend on each other.
2018:12:27 18:35:47 manutter51 ( try three backticks to format that without emojis)
2018:12:27 18:37:07 thomas thank you @manutter51
2018:12:27 18:37:51 thomas so I either have to first version of the second one.
2018:12:27 18:39:41 thomas I assume I need something with an or and an and but how exactly?
2018:12:27 18:41:04 manutter51 I’m fairly shaky on spec still, but I’m thinking maybe you could take advantage of namespaced vs. unnamespaced keys, let me see if I can type up what I’m thinking
2018:12:27 18:43:31 thomas thank you... I don't get it either yet I'm 'fraid
2018:12:27 18:46:11 manutter51 I’ve got a suspicion that won’t work, because maps are spec’ed by key, not by value, but a perverse part of my brain is saying maybe it would, because the keys are defined to take two mutually exclusive sets.
2018:12:27 18:46:36 thomas let me give that a try @manutter51
2018:12:27 18:48:33 thomas I just did a few generates on that and that looks like it does work!
2018:12:27 18:53:07 manutter51 Generates seem like they’d be more likely to succeed, but I’m not as confident about conforms with “bad” data.
2018:12:27 18:55:52 thomas well... I assume if it generates the correct data, then the other way round should work as well.
2018:12:27 18:56:04 thomas but let me try...
2018:12:27 18:58:52 thomas yes, validating works as well as expected.
2018:12:27 18:59:04 thomas Thank you again @manutter51!!!
2018:12:27 18:59:28 manutter51 That’s pretty wild, I’ll have to remember that one myself.
2018:12:27 18:59:43 thomas a good trick to use the namespaces for that... I wouldn't have thought of that.
2018:12:27 18:59:58 thomas I suspect it is something that happens more often.
2018:12:27 19:01:10 taylor I think you could also combine your s/keys spec using s/and with another predicate, and do any "custom" assertions in the extra predicate
(where a formula is later defined by referencing :elogic/negation)
2018:12:28 21:42:11 hmaurer actually sorry, I just realised in this specific case the formula definition can be hoisted above the negation definition, even thought it references it
2018:12:28 22:41:38 jaihindhreddy Here's a good example of mutually recursive specs by @U0FR82FU1
2019:12:30 18:13:20 beders hello spec-fans,
is there a way I can describe a map with spec that uses uuids as keys?
It seems that this is outside the scope of spec and thus makes spec unsuitable for describing general purpose data schemas. Am I missing something here?
2019:12:30 18:18:46 mpenet (s/map-of uuid? any?)? Unless you mean mapping specific uuid values
2019:12:30 18:30:12 beders Thank you! That was what I was looking for !
2019:12:31 12:21:09 hmaurer (I want to write a test that runs check on a function, passes if all cases pass, fails otherwise and reports the failing case)
2019:12:31 15:11:13 hmaurer I am getting Exception: java.util.concurrent.ExecutionException: Syntax error compiling at (clojure/test/check/clojure_test.cljc:95:1). with:
2019:01:01 19:35:13 hmaurer @borkdude the test definition works fine for me as well; it’s at the point when it’s run that the problem arises 🤔
2019:01:01 19:35:56 hmaurer @borkdude actually weirdly enough calling it manually works… but not as part of the test suite. SOmething must be fucked in my setup
2019:01:01 19:42:37 borkdude if you don’t wrap the call to stest/check in successful? but in a doall, does it work?
2019:01:01 19:43:17 hmaurer @borkdude yep that works in the repl for me as well, but when running lein test it throws.. 😞 I don’t think the issue is with your library at all; I replaced the successful? function with a function that always returns false and got the same error
2019:01:01 19:43:57 borkdude and if you remove the call to stest/check, do you still get the error?
2019:01:01 19:44:03 hmaurer @borkdude with doall, same error
2019:01:01 19:44:23 borkdude it might be that you need to include the dependency clojure.test.check, but it depends on your error of course. I haven’t seen the full stacktrace
2019:01:01 19:44:29 hmaurer @borkdude nope, if I remove it I don’t get the error
2019:01:01 19:45:45 hmaurer (with lein test; I normally use kaocha)
2019:01:01 19:46:44 borkdude it might be your spec?
I see this in the error:
Caused by: java.lang.ClassCastException: clojure.lang.AFunction$1 cannot be cast to clojure.lang.MultiFn
2019:01:01 19:47:26 hmaurer @borkdude well if I run stest/check in the REPL it works fine (it generates inputs, etc, and the output conforms to the :ret spec)
2019:01:01 19:47:30 hmaurer 🤔
2019:01:01 19:48:38 hmaurer hell, I even ran clojure.test/run-all-tests in the repl and it worked
2019:01:01 19:48:59 borkdude maybe remove the target folder?
2019:01:01 19:49:24 borkdude do you run the repl with the same dependencies and build system as the tests?
2019:01:01 19:50:18 hmaurer @borkdude removing the target folder didn’t fix it, and yes I do. lein repl and lein test respectively
2019:01:01 19:51:05 hmaurer @borkdude the dependencies are almost the same except for respeced and fulcrologic/fulcro-spec with have :scope "test" set on them
2019:01:01 19:51:41 borkdude should work. maybe make the smallest possible repro and put it in a separate repo
2019:01:01 19:51:53 borkdude using clojure.spec and not my lib
2019:01:01 20:02:09 hmaurer @borkdude ok I reproduced the error, pushing
2019:01:01 20:13:17 hmaurer @borkdude oh, interesting. I want to use kaocha though, so I’ll have to figure out what their equivalent is. That’s still a solution though; thanks a lot 🙂
2019:12:31 13:08:33 borkdude (look at the script/test step)
2019:12:31 23:31:22 richiardiandrea say I have a password key in a map that is speced...and some other key does not validate, this will throw an exception...is there a way to tell spec NOT to include that sensitive key in the ex-data at all?
2019:01:01 00:33:30 seancorfield @richiardiandrea Isn't the concern there more than you need to not display ex-data values "as-is" to anyone?
2019:01:01 00:34:07 seancorfield It would be the same as any other ex-info you throw -- it could well include sensitive data in some context...
2019:01:01 00:34:22 richiardiandrea that's a tough one, because then my display thing needs to know about what is displaying
2019:01:01 00:35:04 richiardiandrea aka, I need to call dissoc on things, maybe it should
2019:01:01 00:35:06 seancorfield I wouldn't expect production code to just display any spec failure as is -- those exceptions are intended for code, not humans, and certainly not end users.
2019:01:01 00:36:22 seancorfield FWIW, our application error logging/reporting code has always contained logic to strip known, sensitive fields from any data logged or reported, long before spec was a thing...
2019:01:01 00:38:30 richiardiandrea ok yeah maybe that's something I would need to do anyways
2019:01:01 00:39:00 richiardiandrea I wonder though if things change ...say the string for password changes now I need to change spec AND dissoc
2019:01:01 02:12:32 seancorfield @richiardiandrea That's a good reason to use a globally-unique qualified key name 🙂
2019:01:03 09:48:31 y.khmelevskii Hi everyone! Just curious, does anybody know when clojure/clojurescript with spec/schema and spec/select will be released? I like this idea and want to try it 🙂
2019:01:03 13:34:13 alexmiller We have not started developing it yet, so not soon :)
2019:01:03 13:34:13 alexmiller We have not started developing it yet, so not soon :)
2019:01:03 20:18:28 y.khmelevskii thank you for info 🙂
2019:01:03 16:25:04 borkdude what is generally preferred in specs?
(s/nilable (s/or ...) vs (s/or :nil nil? …) when enumerating alternatives
2019:01:03 16:25:20 alexmiller the first is perf optimized
2019:01:03 16:25:25 borkdude thanks
2019:01:03 16:26:15 borkdude it might also be more robust, since some predicates can crash on nil
2019:01:03 16:26:16 alexmiller the generator for nilable is also intentionally unbalanced so it only produces nils 10% of the time
2019:01:03 16:26:50 alexmiller whereas the latter will be 1/n where n = # of options
2019:01:03 16:28:24 borkdude can something similar be said about s/alt vs s/or in a regex?
2019:01:03 16:29:01 alexmiller not sure what you mean. do you mean nilable vs alt on nil?
2019:01:03 16:29:51 alexmiller if so, then that’s a different situation as s/nilable is not a regex and won’t compose in the same way as s/alt with other regex ops
2019:01:03 16:29:56 borkdude no sorry, (s/cat :foo int? :x (s/alt ...)) vs s/or instead of alt
2019:01:03 16:30:11 borkdude so independent of nil
2019:01:03 16:30:26 borkdude I’m looking for an argument other than “alt belongs in regex”
2019:01:03 16:30:31 alexmiller s/alt is a regex op and composes with other regex ops to describe a single sequential structure
2019:01:03 16:30:48 alexmiller s/or is not a regex op
2019:01:03 16:31:03 alexmiller so acts as an independent value
2019:01:03 16:31:19 borkdude sometimes the spec for :x can be taken apart and then it only makes sense to do it with or to make it re-usable on a single value
2019:01:03 16:31:49 alexmiller it depends here on what … is
2019:01:03 16:32:09 borkdude right, … can contain other regex specs
2019:01:03 16:32:21 alexmiller in some cases, there is no perceptible difference
2019:01:03 16:32:23 borkdude so, it depends then
2019:01:03 16:32:43 alexmiller there are cases where you might want s/alt and some where you might want s/or
2019:01:03 16:33:10 alexmiller if you’re describing alternatives of the sequential structure, then probably s/alt
2019:01:03 16:33:42 alexmiller if you’re describing alternate value sets that occur at a particular point in the structure, then probably s/or
2019:01:03 16:34:25 alexmiller usually the s/alt case will contain more regex ops inside the … whereas the s/or will not
2019:01:03 16:34:33 borkdude clear, thanks!
2019:01:04 17:44:25 dacopare What's the best way to integrate spec with clojure.test? I want to run my spectest/checks as part of the test suite.
2019:01:04 18:33:49 taylor I've used this pattern before, not sure if it's good or not:
2019:01:04 18:34:31 taylor doesn't give meaningful output on failure
2019:01:04 18:20:50 rapskalian @dacopare Clojure.test.check has the clojure.test.check.clojure-test/defspec macro for that purpose.
https://github.com/clojure/test.check
2019:01:04 18:29:48 alexmiller that’s not going to help you here
2019:01:05 20:20:13 misha entities are lazy. validating/conforming with s/keys would imply realizing and walking the entity graph (friends of friends of friends ...)
2019:01:06 04:21:07 mathpunk This doesn't look right at all. Am I using instrument incorrectly?
2019:01:06 04:22:29 lilactown @mathpunk :args needs to be, at the very least, a collection
2019:01:06 04:22:51 lilactown you can think of it as, you’re verifying the [s] collection
2019:01:06 04:24:05 lilactown usually I use s/cat. e.g.:
(s/fdef friendlier
:args (s/cat :s string?))
2019:01:06 04:24:13 mathpunk ohhhhh
2019:01:06 04:24:45 mathpunk ok i think I see this
2019:01:06 04:25:18 lilactown (s/cat :s string?) checks if the first element in the sequence is a string
2019:01:06 04:25:53 mathpunk and we're verifying an args /vector/, which happens to have one element
2019:01:06 04:26:03 lilactown right
2019:01:06 04:26:31 lilactown it’s actually a sequence, I think. you can see it’s printed as ("World")
2019:01:06 04:27:44 mathpunk ok this makes sense, thank you
2019:01:06 04:28:00 mathpunk and yeah it's right there but i guess i thought those were decorative parens 🙄
2019:01:06 04:28:35 lilactown yeah 😅 trust me I went through the same confusion the first time I tried to use fdef
2019:01:07 14:50:23 urzds Are logs of this channel available somewhere?
2019:01:07 21:41:35 alexmiller I’ve looked at it, and generally I’d say no
2019:01:07 21:42:06 alexmiller this is well outside the sweet spot for spec
2019:01:07 21:52:17 kenny I'm seeing strange behavior when running clojure.spec.test.alpha/check on an fdef'ed function. I have a prn as the first line in my function that prints out the size of the args. When the generative tests first start running, I get print statements out rapidly. After running for a few seconds, the print statements slow down to once every second. After 10-20s more, they speed up again. This goes on for about 10 mins until check returns. At first I thought it was due to test.check generating large maps so I changed the :args in my fdef to use (s/map-of any? any? :gen-max 5). Then I thought it was a JVM memory issue but (.maxMemory (Runtime/getRuntime)) says the JVM has over 7gb available. I haven't seen this behavior with spec check before. Any ideas on what could cause this?
2019:01:07 21:58:55 alexmiller sounds awfully like GC churn to me
2019:01:07 21:59:37 alexmiller do you have an fspec anywhere?
2019:01:07 22:00:12 kenny Not in this function.
2019:01:07 22:01:30 alexmiller you can add -verbose:gc to watch the gc
2019:01:07 22:02:15 alexmiller you could isolate just the gen on the args if you suspect that it’s it
2019:01:07 22:48:57 kenny Generating the args like you have there takes about 30s. I'll add the gc flag to see if it sheds any light.
2019:01:07 22:50:54 kenny Added the gc flag. I get a message that looks like this [GC (Allocation Failure) 2477246K->1068161K(3133952K), 0.1575999 secs] printed out every second or so.
2019:01:07 23:06:08 kenny This is what the output with the -verbose:gc flag looks like https://pastebin.com/raw/XCxMykgq. The numbers are the count on the two args this function takes.
2019:01:07 23:08:58 kenny Updated to test.check 0.10.0-alpha3 and the problem goes away.
2019:01:07 23:53:52 lilactown I have a map like:
{::kind :bool
::default true}
that, given a different ::kind, might need a different predicate for ::default. E.g.:
{::kind :string
::default "foo"}
What's the best way to model this in spec? I thought multi-specs would work, but I'm still struggling with the fact that the name of the spec I pass into s/keys needs to match the key, which makes overloading ::default difficult
2019:01:07 23:57:30 mattly @lilactown I've done this with s/or specs
2019:01:07 23:57:37 mattly I can give you an example in our work slack
2019:01:08 00:14:53 lilactown AFAICT there's no way to do this without using :req-un😞
2019:01:08 00:20:11 favila spec is super duper opinionated on this point though
2019:01:08 00:20:25 favila hence the complexity of the hack
2019:01:08 00:21:57 favila use like (keys+ :req [::whatever] :conf {::whatever narrower-specish-thing})
2019:01:08 00:23:00 favila ::whatever value will be asked to validate against both its own spec and narrower-specish-thing (so ::whatever should be the widest possible spec you could have for that key)
2019:01:08 00:23:13 favila but narrower-specish-thing will be used for conforming and generators
2019:01:08 00:25:43 favila The idea is that contextually (in a specific map) a key may have a narrower spec than normal
2019:01:08 00:26:05 favila which for some reason happens to me all the time and made spec very painful
2019:01:08 00:26:30 favila the alternative is s/or with more predicates, and with-gen to adjust the generator
2019:01:08 00:26:33 mattly you could also just forgo using s/keys and do it manually (spec/def :my-union/shape (fn [thing] (case (::kind thing) :bool (if (boolean? (::default thing)) true ::s/invalid) ::s/invalid)))
2019:01:08 01:09:45 lilactown hm. yeah, I think I settled on:
2019:01:08 03:26:23 alexmiller s/multi-spec is kind of designed to use different specs based on data
2019:01:08 03:27:03 alexmiller you’d need to use it with s/keys and :req-un here though since you have the same attribute name with different specs apparently
2019:01:08 14:28:57 urzds Back in December I asked whether it was possible to spec protocol methods, where the answer was "no" and "because of the implementation that is targetted at performance". I am wondering whether I could instead use multi-methods instead of protocols and methods and spec them. (My code should allow replacing the record with a map and the protocol methods with multi-methods.) Can I just use s/fdef on the multifn and spec :args and :ret that have to be valid for all implementations? An alternative would be to use pre/post conditions, but I cannot see anything resembling a pre-post-map (as is present for defn) in the docs for defmulti or defmethod.
2019:01:08 14:37:22 alexmiller Currently, I do not believe that works, but I think it could be made to work
2019:01:08 14:54:57 urzds I just tried the following code and got no error, which I guess suggests that it does indeed not work:
2019:01:08 16:00:06 urzds You're right, the following code at least throws an exception:
(require '[clojure.spec.alpha :as s])
(require '[clojure.spec.test.alpha :as stest])
(defmulti testfn :type)
(defmethod testfn :atype [m] 1)
(s/fdef testfn :args (s/keys :req-un [::type ::does-not-exist]))
(stest/instrument `testfn)
(testfn {:type :atype})
=> clojure.lang.ExceptionInfo: Call to #'user/testfn did not conform to spec.
But it also throws this exception for arguments that should conform:
(testfn {:type :atype :does-not-exist 1})
=> clojure.lang.ExceptionInfo: Call to #'user/testfn did not conform to spec.
Sadly there is no explanation why...
2019:01:08 16:06:25 alexmiller I don’t actually see a CLJ issue for just “multimethods can’t be instrumented” but would be ok to make one if you like! Like I said, I think this is something that is fixable.
2019:01:08 16:06:53 alexmiller actually I think your spec is wrong
2019:01:08 16:07:03 alexmiller you’re missing the top level args sequence
2019:01:08 16:12:35 urzds Can specs be redefined? I just tried to execute (s/fdef testfn :args (s/cat :m (s/keys :req-un [::type ::does-not-exist]))) in the same REPL session, which appeared to be successful, but spec would still throw an exception even when I passed in the correct arguments. Only restarting the process fixed that.
2019:01:08 16:15:35 alexmiller you need to instrument again
2019:01:08 16:16:13 alexmiller or possibly unstrument / instrument (although I think either will work)
2019:01:08 16:16:30 urzds yes, just calling stest/instrument again worked.
2019:01:08 16:23:19 borkdude @urzds if you use something like component, you can hook up re-instrumentation with the start/stop lifecycles
2019:01:08 17:16:57 alexmiller So just repeating my comment in the jira, the idea behind instrument is to check whether a function has been correctly invoked
2019:01:08 17:17:18 alexmiller The idea behind check is to verify that a function produces proper outputs in response to valid inputs
2019:01:08 17:31:01 urzds Hm, I got that, but I would also like to see, during development, that a function is being invoked correctly and it responds correctly in live workloads. I.e. in my use-case I want more than to just check whether it was invoked correctly, but I also am not running the function in a test where I could use check. How is my use-case to be handled?
2019:01:08 17:32:34 urzds What I did not understand is the technical necessity for instrument not to also check the return value. I assumed knowing why you decided against that might help me understand the context better.
2019:01:08 17:42:30 borkdude @urzds instrument can take a performance hit when you have a lot of fdefs. if you check the return values, you will often check those twice, since they are arguments for another function. so I think it’s a sane default
2019:01:08 17:43:30 borkdude this may not the reason that core decided to do this, but I have come to appreciate it for this reason
2019:01:08 17:44:54 urzds In one of my cases these functions are GraphQL resolvers, invoked by Lacinia. So there really is nothing coming afterwards in my own code where I could check the return value... (And GraphQL schemas are not as expressive as Clojure Spec.)
2019:01:08 17:45:29 borkdude @urzds you can try to write a generative test for it.
2019:01:08 17:45:47 urzds Maybe my understanding of Spec is wrong, though. I thought I should spec everything that goes into my system and everything that goes out of it, in order to ensure that it behaves nicely with others.
2019:01:08 17:46:13 alexmiller instrument has functionality to do stubs and mocking too, which can be used in combination with check
2019:01:08 17:47:51 borkdude a ret and fn spec is really useful to check if your implementation is correct in combination with generative testing
2019:01:08 17:49:04 borkdude @urzds you can always plug in the ret spec checking manually, if you want. just separate out the spec and call valid?
2019:01:08 17:49:34 borkdude or use :pre and :post
2019:01:08 17:50:07 urzds Hm, maybe I should find ways to split up those functions better in order to allow generative testing. Seems a bit difficult right now, because they rely on input from external services. So I just have unit tests for my internal functions, and wanted to rely on spec throwing exceptions when the interaction with the outside world shows signs of problems.
2019:01:08 17:50:51 borkdude you can also use spec/assert
2019:01:08 17:50:53 borkdude lots of options
2019:01:08 17:51:27 urzds :pre and :post do not work here, because the functions are actually protocol methods (could be changed to multi-methods, but those also do not support :pre and :post).
2019:01:08 17:51:27 borkdude spec/assert can also be elided with compile time options.
2019:01:08 17:52:53 urzds spec/assert is what I am using right now. The whole body of the function is wrapped in spec/assert, which looks a bit ugly TBH and the commit introducing this will cause a large amount of code to be reformatted for indention, which is also undesirable.
2019:01:08 17:53:40 borkdude then don’t wrap. you don’t have to. and when you elide the call, you’ll be left with an empty function
2019:01:08 17:56:40 urzds How would I not wrap and still check the return value, in the absence of :post?
2019:01:09 11:43:23 borkdude yeah same idea. you could also do it with a macro to receive better line errors
2019:01:09 11:45:49 urzds Is there a way to retrieve the specs attached to the symbol using s/fdef? That might make the above function integrate a bit better with the rest of spec.
2019:01:09 11:46:16 borkdude yes, (s/get-spec foo)` and then call :args or :ret on it.
2019:01:09 11:46:56 borkdude but another option is to spec the args and ret spec separately, so you can write a more specific generator for it
2019:01:09 11:49:36 borkdude so you’re better off passing the var then
2019:01:09 11:49:48 urzds > but another option is to spec the args and ret spec separately, so you can write a more specific generator for it
How do you mean? Doesn't s/fdef have the args and ret spec separately already?
2019:01:09 11:55:58 borkdude I’m not saying you have to, but it gives options
2019:01:09 11:56:10 urzds Thanks!
2019:01:08 17:53:41 borkdude gotta go
2019:01:08 17:57:31 urzds Guess I gotta read more docs about this subject.
2019:01:08 18:39:51 dustingetz Is it possible to spec this: #<Track: [nil :fiddle/links 1234] – a tuple inside a deftype
2019:01:08 19:39:26 alexmiller deftypes do not expose their structure (other than the type)
2019:01:08 19:39:50 alexmiller whether you can spec it in some other ways depends on what interfaces/protocols you implement and which predicates you want to spec it with
2019:01:08 19:40:26 alexmiller so without more info, I would default to: no :)
2019:01:09 16:07:42 ShaneLester What’s the normal way I should be handling speccing functions that have optional arguments?
2019:01:09 16:12:52 alexmiller generally you should use regex ops (like s/cat) to spec args and ops like s/? to handle optional arguments within that
2019:01:09 16:13:39 ShaneLester Cool. Thanks 🙂
2019:01:09 20:17:54 jstew Is there a way to match an exact number of items in a collection or sequence? Let’s say I want to spec a seq or collection to have 3 and only 3 items in it?
2019:01:09 20:19:10 jstew I suppose I could always assert that in the spec for the fn that uses the collection.
2019:01:09 20:23:38 tangrammer As spec guide states 🙂 …
2019:01:09 20:35:15 alexmiller s/tuple is good for heterogeneous fixed-size (“slotted”) vectors
2019:01:09 20:35:44 alexmiller s/coll-of with the :count option is good for homogenous (all items match the same pred) collections
2019:01:09 20:36:01 alexmiller like (s/coll-of double? :count 3)
2019:01:09 20:36:32 jstew That fits this use case better. Thanks, Alex.
2019:01:09 20:36:50 alexmiller either will work and they conform and gen pretty similarly but I think one or the other usually has a better match on intent
2019:01:09 21:07:50 ShaneLester I’m attemping to make a spec for something that can either be any number of vectors, or 0 or 1 strings. currently I have (spc/or :vectortype (spc/* vector?) :stringtype (spc/? string?)) but that doesn’t really seem to be working. (I know this is whacky but, using it to learn the nuances of spec
2019:01:09 21:08:04 ShaneLester Anyone see if I am doing something obviously wrong there?
2019:01:09 21:09:34 manutter51 Looks fair to me, what are you getting from that that makes you say it’s not working?
2019:01:09 21:10:46 ShaneLester Getting a message in-console saying that the function call does not match the spec when the argument that that spec covers is just a string
2019:01:09 21:13:48 manutter51 Function arguments need to be inside (spc/cat ...) maybe you’re missing that?
2019:01:09 21:14:34 ShaneLester I guess this would have been more useful- here is the whole spec for the function currently (spc/fdef typog :args (spc/cat :props map? :children (spc/or :vect (spc/* vector?) :stringt (spc/? string?))))
2019:01:09 21:14:42 ShaneLester so I do have cat. but not sure if im using it right
2019:01:09 21:16:33 manutter51 Do you have the :ret key in your fdef too? I’ve seen odd results when that was missing.
2019:01:09 21:16:54 ShaneLester Hmm I dont, ill try adding that
2019:01:09 21:25:09 manutter51 Everything else looks good to me, with the caveat that I’m still getting comfortable with spec myself, so I might be missing something.
2019:01:09 21:30:40 ShaneLester I just threw a :ret any? in there, but same results sadly. Yeah. It seems to logically make sense to me, and to others I’ve showed it to. Something strange going on perhaps…
2019:01:09 21:30:44 ShaneLester I appreciate your help 🙂
2019:01:09 21:43:48 alexmiller the s/or is not a regex op so inserts a new “level” of nesting
2019:01:09 21:45:15 alexmiller so that is always expecting 2 args with the first a map and the second either a collection of 0 or more vector or an empty collection or a collection containing one string
2019:01:09 21:45:25 alexmiller I don’t think that’s what you want
2019:01:09 21:46:02 alexmiller I suspect changing the or to alt is probably closer, would at least match the string case you want
2019:01:09 21:46:38 alexmiller but depends what you mean by “0 or 1 strings”
2019:01:09 21:47:52 alexmiller if you mean “no 2nd arg” for “0 strings”, then that case is already covered by (* vector?), which can be 0 vectors
2019:01:09 22:02:32 ShaneLester Ahh it was alt! That is what I needed. I did gloss over that in the api but apparently that did not make enough sense for me. haha. Thank you alex.
2019:01:09 22:19:01 ShaneLester Also good point about the case of the arg not being there handled by the *, that is definitely true.
2019:01:11 14:35:08 mping Hi
2019:01:11 14:35:41 mping how can I create a spec for a map like
2019:01:11 14:52:53 mping tks Alex, pretty much what I had, problem must be somewhere else.
2019:01:11 15:03:59 ikitommi @mping you might want to add :conform-keys true to map-of
2019:01:11 15:07:22 alexmiller with a pred, the keys will will just conform to themselves, so that’s just making it slower
2019:01:11 15:23:24 mping @alexmiller in what cases would conform-keys be needed?
2019:01:11 15:23:32 mping core specs?
2019:01:11 15:26:38 alexmiller cases where the keys need to be conformed
2019:01:11 15:27:02 alexmiller :)
2019:01:11 15:27:59 alexmiller most maps have keys that are strings, numbers, keywords, uuids, etc - all of which conform to themselves, so there is no reason to conform the map keys
2019:01:11 15:28:51 alexmiller but if you had a map whose keys conformed to a value other than themselves (a regex spec, an s/or, a nested map, etc) then you would want to consider it
2019:01:11 15:29:48 alexmiller but even in that case, you have to think carefully to make sure the conformed values don’t create a case where two map keys have the same conformed value. if so, they will collide in the conformed map.
2019:01:11 15:30:32 mping I see.
2019:01:11 15:31:00 mping thanks guys
2019:01:12 00:08:21 blance Is it idiomatic to use spec to validate data with remote service through http request?
2019:01:12 00:09:22 seancorfield @blance I think it's idiomatic to use spec for validating data in a lot of cases. Can you be a bit more specific?
2019:01:12 00:10:46 seancorfield For example, we use to spec to validate (and conform) input arguments to our REST APIs and in some other places either before or after interaction with an external system.
2019:01:12 00:11:34 blance say I have a user id that needs to be an int, normally i would just validate it using int?. i'm just curious does it make sense to also do
(def ::user (s/and int? user-exist?))
2019:01:12 00:11:55 blance where user-exist would have to call to an external service to check if user exists
2019:01:12 00:16:28 seancorfield I'd be careful about using predicates that rely on outside systems for your specs. That feels wrong to me.
2019:01:12 00:16:33 lilactown if you try and use ::user with generators, are you going to be upset if it sends hundreds of thousands of requests to your external service?
2019:01:12 00:17:09 seancorfield Yup. I would say it's worth trying to stick with specs that can be used in generators.
2019:01:12 00:17:35 lilactown sometimes it's surprising which spec functions use generators too. e.g. instrumentation
2019:01:12 00:19:00 blance i haven't really start looking at generative testing yet
2019:01:12 00:19:31 blance i suppose http request can be mocked during testing?
2019:01:12 00:25:02 blance reading more on the doc,
A key design constraint of spec is that all specs are also designed to act as generators of sample data that conforms to the spec
sounds like I shouldn't be using any non trivial pred that can't easily generate good sample data
2019:01:12 12:37:07 misha You can use non-trivial predicates (and, I think, it is one of the value props of spec), but if you end up using it for generative testing you will have to help that predicate with a custom generator to minimize "Couldn't satisfy such-that predicate after 100 tries" errors (https://clojure.org/guides/spec#_custom_generators)
You can write predicates with side-effects, but as with usual code – try to isolate them from the pure ones, and know all the contexts where those might be used. So checking if user exists is perfectly fine, but probably should be done at a very specific points of the system, separate from checking hypothetical user's structure.
Also: "not everything needs to be a spec", as well as "not everything needs to be specced". @blance
2019:01:12 23:51:45 blance that make perfect sense, thanks!
2019:01:12 20:38:30 trevor I'm struggling on how to create a generator that returns a constant value
2019:01:12 20:38:43 trevor my example is UUID or constant id
2019:01:12 20:39:47 trevor let's say the constant is 0
2019:01:12 20:45:09 alexmiller (s/gen #{::uuid})
2019:01:12 20:46:46 trevor I was looking for gen/return
2019:01:13 11:45:46 borkdude I have a spec of map-entry and seqable-of-map-entry. Sometimes I get this exception while generating values:
(require '[clojure.spec.gen.alpha :as gen])
(require '[clojure.spec.alpha :as s])
(s/def ::map-entry
(s/with-gen map-entry?
(fn []
(gen/fmap first
(s/gen (s/and map? seq))))))
(s/def ::seqable-of-map-entry
(s/coll-of ::map-entry :kind seqable?))
(gen/sample (s/gen ::seqable-of-map-entry))
Error printing return value (ClassCastException) at clojure.core/conj (core.clj:82).
java.lang.String cannot be cast to clojure.lang.IPersistentCollection
2019:01:13 11:46:35 borkdude I wonder why it tries to call conj at all here
2019:01:13 11:49:29 borkdude I can imagine it tries to build a seqable, so it starts with an empty string, and then it tries to conj map-entries to it:
(conj "" (first {:a 1}))
2019:01:13 11:51:18 borkdude if this is a bug, I would be happy to file it in JIRA
2019:01:13 11:52:40 borkdude for now I can use this workaround:
2019:01:13 12:25:29 borkdude so kind must be a predicate and cannot be a spec, but it must also generate. in other words, you can only use pre-defined predicates?
2019:01:13 15:04:38 borkdude nope, it really should be a seqable, since (java.util.HashMap. {:a 1}) is also supposed to work.
2019:01:13 15:08:28 borkdude every might be the one I should use then
2019:01:13 15:17:07 borkdude that’s it. every also only checks a maximum number of elts, so a lazy infinite seq would still be supported. thanks. :duck:
2019:01:13 15:19:15 borkdude isn’t this a bit inconsistent, since nil puns as an empty sequence?
2019:01:13 15:58:45 borkdude (s/conform (s/every string? :kind seqable?) nil) works though, but then I’m back into the same problem where I started:
user=> (gen/sample (s/gen (s/every string? :kind seqable?)))
Error printing return value (ClassCastException) at clojure.core/conj (core.clj:82).
java.lang.String cannot be cast to clojure.lang.IPersistentCollection
2019:01:13 15:59:53 borkdude so maybe (s/nilable (s/every ::map-entry)) is best then
2019:01:13 16:12:02 alexmiller :kind seqable? is just not right here. every (like coll-of) is a spec for collections (not seqables). you are using a broader predicate for :kind than the spec is intended for.
2019:01:13 16:15:36 borkdude why is (s/nilable (s/every ::map-entry)) not the right fit for seqables? the implementation uses seq on the input and then checks every element up to a limit. so if this is not it, what’s the alternative?
2019:01:13 16:19:37 borkdude if you don’t specify the kind to every, what’s the default?
2019:01:13 16:42:54 alexmiller I’m saying seqable? does not make sense with coll-of/every because some seqables are not collections
2019:01:13 16:43:09 alexmiller The default is vector iirc
2019:01:13 18:39:00 borkdude @alexmiller would this be OK?
2019:01:14 06:25:30 devth i wonder why aren't specs first class things we can define literally and pass around, and why they need their own special registry when we already have namespaces and vars? 🤔
i'm playing with building up specs in an automated way (e.g. reducing a data structure into a spec representation). looks like i'm gonna have to s/def things along the way. isn't that some kind of PLOP? maybe this is something that will be improved in future version 😄
2019:01:14 09:34:47 mpenet The latter it seems
2019:01:14 09:36:49 mpenet Why another registry, I believe to not clutter vars/nses more and make their potential evolution separate?
2019:01:14 15:33:33 devth then why not have a registry for all atoms? and another for all fns? and on and on. 😂
2019:01:14 15:35:57 mpenet There are already issues with vars initialization at startup right now, I guess that's not to make it worse among other thing. Then it's a "private" thing, we never get exposed to the fact it's separate
2019:01:14 11:04:55 misha @devth there is s/spec which gives you spec w/o being registered, which might be useful for you if you build specs dynamically, use, and throw away right away.
2019:01:14 13:58:31 alexmiller there are no plans to change the registry aspect of spec
2019:01:14 13:59:42 alexmiller all of the programmatic construction aspects will be different in spec 2
2019:01:14 15:45:03 flyboarder @alexmiller could you elaborate more on that? Will existing dynamic specs break?
2019:01:14 15:56:02 alexmiller hard to say yet
2019:01:14 16:20:12 devth
2019:01:14 16:31:03 rapskalian Does there exist a flavor of s/merge for disjunctions, i.e. s/or? I have a base set of disjunctions (A or B) that I’d like to be able to extend in certain contexts (base or C or D).
2019:01:14 16:45:27 borkdude @devth Isn’t this the flaw of current spec that Rich spoke about in his recent Clojure Conj talk?
2019:01:14 16:45:27 borkdude @devth Isn’t this the flaw of current spec that Rich spoke about in his recent Clojure Conj talk?
2019:01:14 16:49:05 devth i'm not sure i've seen his latest conj talk. i'll have to look it up
2019:01:14 16:59:25 devth oh, i have seen part of that one.
2019:01:14 17:54:01 devth still curious about the place vs value oriented nature of spec 🤔 any thoughts?
2019:01:14 17:54:22 borkdude I recommend watching the talk.
2019:01:14 17:54:41 devth k, i'm 30 min in. still watching 🙂
2019:01:14 17:55:02 borkdude let it sink in on the hammock 😉
2019:01:14 18:01:04 devth taking notes
2019:01:15 00:54:14 devth Big ideas:
1. separating out schema (shape) and selection (optionality). cool stuff!
- it's still not "just data". why the departure? everything in Clojure is just data, and for good reason. the entire core lib is built around manipulating data. it's the best part of clj.
- it's still place oriented, at least from what i gather so far 😞
2. Better programmatic manipulation of specs. great! but again: why not make it data? then programmatic manipulation comes with.
Imagine if the schema for:
There's elegance in mirroring the data structure you're specifying, kinda like how Datomic Pull queries mirror the data you get get back.
🤔
2019:01:14 16:46:55 borkdude or do you mean “why do I have to gives names to each sub-spec, can’t you inline them?” ?
2019:01:14 16:49:07 borkdude it may be related to each other
2019:01:14 16:49:21 devth yeah, both i think. i want values
2019:01:14 16:50:15 borkdude future spec will allow you (I think!) to spec the whole schema and then define selections on them, so you don’t have to name each selection.
2019:01:14 16:50:27 borkdude I’m not entirely sure if this maps to your issue with spec, but just my 2cts
2019:01:14 16:51:13 devth any word on timeline for the next version of spec?
2019:01:14 16:54:58 rapskalian Whenever I’ve seen this asked, the short answer is “when it’s ready” 🙂
2019:01:14 16:55:18 devth haha. fair
2019:01:14 16:56:57 alexmiller I’m working on it every day
2019:01:14 16:58:13 alexmiller hard to say when the next point will be where it’s useful to look at but it won’t be “a” next version - it’s going to be a series of releases
2019:01:14 17:00:02 devth awesome
2019:01:14 17:00:31 devth so to be clear: currently there isn't necessarily a better way to achieve what i'm doing? i have to dynamically s/def a bunch of stuff?
2019:01:14 17:01:10 borkdude @devth there is a lib called spec-tools which may help you accomplish this, but I expect breaking changes when new spec comes out
2019:01:14 17:01:23 borkdude I haven’t used it in anger myself
2019:01:14 17:04:39 devth > Just data, no macros
💯
2019:01:14 17:04:50 devth long READMEs are nice 🙂
2019:01:14 17:06:11 alexmiller most of things spec-tools is built on are going away, but there will be one or maybe even two alternative paths to this goal
2019:01:14 17:06:19 borkdude I’m not sure if I agree. Do one thing well also has benefits 🙂
2019:01:14 17:07:24 devth not familiar enough with spec-tools to have an opinion on whether it should have been multiple libs. but long searchable READMEs with lots of examples are always nice.
2019:01:14 17:07:51 devth i think i'll play with spec-tools for now. if things change i can always port to the new way
Is this trick for shorter namespaced keywords likely to get me scolded by experienced clojure spec devs? Is it better to formally create the ns file even if it’s largely empty?
2019:01:14 17:14:43 borkdude @cjsauer there is something like this in spec itself, so I think it’s ok
2019:01:14 17:15:55 borkdude @cjsauer if you’re writing portable code, be aware that this may not work on CLJS
2019:01:14 17:17:27 borkdude @cjsauer I tried moving this to a proper ns, so CLJS could use the alias as a namespaced keyword as well, but that was rejected:
https://dev.clojure.org/jira/browse/CLJ-2421
2019:01:14 17:19:24 rapskalian Ah interesting, thank you. in-ns is a bit better. Which part of this does cljs not like? I am hoping to use these specs on both server and client.
2019:01:14 17:19:51 borkdude @cjsauer in-ns simply doesn’t work in CLJS.
2019:01:14 17:20:08 borkdude it does, but only in the REPL.
2019:01:14 17:20:29 rapskalian Bummer…does the same go for create-ns?
2019:01:14 17:20:32 borkdude yes
2019:01:14 17:20:39 borkdude (I think so, gonna check now)
2019:01:14 17:23:54 borkdude
(ns dude)
(in-ns 'foo)
(defn foo [x])
(in-ns 'dude)
(defn dude [x])
$ clj -Sdeps '{:deps {org.clojure/clojurescript {:mvn/version "1.10.439"} org.clojure/test.check {:mvn/version "RELEASE"}}}' -m cljs.main -t node -c dude
WARNING: dude is a single segment namespace at line 1 /private/tmp/repro/src/dude.cljs
WARNING: Use of undeclared Var dude/in-ns at line 3 /private/tmp/repro/src/dude.cljs
WARNING: Use of undeclared Var dude/in-ns at line 7 /private/tmp/repro/src/dude.cljs
2019:01:14 17:25:46 rapskalian Shoot…so then portable code will have to use the qualified keyword everywhere at the moment?
2019:01:14 17:26:04 borkdude the fully qualified one yes
2019:01:14 17:28:37 rapskalian I may actually fall back to formally creating the namespace file for now, even if its mostly empty. Keyword fatigue is real…
2019:01:14 17:28:37 borkdude it’s not a huge pain. you could make a function that creates the keyword if you want to save characters
2019:01:14 17:29:00 borkdude @cjsauer we also have that in the app I’m working one, several almost empty ns-es for this…
2019:01:14 17:30:03 rapskalian Once CLJ-2123 is patched, I imagine it should be quick work to refactor
2019:01:14 17:30:41 borkdude cool. hopefully CLJS gets this too
2019:01:14 18:06:49 rapskalian What is the philosophy behind spec’ing attributes that are intended to be used as references to other entities? For example, I might have ::company/employees which models a one-to-many relationship. How would I spec this attribute? My intuition is to use something like (s/coll-of ::employee), but the issue here is that I now have to define some minimum key-set that is ::employee, which is a spec that I think is largely contextual. Further, there may be contexts where (s/coll-of (s/tuple #{::employee/uuid} ::employee/uuid)) are valid ::company/employees, e.g. datomic idents.
Ultimately what I think I’m wondering is: can reference attributes only be spec’d within a given context, or at specific boundaries (e.g. fdef)?
2019:01:14 19:17:31 rapskalian I think my confusion stems from the fact that a :db.valueType/ref attribute can take on lots of different forms throughout an application, and so it’s difficult to spec. For example, it could be a raw entity id, a {:db/id 000} map, a [:db/id 000] ident, some kind of custom [:employee/uuid #uuid "000"] ident, or it could be a fully formed employee map…all these can values live under the same ref attribute at one time or another.
2019:01:14 20:02:15 favila I think only spec-ing the pull form (ie what you would get from a pull)
2019:01:14 20:02:49 favila Those other variants are all part of the Tx map DSL
2019:01:14 20:03:16 favila This just shifts the problem around though
2019:01:14 20:03:53 favila I think the TX spec would only spec the coll not the keys individually
2019:01:14 20:06:23 rapskalian >I think only spec-ing the pull form (ie what you would get from a pull)
I’ve been playing with a similar strategy. This way you can still make good use of generators without the concept of a database getting involved.
2019:01:14 20:12:12 rapskalian >I think the TX spec would only spec the coll not the keys individually
@favila by this do you mean something like (s/coll-of (s/keys))?
2019:01:14 20:16:05 favila Coll-of whatever a tx is
2019:01:14 20:20:58 favila Coll of either Vecs with entity ref (= 2-tuple, long, or keyword) first item (Tx fns and raw add/retract) or tx maps; which are maps with entity ref keys and Tx map or col-of-txmap or entity ref or non-ref value or coll of non-ref value)
2019:01:14 20:21:07 favila So pretty open...
2019:01:14 20:28:37 rapskalian In my tx function fdefs I’ve been specifying the :ret much more tightly. I’m less interested in spec’ing the :ret of any tx function, and more interested in spec’ing this specific tx function.
2019:01:14 20:35:49 rapskalian I’ve been staring at the code more, and I think what’s troubling is that when I want to specify the form of some ref attribute in general, it can’t really be done…the question of when is inescapable so to speak. Because if I specify the pull form for all of my ref attributes, now I’ve bound the keyword :company/employees to one specific context: the fully realized one. I can’t use the name :company/employees in other contexts in which it might make sense, because spec has already registered the “one true form”. Does this make sense, or am I complicating the issue? 🤔
2019:01:14 20:37:20 rapskalian For example, a tx function might return a collection that includes a map with the :company/employees keyword in it, but it may not conform to the pull form. It may be a collection of idents instead, meaning it references pre-existing entities.
2019:01:14 20:38:19 rapskalian Because I’ve already spec’d it to conform to the pull form, I can’t use that keyword again in a new context…
2019:01:14 20:42:47 lilactown Assuming it's a map using s/keys, you could make everything optional
2019:01:14 20:43:48 rapskalian I had the same idea, and it works okay. I think the downside is that it forces you to resort to custom generators pretty much everywhere…
2019:01:14 20:44:06 lilactown But this is the exact problem Rich stated in his last conj keynote Maybe Not
2019:01:14 20:44:25 lilactown There's not a great solution for it in spec
2019:01:14 20:46:43 rapskalian >But this is the exact problem Rich stated in his last conj keynote Maybe Not
I think so as well. Pretty much anywhere that a map has to be specified, this problem will arrive. The concept of “entity” is still difficult to grasp in my opinion. “Entity” doesn’t seem to become concrete until some data arrives at a boundary/function.
2019:01:14 21:16:57 favila @cjsauer you are exactly right about the problem
2019:01:14 21:17:08 favila I don't think this is the same problem Rich has acknowledged
2019:01:14 21:17:48 favila spec has a strong opinion that the keyword of a map is (essentially) describing the type of its contents
2019:01:14 21:18:17 favila but there are cases where that is not true and the keyword is really a structure rather than a type tag
2019:01:14 21:18:42 favila one is DSLs: e.g. tx map dsl, or even the pull expression dsl
2019:01:14 21:19:14 favila in those cases the key of the map is a structural tag, not a type tag
2019:01:14 21:19:59 favila and foreign system's data routinely make their map entries contextual to the type of thing they are in
2019:01:14 21:20:20 favila both these use cases I have found are bad fits for spec; the only workaround is predicates
2019:01:14 21:20:29 favila or more keys and lots of key renaming
2019:01:14 21:21:15 favila "structural tag" is wrong
2019:01:14 21:21:34 favila what I mean is they are like structurally-expressed arguments to some function call implied by the dsl
2019:01:14 21:22:37 favila {:my/attr [:db/id :db/ident]} is not expressing a :my/attr value, it's telling you to do something to a :my/attr value in some different context
2019:01:14 21:23:09 favila they can't be interpreted apart
2019:01:14 21:23:24 favila so it's the type of the entry itself that matters rather than the type of the value
2019:01:14 21:24:08 favila so, maybe multi-spec on the key as the dispatch value is getting towards the right answer?
2019:01:14 21:52:29 rapskalian >I don’t think this is the same problem Rich has acknowledged
@favila I’ve just rewatched Maybe Not, and I think you’re right, it’s not the same issue. It is exactly the type/structural conflation that you’ve identified.
2019:01:14 21:54:12 favila if anything, rich is moving even further away from keywords having any contextual meaning
2019:01:14 21:54:28 favila s/keys at least let you attach predicates to the whole
2019:01:14 21:57:42 rapskalian He actually also mentions that :ret in function specs is starting to smell, and it’s in those :ret specs that I encountered this issue. So that may be telling…
2019:01:14 21:58:49 rapskalian If I ease up on trying to “nail everything down” with specs, as Rich puts it, it may alleviate some of these limitations
2019:01:14 21:59:43 borkdude What did he say about ret specs again?
2019:01:14 22:09:04 borkdude what he’s saying there is that ret specs don’t tell you much, you almost always want fn specs. so he wants to refine fn specs (make them easier to use?)
2019:01:14 22:30:43 rapskalian True…I think the root of it is definitely this “contextual keys” idea (keywords have different specs in different contexts). What might be cool is the ability to define a “disjunctive schema”:
(s/def ::widget (s/or* :A string? :B pos-int?))
and then use a function similar to Rich’s proposed s/select function in order to mask the contextually relevant predicates, e.g. (s/select* ::widget [:A]) would mean “in this context, only condition :A can match (anything other than a string would result in error)“.
2019:01:14 22:33:09 rapskalian Almost reminds me of tools.deps’ alias flag clojure -A:a:b:c
2019:01:15 01:04:45 favila
2019:01:15 01:05:31 favila This was something I wrote to tackle the specific case of having a more general spec (as a key in a map) that context (what kind of map it is in) would narrow
2019:01:15 01:06:28 favila e.g. :a could be 1 2 or 3, but when it's in a map with :map-type "foo" :a must be 1 or 2
2019:01:15 01:08:01 favila dealing with that case over and over with predicates was a hassle, so this macro (keys+) allowed adding an additional keyword->spec mapping for validity, conformance, and generator purposes
2019:01:15 01:08:20 favila but the original "natural", "contextless" spec still had to validate
2019:01:15 01:09:33 favila so it's not aimed at the DSL use case, this was to deal with data from a foreign system that had a kind of tagging/inheritance relationship. their fields and entites had a base definition that was overly broad, and could be narrowed without changing their structure
2019:01:15 01:10:26 favila think xsd facets where the element names don't change
2019:01:15 01:11:14 favila the purpose was to allow generic processors to understand everything, but particular producers/consumers to add additional constraints about what they would produce/consume
2019:01:15 01:11:44 favila e.g. a field is cardinality-many, but some processor guarantees they only ever put one value in that field
2019:01:15 12:25:30 misha @cjsauer you can s/or or s/alt lookup-refs spec with s/keys. Or avoid speccing maps with lookup-ref values (`{:my/attr [:db/id :db/ident]}`). Or, if you can isolate lookup-ref key-values from actual data key-values, spec lookup-refs with key-agnostic spec, something like (s/map-of qualified-keyword? lookup-ref-spec).
I don't think having s/select would solve spec overload (data or lookup) in this case, because: https://clojurians.slack.com/archives/C1B1BB2Q3/p1547474311308100
2019:01:15 12:27:59 misha Another option is to have s/or + (s/map-of keyword? not-a-look-up-value?) where you expect already looked-up values.
2019:01:15 16:43:13 ShaneLester So I have [org.clojure/test.check "0.9.0"] in my dependencies, but when I’m trying to use generators with spec I’m still getting a clojure.test.check.generators never required error…. Anyone have an idea of what I should try? Is there a different one for cljs potentially?
2019:01:15 16:52:23 borkdude have you tried requiring it manually?
2019:01:15 16:52:29 borkdude @shanelester55 are you on CLJS or CLJ and which version?
2019:01:15 16:53:58 ShaneLester I’m on CLJS. Using shadow-cljs so… not positive which version. Whichever version it is currently using, which I imagine is the latest.
2019:01:15 16:54:10 ShaneLester Is requiring it manually just specifiying it at the top of the file where it is used?
2019:01:15 16:54:44 borkdude you should be able to know the CLJS version in order to deal with known issues. can you try *clojurescript-version* in the REPL?
2019:01:15 16:57:02 borkdude there were some changes around this recently, but they are only on master, not released yet, that’s why I’d like to know
2019:01:15 16:57:25 ShaneLester Sure thing. trying that now… for some reason giving me trouble
2019:01:15 16:58:56 borkdude if you don’t have a REPL, just print it the console from your app
2019:01:15 16:59:57 borkdude or type cljs.core._STAR_clojurescript_version_STAR_ in the browser console
2019:01:15 17:00:28 ShaneLester Ah that worked easily enough. "1.10.439"
2019:01:15 17:01:52 borkdude ok, how do you use the generators. using clojure.spec.test.alpha?
2019:01:15 17:02:38 ShaneLester Currently using these
2019:01:15 17:02:57 borkdude ok, I imagine you want to call gen/sample to just see some generated data right?
2019:01:15 17:03:00 ShaneLester correct
2019:01:15 17:03:34 borkdude you’ll need a require to [clojure.test.check.generators] in your ns as well
2019:01:15 17:04:08 ShaneLester Ahhh that did it.
2019:01:15 17:04:12 ShaneLester Thank you very much
2019:01:15 17:05:43 borkdude actually just [clojure.test.check] should work too
2019:01:15 17:06:26 borkdude since it requires all the other needed namespaces
2019:01:15 17:06:50 ShaneLester Makes sense. That indeed works as well.
2019:01:15 17:07:53 ShaneLester Thanks again 🙂 Works well now
2019:01:15 17:08:10 borkdude Enjoy 🙂
2019:01:15 17:29:50 devth reposting my notes on spec-related content in the 'Maybe Not' talk (https://www.youtube.com/watch?v=YR5WdGrpoug) since the thread is buried.
1. separating out schema (shape) and selection (optionality). cool stuff!
- it's still not "just data". why the departure? everything in Clojure is just data, and for good reason. the entire core lib is built around manipulating data. it's the best part of clj.
- it's still place oriented, at least from what i gather so far
2. Better programmatic manipulation of specs. great! but again: why not make it data? then programmatic manipulation comes with.
Imagine if the schema for:
2019:01:15 20:53:31 jstaab I came here to ask a very similar question. I do agree with the two points you linked above, but it does seem very strange to use static place orientation to achieve it. I'd like to programmatically generate specs based on my own domain modeling dsl, rather than repeat my entity shapes multiple times, use spec as my domain modeling language, or write complex macros to do both at once. I've heard the objection of "why is it so hard to generate composed specs programmatically", and I still haven't heard a good answer.
2019:01:15 20:55:19 schmee they’re working on a new version of spec which is supposedly more data-oriented, there are some details about the development here, but nothing is released publicly yet: http://insideclojure.org/2019/01/11/journal/
2019:01:15 20:55:33 jstaab I've heard of that, I have high hopes 🙂
2019:01:16 06:19:46 ikitommi @U066S8QGH spec-tools has a data-layer on top of specs like you described:
2019:01:16 06:22:48 ikitommi the forms are mapped with a multimethod, so string? gets a 'clojure.core/string? form. doesn’t help with anonymous functions.
2019:01:16 15:01:31 devth looks good! i'm planning to use it
2019:01:15 20:58:14 dustingetz but string? is code, not data
2019:01:15 20:58:37 dustingetz the idea of predicates is core to spec, i dont see how spec could ever be jsut data
2019:01:15 20:59:01 devth data for edges, predicates for leaf nodes
2019:01:15 20:59:23 alexmiller the symbol string? is data
2019:01:15 20:59:39 alexmiller the form (s/coll-of string?) is also data
2019:01:15 20:59:39 dustingetz that’s thin
2019:01:15 20:59:46 alexmiller it’s not thin, this is the whole basis of lisp
2019:01:15 21:00:14 dustingetz homoiconicity is about macros
2019:01:15 21:01:45 jstaab If you call describe it does return a list of symbols at runtime, so spec isn't just confined to macros. Admittedly it could be hard to do anything with more complex predicates
2019:01:15 21:02:26 alexmiller that’s just … not true
2019:01:15 21:03:09 dustingetz for string? to have meaning we’d have to syntax quote it for starters, and now we’re in the clojure reader, not the edn reader
2019:01:15 21:03:10 alexmiller and spec forms will be even more central in spec 2
2019:01:15 21:04:14 alexmiller string? has meaning in the context of a namespace
2019:01:15 21:04:37 dustingetz so now we hve (ns) forms and are very solidly in the clojure reader, not edn reader
2019:01:15 21:05:14 dustingetz that’s how i udnerstand it anyway, always looking to deepen 🙂
2019:01:15 21:05:26 alexmiller if you have fully qualified symbols, then the edn reader is sufficient
2019:01:15 21:05:43 alexmiller for example, s/form always give you that
2019:01:15 21:06:31 alexmiller we are also moving towards a world where spec forms are a data form, but may not be the only data form
2019:01:15 21:06:48 alexmiller so some “map” data form could also exist
2019:01:15 21:07:37 alexmiller but that’s not more or less data than the list form (it may be easier to perform some kinds of transformations on)
2019:01:15 21:08:08 dustingetz yo’re saying (s/coll-of ...) is sugar for something deeper?
2019:01:15 21:08:23 alexmiller no
2019:01:15 21:09:09 alexmiller I’m saying that (s/coll-of string?) is data. Some map syntax {:op coll-of :pred string?} (not a real syntax) is also data. both are sufficient to describe a spec.
2019:01:15 21:09:50 favila I don't think this matters. the power and problem is the predicates. predicates can only be data if you know what all of them mean (independently of an implementation--a well-known list of predicates)
2019:01:15 21:10:53 alexmiller yes, that is essential in the design of spec which from day 1 was predicate based
2019:01:15 21:11:07 favila and that is what people mean when they say "specs are not data"
2019:01:15 21:11:15 dustingetz i accept that a qualified symbol is equivalent to a qualified keyword – but not a closure or fn reference, only symbols
2019:01:15 21:11:37 favila > the idea of predicates is core to spec, i dont see how spec could ever be jsut data
2019:01:15 21:11:48 favila that's I think what you mean @dustingetz?
2019:01:15 21:11:58 favila certainly how I understood it
2019:01:15 21:12:07 favila in my own thinking about the "dataness" of spec
2019:01:15 21:12:08 dustingetz As soon as someone puts #(and x y) as a spec pred, it’s all over, no logner data
2019:01:15 21:12:32 alexmiller well, that’s false and we support that now
2019:01:15 21:12:39 dustingetz how can you serialize that
2019:01:15 21:12:51 alexmiller (fn [&] ...)
2019:01:15 21:12:59 dustingetz its a reference
2019:01:15 21:13:02 dustingetz it could be jvm interop there
2019:01:15 21:13:06 alexmiller or you mean x and y as closed over values?
2019:01:15 21:13:11 alexmiller if so, then agreed
2019:01:15 21:13:22 dustingetz right, it could be anything
2019:01:15 21:13:24 alexmiller although we have some thoughts on how to allow you to do that as well
2019:01:15 21:13:36 dustingetz now that has my interest piqued …
2019:01:15 21:13:56 alexmiller if x and y are vars you’re invoking
2019:01:15 21:14:27 alexmiller if it’s values, then you’re in the realm of custom spec creation (which also is now more codified and easier to do)
2019:01:15 21:14:49 alexmiller well, maybe not that much easier, but I think we’re ready to commit to what that means
2019:01:15 21:16:20 alexmiller I would still say that pred references are data if it’s possible to re-establish their meaning remotely (in the context of namespaces) and var serialization / hydration is something that we’ve been slowly working on for the last year or so. some of it’s in clojure already.
2019:01:15 21:16:55 dustingetz Aren’t you going to run up against the halting problem
(= #(even? (inc (inc %)))
#(even? (+ 2 %)))
2019:01:15 21:17:04 alexmiller we may ultimately end up going beyond var quote to define something new, but we’re still just thinking about those parts
2019:01:15 21:17:21 alexmiller why do you need to do that?
2019:01:15 21:17:31 alexmiller we’re never comparing specs for equality
2019:01:15 21:17:58 dustingetz because you said it’s data, so i want to do data things, otherwise doesn’t that defeat the point
2019:01:15 21:18:25 alexmiller this is too theoretical. what problem are you trying to solve?
2019:01:15 21:18:27 dustingetz
(= `string? `string?)
is well defined since they are data and qualified
2019:01:15 21:18:45 alexmiller why are you even doing that?
2019:01:15 21:19:10 dustingetz Well for example if specs are data then i want to store them efficiently in a database
2019:01:15 21:19:23 alexmiller you’re inventing problems that imho don’t exist
2019:01:15 21:20:10 dustingetz it is essential to hyperfiddle that datomic schema is actually data and stored in a database
2019:01:15 21:20:53 dustingetz we match on :db.type/string, for example, to decide what widget to draw
2019:01:15 21:22:07 alexmiller well you’ll be glad we’ve scoped it to just data then - symbols, sets, and forms combining them
2019:01:15 21:22:27 dustingetz lol
2019:01:15 21:22:38 favila inline fns are no longer allowed?
2019:01:15 21:22:44 alexmiller inline fns are forms
2019:01:15 21:22:49 dustingetz well, looking forward to seeing whatever you come up with
2019:01:15 21:23:47 alexmiller debating how bad it would be to push wip stuff there, but maybe I’ll just put it on a branch
2019:01:15 21:24:31 favila spec is data if you have eval and a full clojure runtime and possibly other required nses got it
2019:01:15 21:24:48 dustingetz Like for example, imagine a big honking match, like this but huge, and it tested the specs too. If specs are data that should be possible and could be very interesting.
2019:01:15 21:25:54 alexmiller well specs are an open protocol, so no match is big enough to cover custom extensions
2019:01:15 21:26:11 dustingetz could be multimethod, you get the idea
2019:01:15 21:26:16 alexmiller but sure
2019:01:15 21:27:39 alexmiller specs have a form, keyed by the op in first position (and that’s how resolution occurs in the spec 2)
2019:01:15 21:27:50 misha what is the difference between form and data?
2019:01:15 21:27:57 alexmiller none
2019:01:15 21:28:13 favila data can be understood without evaluation
2019:01:15 21:28:37 favila that can be true for a well-defined subset of spec
2019:01:15 21:28:41 alexmiller as we’re talking forms above, I mean combinations of lists, symbols, keywords, and sets
2019:01:15 21:29:18 misha > data can be understood without evaluation
well, it depends, because your brain is the evaluator which "understands", and knowing format is essentially :require in brain
(if POV is a person)
2019:01:15 21:29:47 alexmiller it depends what you mean by “understood”
2019:01:15 21:32:06 misha arguably, string? can also be understood w/o evaluation, even w/o namespace. so yeah, it depends on definition of "understood"
2019:01:15 21:33:03 favila yes, most derivative uses of spec rely on some implementation-independent meaning for the predicate symbols
2019:01:15 21:33:47 favila but you need to limit that set, or else have some way to mark useful traits about a predicate; otherwise you can't "understand" it without getting a clojure runtime to run it
2019:01:15 21:33:59 misha more specific you (want to) get – more it'll depend on actual implementation behind the symbols
2019:01:15 21:34:42 favila that's what I mean by a well defined subset
2019:01:15 21:38:02 misha it starts to sound like static types argument. There is value in being able to use anything clojure allows to construct a predicate. But if you want it to be understandable over the wire – use subset of what is allowed. Instead of having spec forbid e.g. java interop, because it would not be understandable in 0.0000001% cases
2019:01:15 21:38:12 alexmiller if you want use specs with the spec api calls (valid?, conform, explain, etc), then yes, you need implementations for all predicates mentioned in the specs
2019:01:15 21:38:42 alexmiller this has been explicit in the design of spec from the beginning
2019:01:15 21:39:00 alexmiller none of this is new or really any different
2019:01:15 21:39:06 favila no one is arguing it's not been explicit, or even that it is bad or wrong
2019:01:15 21:39:16 favila it's just annoying to hear that called "data"
2019:01:15 21:39:23 alexmiller that’s baffling to me
2019:01:15 21:40:29 alexmiller as this is no different than the whole “code as data” / macros thing that is part of Clojure, which presumably you are also familiar with
2019:01:15 21:40:38 misha what is "data" to you? @favila
2019:01:15 21:40:41 dustingetz data makes possible a whole new universe of systems in 20 years that we barely even have the words to talk about today (thanks to Clojure) – what seems pedantic today is pretty important in hindsight, i think
2019:01:15 21:41:35 alexmiller just because parts of the data are linked to semantics, does not mean it’s not data
One of these is data, one of these is code (and code is data), but there is a huge difference in terms of what it is coupled to and what it means
2019:01:15 21:42:37 dustingetz One of these can be used to generate UI or compile to SQL, the other can’t
2019:01:15 21:43:17 misha both is edn data, no? vectors, lists, keywords, symbols
2019:01:15 21:43:40 alexmiller well this has been fun, but I’m going to bow out here and go work on the code
2019:01:15 21:43:45 dustingetz Thanks Alex
2019:01:15 21:44:33 lilactown I think that there is a valid desire to have specs be represented in a form that doesn't require a Clojure interpreter in order to use
2019:01:15 21:45:12 lilactown we'd probably prefer to have a simpler DSL, using a subset of EDN, that would enable easier parsing and consuming
2019:01:15 21:45:19 misha for example, how do you know what ?e is in 1st line? you need to provide "this is variable placeholder to be bound because datomic" context somehow somewhere.
if you don't - this is exactly the same as 2nd line, but in a list, not vector
2019:01:15 21:45:49 dustingetz Once it is coupled to Clojure, it is an order of magnitude harder to build a data driven system on top of it
2019:01:15 21:46:04 dustingetz XML is data too
2019:01:15 21:46:33 lilactown the trick is to tease out, what is the minimum set of semantics required to express a spec?
2019:01:15 21:47:01 lilactown we can also turn the knob on that by limiting the ease and power of specs
2019:01:15 21:47:39 lilactown e.g. right now, specs use predicates that you can easily inline. what if we disallowed that (either by convention or some other tricky mechanism)?
2019:01:15 21:47:59 lilactown that would make it easier to serialize, but potentially less powerful and harder to write
2019:01:15 21:48:11 lilactown "every predicate must have a qualified name"
2019:01:15 21:48:29 dustingetz Or just say “hey this thing is programmable, have at it”, that’s fine too, but the halfway world is pretty weird and i think leads to fragility
2019:01:15 21:49:16 devth wonder if free monads and the interpreter pattern is useful to consider in this context. spec as a pure data AST.
2019:01:15 21:49:28 lilactown or we could have a convention that says, any predicate must not close over any vars
2019:01:15 21:49:52 misha > One of these can be used to generate UI or compile to SQL, the other can’t
see above
2019:01:15 21:51:01 dustingetz (d/q (->> [:find '?e] (concat [:where ['?e :dustingetz.reg/email]]))) i can play that game all day
2019:01:15 21:51:09 misha it has extra step, yes, but it does not make it not data
2019:01:15 21:51:44 misha well, how do you know the 1st one can be converted to sql?
2019:01:15 21:52:05 misha by looking at it first? you interpret while you look. your brain is repl
2019:01:15 21:52:40 misha brain of someone w/o knowing what edn is will not be able to tell that 1st can be converted and second cant
2019:01:15 21:52:57 misha it's a repl w/o edn/reader required yet
2019:01:15 21:53:23 favila I don't think data = edn
2019:01:15 21:53:39 misha yes, it is less convenient, and requires extra step. if this is the argument - I agree, there is no limit to convenience opieop
2019:01:15 21:53:58 favila if data is just "anything you can quote" then what is code?
2019:01:15 21:54:44 favila when people say "this is data" they are saying it has a constellation of useful properties that are strictly less powerful than a programming language
2019:01:15 21:55:23 misha what does it mean? in this spec conversation context, and ... in general?
2019:01:15 21:55:49 favila e.g., being able to understand and interpret it multiple ways; draw inferences from it that are not stated explicitly; compare it to other things for equivalence; build different interpreters on top of it
2019:01:15 21:56:25 misha you somehow need to parse/read/interpret data to "draw inferences"
2019:01:15 21:56:48 favila yes @misha, you do. You need to assign meaning to things without implementing them
2019:01:15 21:56:57 favila this is possible for spec until you hit a predicate
2019:01:15 21:57:11 misha otherwise how do you know what is in the string? how do you know that this is map: {:a 1}, etc.
2019:01:15 21:57:16 favila if you have a well-known list of predicate symbols, then you can hard-code that meaning in
2019:01:15 21:57:18 misha you have to have some context, it might be in your brain, but you need to communicate it to the system "which might not be using clojure, etc."
2019:01:15 21:57:58 favila but once you have a custom predicate, it's meaning cannot be shared without sharing it as code
2019:01:15 21:58:03 lilactown I don't think it even needs to be well-known
2019:01:15 21:58:11 favila there's no data language that can express a new predicate
2019:01:15 21:58:27 favila spec has parts of its language that can express new specs
2019:01:15 21:58:27 dustingetz Data is also secure, imagine a REST service that returned Clojure code
2019:01:15 21:58:31 lilactown you could receive a spec, and then read all of the symbols from that spec to see what needs to be implemented
2019:01:15 21:58:33 favila e.g. s/or s/and etc
2019:01:15 21:58:55 favila but below predicates, you cannot know anything unless you have independent knowledge of its meaning
2019:01:15 21:59:15 misha I think, that "assigning meaning w/o implementation" is a very blurry line
2019:01:15 21:59:18 favila even executing it you can only know what it means for a specific value you give to it
2019:01:15 21:59:32 misha hardcode that meaning where?
2019:01:15 21:59:34 favila you cannot make any universal inferences about the behavior of a predicate without knowing its meaning
2019:01:15 21:59:48 misha because that is just "global state", and you need to make sure system you are sending it to - is in the same context
2019:01:15 22:00:14 dustingetz Datomic’s :db.type/string is not ambiguous or blurry
2019:01:15 22:00:50 favila @misha having to send it to the same runtime environment to use it sounds an awful lot like a property of code rather than data
2019:01:15 22:01:00 misha show it to java dev and it would be pretty ambiguous to him opieop
2019:01:15 22:01:34 dustingetz The success of HTML over desktop gui toolkits is also because it is data
2019:01:15 22:01:59 dustingetz You can ship HTML over the wire to arbitrary platform, and somehow they manage to interpret it usefully
2019:01:15 22:02:05 misha well, then yours "well defined symbols we could just hardcode" just means "we have implementation :ruquired here in clojure and there in python, so dont worry about it"
2019:01:15 22:02:21 misha which smells like transit
2019:01:15 22:02:41 dustingetz You can render arbitrary HTML fearlessly, you aren’t worried about getting hacked
2019:01:15 22:02:50 favila "well defined" just means we know the list ahead of time
2019:01:15 22:03:09 favila I know what "clojure.core/string?" means
2019:01:15 22:03:19 favila I don't know what "foo.core/blah?" means
2019:01:15 22:03:44 favila If I had the code I could possibly execute foo.core/blah?
2019:01:15 22:03:46 misha html just has very small set of tokens
2019:01:15 22:03:58 dustingetz You do however know what github/markdown? is, you know who the maintainer is, you know where the docs are and where you might find useful code to operate with it
2019:01:15 22:04:26 favila maybe another angle: spec can express things that are not in spec
2019:01:15 22:04:40 favila i.e. the predicates
2019:01:15 22:05:08 misha well, html makes no sense to my mom, neither does english, so can I say it is not data then? ¯\(ツ)/¯
2019:01:15 22:05:30 dustingetz Yet somehow I manage to pay my bills
2019:01:15 22:05:49 misha her brain is a system, and it does not support html. mine does.
2019:01:15 22:06:45 misha @favila you know the list of clojure code ahead of time too. it is just way too large.
2019:01:15 22:07:17 misha the same way we know the set of real numbers
2019:01:15 22:07:42 favila will foo.core/blah? return true for a real number?
2019:01:15 22:08:00 misha one difference would be - you cant eyeball it and say "yep, it's fine, we are not getting hacked"
2019:01:15 22:08:34 misha will the next system interpret 1 as true or int?
2019:01:15 22:08:35 lilactown sometimes you have to reject invalid data if you don't know how to parse it, too
2019:01:15 22:09:03 favila or you convert it to a less meaningful form, e.g. clojure tagged-value
2019:01:15 22:09:20 favila or, a fully qualified predicate symbol
2019:01:15 22:09:53 favila but then the power the spec gained by being able to use any predicate it wants is lost by any other system that wants to make sense of the spec
2019:01:15 22:10:50 favila by making s/valid? easier something else has been made harder
2019:01:15 22:10:57 misha @dustingetz if stuff on github - is data, code is data in exactly same way, so we are on the square 1 again
2019:01:15 22:11:10 lilactown this is going to sound :male-astronaut: but: it becomes just like any other part of our system, where we need to keep a single source of truth for the semantics of that data
2019:01:15 22:11:26 lilactown e.g. maybe in the future we'll be building "spec-services"
2019:01:15 22:11:35 lilactown (I'm thinking of building a "spec-service")
2019:01:15 22:11:42 favila that just sounds hard to do given predicates
2019:01:15 22:12:08 dustingetz @lilactown What is a spec service?
2019:01:15 22:12:36 lilactown well, because serializing the spec itself requires a lot of context, what I'm thinking of doing is just having, literally, spec-as-a-service
2019:01:15 22:12:55 dustingetz so like an xmlns
2019:01:15 22:13:14 lilactown GET valid/foo.core/blah => (s/valid? :foo.core/blah (:body req))
2019:01:15 22:13:34 dustingetz ah cool
2019:01:15 22:15:42 lilactown for instance, we have a lot of disparate services that need to validate similar sets of data according to the same rules
2019:01:15 22:15:54 lilactown and they're not all Clojure 😞
2019:01:15 22:17:51 dustingetz Yes, e.g. imagine internet protocols like HTTP had a formal spec written instead of English, in clojure.spec, and thus it becomes much easier to, well im not sure what i would want to think about it
2019:01:15 22:24:25 jaihindhreddy I was just about to point out hyperfiddle. Then I read your name 🙂
2019:01:15 22:25:05 dustingetz Lol, thanks
2019:01:15 22:18:14 dustingetz It is a bit like HATEOAS
2019:01:15 22:18:38 dustingetz in fact it is HATEOAS
2019:01:15 22:21:04 dustingetz I retract that, but it is really interesting to imagine
2019:01:15 22:21:53 dustingetz If spec is clojure code, it gets a little harder to start throwing them around on wires though, because how can you trust if it is safe to clojure.core/eval
2019:01:15 22:22:07 dustingetz But there is still a trust chain, so its probably fine
2019:01:15 22:22:32 dustingetz But you’re saying a service, so that bypasses the trust issue
2019:01:15 22:22:49 lilactown right. just like in clojure, you have a single, global spec registry
2019:01:15 22:23:28 lilactown your system should have the same: a single, global registry of what your specs are
2019:01:15 22:23:51 favila yeah that's a neat idea
2019:01:15 22:24:15 favila pretty scary to run that server
2019:01:15 22:24:35 favila hopefully no one manages to get a bitcoin mining predicate on there
2019:01:15 22:24:55 favila how to make it fast?
2019:01:15 22:25:06 favila that's a lot of data to push over the wire
2019:01:15 22:25:19 favila (uploads of clojure data to s/valid?)
2019:01:15 22:26:32 dustingetz Right, the huge difference between Datomic schema and Clojure specs today, is that you can recklessly throw them around. For example it’s easier to deploy them into a validated healthcare enterprise environment
2019:01:15 22:26:43 dustingetz Slurp in some EDN updates, not that big a deal
2019:01:15 22:26:54 dustingetz Slurp in some clojure specs to pass to eval – yeah right
2019:01:15 22:28:17 dustingetz I can imagine a dialect of clojure/core that does not have infinite recursion, and no unsafe operators like !, that might be safe enough
2019:01:15 22:30:46 dustingetz You can also make the requestor pay for compute
2019:01:15 22:34:51 lilactown yeah, easier to tag things with specs and send the data you want validated along with the names of specs you want it to be validated against
2019:01:15 23:03:12 dustingetz Here is a cool application of specs as data: [:select.ui (select-keys props [:value :class :style :on-change :read-only :on-click])] aka https://github.com/facebook/prop-types
2019:01:15 23:43:42 misha it funny how
// An object that could be one of many types
optionalUnion: PropTypes.oneOfType([
PropTypes.string,
PropTypes.number,
PropTypes.instanceOf(Message)
]), ...
2019:01:16 13:44:44 dustingetz Alex you convinced me that (s/coll-of …) is data and that predicate equality isn’t a practical objection
2019:01:16 13:44:53 dustingetz Thanks for the deep dive yesterday
2019:01:17 18:23:48 rickmoynihan Is it possible to (s/merge (s/keys :req-un [:foo/foo]) (s/keys :req-un [:bar/foo])) such that the rightmost stomps on keys to the left? i.e. looking for a way to combine s/keys with merge semantics for merging and precedence
2019:01:17 18:26:38 alexmiller define “stomps on”
2019:01:17 18:27:22 alexmiller I guess replaces?
2019:01:17 18:28:17 alexmiller seems like you want some kind of operation on specs themselves rather than composite specs
2019:01:18 09:44:08 rickmoynihan Yes, by stomps on I mean replaces. And yes I’m looking for an operation on specs themselves.
Essentially I want two different specs that share the majority of their keys, but have a different spec for one or two keys.
If such a thing doesn’t exist I guess I can rework it so the commonanility is together in a shared spec, and merge that with the extra different keys… rather than trying to merge the replacement keys over the top.
2019:01:18 09:46:09 rickmoynihan I guess it’s just a little confusing that s/merge has different semantics WRT keys than merge - but I get why s/merge is like this.
I guess I’m after an s/merge-keys for merging s/keys specs.
2019:01:18 09:46:30 rickmoynihan or maybe s/keys-merge would be a better name.
2019:01:18 13:24:51 alexmiller Having different semantics for the same key is generally a smell, unless your override is a subset (rather than a different set). I think there may be some new options in spec 2 but I’d say right now I would separate the non-different keys into one spec, then merge with different partial map specs as needed.
2019:01:18 14:20:25 rickmoynihan Thanks, I’d already done what you suggest…
It’s 3rd party data — so I don’t get to define the keys… but yes the override is actually a subset. Also I’m not sure if this a smell or not; but the superset spec really just exists to represent “noise” that I don’t care about but have to ignore gracefully… I’m not using those specs as specs so much as I’m using the spec API as a way to write generators of noise I need to ignore not barf on.
e.g. I have a spec :raw/Body representing all message bodys I should just skip over and a spec :specific/Body for ones I care about. I could use s/or to model this but Body is nested in some other specs e.g. a Message and essentially I don’t want to have to redefine all the intermediates. I think this is essentially similar to the “Maybe Not” problems.
2019:01:18 14:48:08 alexmiller well you can use s/and now to combine your generic spec and (when necessary) the more restrictive spec
2019:01:18 14:48:23 alexmiller so it doesn’t replace, but only the more precise thing will pass both
2019:01:18 14:50:08 rickmoynihan yeah I did consider that but how do you do so when the spec isn’t the root spec?
2019:01:18 14:52:34 alexmiller yeah, that’s tricky - you need to rebuild everything back to the root
2019:01:18 14:52:50 alexmiller this is the kind of thing that will be easier in the spec 2 select stuff
2019:01:18 14:52:56 rickmoynihan yeah
2019:01:18 14:53:01 rickmoynihan that was my feeling too
2019:01:18 14:54:08 rickmoynihan thanks for confirming my intentions were good, at least! 🙂
2019:01:18 14:55:01 rickmoynihan FWIW I have rebuilt to the root… and rearchitectured the commonality into another spec etc… so it’s not so bad this time 🙂
2019:01:17 23:20:35 kenny Can you merge a map-of and keys?
2019:01:17 23:23:20 taylor yes
2019:01:17 23:24:17 taylor for example, this won’t conform if :bar value isn’t a string (or if any of the keys aren’t keywords, or if :foo is missing, etc.):
2019:01:18 10:25:22 borkdude this is actually the first time I couldn’t instrument something on CLJ that I could on CLJS, normally it’s the other way around 😉
2019:01:18 10:25:38 bronsa I guess instrument could look at the arglists and figure out if it needs to compile the wrapping fn to a primitive fn
2019:01:18 12:35:07 rickmoynihan How do people manage the seperation of concerns around app code, specs and generators?
Generators are essentially a testing concern, but you also use want them at dev time so you can inspect sample values etc…
Part of my issue is that writing generators eats up quite a few LOCs, so I’m tempted to put them in a different ns to the specs and app logic… I also feel like in clojure that they should be optional… which means you don’t really want them in the require tree in the production app.
Are there any patterns people have for arranging these? Currently I just them all in the same ns — but at some point I worry it’ll be hard to see the wood for the trees.
2019:01:18 12:35:32 rickmoynihan Thinking also are there any lazy-loading patterns I can easily apply to my own generators?
2019:01:18 12:36:13 rickmoynihan Also you often want different generators depending on context
2019:01:18 12:37:01 rickmoynihan I know you can override them via st/check etc — but not really sure how to organise all these bits
2019:01:18 12:44:36 borkdude @rickmoynihan clojure.spec.test/check takes overridable generators. I use those, if I don’t make a generator attached to the spec itself.
2019:01:18 12:44:55 borkdude oh sorry, that was your last sentence
2019:01:18 12:51:30 rickmoynihan yeah
2019:01:18 15:36:40 thomas Hi, I am trying to write a generator... but no luck... I am trying to follow the examples from the test.check page... but no luck (yet)
2019:01:18 15:37:42 thomas it can't find certain functions for instance.. while as far as I can work out they should be in the clojure.spec.gen.alpha namespace
2019:01:18 15:53:10 taylor You may have to reference test check namespaces directly. Spec doesn’t expose all the stuff that’s available
2019:01:18 15:37:56 thomas frequency for instance.
2019:01:18 15:38:43 borkdude @thomas are you using CLJ or CLJS?
2019:01:18 15:38:51 thomas CLJ
2019:01:18 15:39:50 borkdude maybe make a small repro of what you’re trying to do so we can take it from there?
2019:01:18 15:41:23 thomas ok will do. thank you. (I can't do it now, but I'll try later tonight) Thank you @borkdude
2019:01:18 16:20:23 rickmoynihan @thomas you know about lazy loading right?
2019:01:18 16:21:27 rickmoynihan you need to be aware that clojure.spec.gen.alpha generators are subtly different to test.check ones
2019:01:18 16:23:28 rickmoynihan I always forget the details… but IIRC generators from spec are wrapped in a 0 arg function to allow lazy loading… so if you’re trying to use test.check generators you need to wrap them one more time in a fn call… or something like that. 🙂
2019:01:18 16:30:25 rickmoynihan @borkdude: respeced looks useful… do you have any machinery to trigger st/check in a deftest and get a good error out from clojure.test when things go wrong?
2019:01:18 16:31:04 rickmoynihan I’ve written a few things like this in the past — but the results have always been lackluster.
2019:01:18 16:32:46 borkdude the crucial part is (clojure.test/is (respeced.test/successful? stc-result#)), succesful? realizes the stc-result (it’s lazy) and checks if there’s at least one
2019:01:18 16:35:22 rickmoynihan yeah I’ve done things like that before — but yours looks like it might give better output…
2019:01:18 16:35:37 borkdude it works fine for me
2019:01:18 16:36:00 rickmoynihan I’m guessing clojure.test just prints the error as (is (not (successful? ,,,)))
2019:01:18 16:36:02 borkdude try it in the REPL and you’ll see 🙂
2019:01:18 16:36:09 borkdude yes
2019:01:18 16:36:24 rickmoynihan Yeah I’m going to 🙂
2019:01:18 16:44:20 rickmoynihan @borkdude: would you say a use case for rt/check-call is to check real sample data conforms to the spec then? i.e. as a way to check that the spec is valid, as much as the data/fdef?
2019:01:18 16:54:39 borkdude @rickmoynihan check-call is made for checking fdefs on example based tests
2019:01:18 16:55:05 borkdude e.g. to verify that your ret spec works given an example
2019:01:18 17:00:57 borkdude one problem I’d like to tackle is: your function accepts arity 1 and 2, but you spec’ed only 2. there’s no way to detect this with stest/check since it only generates 2 args.
2019:01:18 17:01:28 borkdude so fdefs could be insuffient. would be nice if that could be automatically detected.
2019:01:18 17:02:38 rickmoynihan agreed
2019:01:18 17:03:25 borkdude it would be nice if you could generate data to detected that your fdef missed a case. I guess you can
2019:01:18 17:03:46 rickmoynihan +1
2019:01:18 17:04:42 rickmoynihan related what I also find useful is a test of a s/conform against some example data… so you know you spec’d the right thing.
2019:01:18 17:05:37 rickmoynihan but then you find that clojure.test/is is effectively testing a boolean s/valid? is insufficient as you want to see the spec failure not just (is (not (s/valid? ,,,)))
2019:01:18 17:06:38 borkdude that’s why I made check I think
2019:01:18 17:06:48 borkdude this library is aimed at fdef checking
2019:01:18 17:06:55 rickmoynihan yeah
2019:01:18 17:07:07 rickmoynihan but I think there’s an equivalent need for spec checking too
2019:01:18 17:07:31 rickmoynihan as you say check requires an fdef
2019:01:18 17:12:08 rickmoynihan I was writing things like (is (not= :clojure.spec.alpha/invalid (s/conform :spec/here {:sample :data})) but you need to be careful using :clojure.spec.alpha/invalid in these macros because it can cause compilation errors due to :clojure.spec.alpha/invalid being used to check the macro syntax
2019:01:18 17:13:07 rickmoynihan but the reasoning there is that comparing (= ::s/invalid sample-result) gives you a meaningful error in clojure.test.
2019:01:18 17:13:18 borkdude hah, yeah, I had a similar issue when spec’ing assoc, because then you cannot assoc the value :clojure.spec.alpha/invalid which Cursive does when you use the REPL 😛
2019:01:18 17:13:41 borkdude solved that by writing my own ::any spec
2019:01:18 17:13:51 rickmoynihan yeah IIRC there’s a ticket open on the issue somewhere
2019:01:18 17:14:15 borkdude yeah. one of the proposals there is to use a singleton object to represent invalid inside spec and only to the outside use the keyword
2019:01:18 17:14:22 borkdude I think that makes sense
2019:01:18 17:23:42 alexmiller you can use s/valid? rather than checking ::s/invalid
2019:01:18 17:24:03 alexmiller or s/invalid?
2019:01:18 17:24:34 borkdude I used s/invalid?
2019:01:18 17:24:42 alexmiller I don’t think a singleton object is a feasible solution
2019:01:18 17:24:51 alexmiller I’m not even sure I agree it’s a problem
2019:01:18 17:26:05 borkdude @alexmiller It’s a problem when you instrument a function that must be able to deal with ::s/invalid as one of the arguments. This is important in tooling such as Cursive where ::s/invalid is remembered as REPL state for example
2019:01:18 17:27:38 alexmiller there are a very small number of such functions and some workarounds even for those cases
2019:01:18 17:27:55 borkdude yes, a workaround is writing your own ::any😉
2019:01:18 17:28:26 alexmiller I consider “do nothing” to be a leading contender alternative for that ticket
2019:01:18 17:29:16 borkdude how many votes are representative of “considered important enough by the community” in general for a CLJ Jira issue?
2019:01:18 17:30:01 alexmiller votes are about attention, not answers
2019:01:18 17:30:44 alexmiller if you’re not spec’ing core functions used by spec, how often is this an issue?
2019:01:18 17:30:52 alexmiller b/c that’s when I’ve seen people raise it
2019:01:18 17:31:23 borkdude probably tooling, but then it would only be an issue for the developer of that tooling
2019:01:18 17:32:03 alexmiller I don’t consider this a settled result (which is why the ticket is still open)
2019:01:18 17:32:10 alexmiller that’s just my current thinking on it
2019:01:18 17:34:07 alexmiller changing topics, I just merged a giant refactor in spec-alpha2 in case anyone is interested
2019:01:18 17:34:46 alexmiller although since it’s a refactoring rather than new stuff, it’s not particularly exciting
2019:01:18 17:35:00 borkdude that’s not only about self-spec’ing
2019:01:18 17:35:14 alexmiller agreed
2019:01:18 17:35:53 alexmiller I haven’t had time to look at it seriously but I am aware of it
2019:01:18 17:36:42 borkdude about spec-alpha2: cool! I’ll try it out soon
2019:01:18 17:37:06 alexmiller there are some subtly important differences in the api for callers
2019:01:18 17:39:12 alexmiller I will try to write about those in my journal today rather than spew them here
2019:01:18 19:19:28 tyler What approach do folks use to handle the namespacing of nested keys. We are trying to spec check the interface to an external service and the payload is a nested map.
Namespaced keywords would seem to imply having to create a namespace for each nesting which isn’t ideal.
Alternatively, we could generate the specs but this seems to have its own problems. As far as I can tell this would imply having to compose macros which seems excessive. Additionally, the caller side would have to construct the long form of these keywords in order to pull out the necessary data since there would be no namespaces to alias with this approach.
Is there another approach that I’m missing? Its also possible that spec isn’t appropriate for this use case.
2019:01:18 19:20:51 borkdude @tyler a similar question was asked not too long ago here
2019:01:18 19:22:49 tyler Is it in the recent history? I’m having trouble searching for it.
2019:01:18 19:24:12 taylor you don't have to create "real" namespaces for the keys, you can qualify keys with arbitrary prefixes e.g. :fake-ns/foo
2019:01:18 19:25:19 manutter51 We sometimes use "dotted-path" namespaces like :order.line-item/qty
2019:01:19 03:24:36 seancorfield @alexmiller Want me to go back to my spec2 branch and start testing our code against this again? Happy to do so starting Monday if you think this should have parity with current spec?
2019:01:19 04:27:13 alexmiller Sure, go for it
2019:01:19 04:30:23 seancorfield Thank you!
2019:01:19 03:24:57 seancorfield (originally posted in #announcements by accident!)
2019:01:19 03:28:12 madstap I see that s/keys is still part of the api in spec2. I'm a bit surprised given RH's talk on how he wants to separate that out into s/schema and s/select. Are these going to be added in addition to s/keys then or is s/keysgoing away in favor of them in the next iteration? If they'll all exist together, will it ever make sense to use s/keys over s/schema/`s/select`?
2019:01:19 03:28:35 seancorfield @madstap That's a bit unfair -- the refactor to spec2 started before Conj.
2019:01:19 03:28:56 seancorfield The shape/required stuff is further down the pike.
2019:01:19 03:29:28 seancorfield The work in spec2 is groundwork to prepare for that.
2019:01:19 03:30:29 madstap Sure, my question is more about the direction spec will take. Is s/keys "living on borrowed time" so to speak?
2019:01:19 03:33:36 seancorfield Yeah, I'd say s/keys is living on borrowed time. Spec is still alpha so it's subject to API breakage.
2019:01:19 03:34:21 seancorfield Given the namespace changes, you can have both libraries loaded and both types of specs defined while you transition.
2019:01:19 03:34:33 seancorfield That's the important part of namespace changes.
2019:01:19 03:37:27 madstap Yeah, just like RH talked about in his penultimate(?) keynote. Pretty cool to see how that approach works out in practice.
2019:01:19 03:40:52 seancorfield This is why next.jdbc will be a new namespace at least and a new artifact more likely 🙂
2019:01:19 04:23:15 alexmiller Unknown yet on s/keys. We haven’t started any of the work yet for the stuff rich talked about at conj
2019:01:19 04:25:55 alexmiller Look, people complain when we work in a private repo, then drop something at the end. We’re trying to do something different here which means you get to see all the steps along the way, so things will be changing over a probably months
2019:01:19 04:26:55 alexmiller Some of it will be experimental
2019:01:19 04:27:04 madstap I'm not complaining, I hope it didn't come across that way
2019:01:19 04:27:18 seancorfield I love that you're dropping stuff we can test against.
2019:01:19 04:27:41 alexmiller Just saying, don’t expect it all at once
2019:01:19 04:27:51 seancorfield I don't much care which way you're going with the API -- we'll follow, regardless, even if we have to keep changing code 🙂
2019:01:19 04:29:08 seancorfield (we're living on the bleeding edge for a reason -- it gives us an edge!)
2019:01:19 04:29:25 madstap And this iteration solves one of the big pain points which were programmable specs
2019:01:19 04:34:45 madstap One observation about that though: The way spec op macros are implemented now breaks clojure.walk/macroexpand-all, which will stack overflow if the form contains any spec macros.
2019:01:19 04:38:04 seancorfield That's good feedback for the development of spec2. Last time I tried our code against spec2, a lot of stuff broke. I'll be trying it again on Monday when I get back to work. Breakage is to be expected in prerelease work and if folks pitch in and try it, the core team get more feedback which is helpful!
2019:01:19 08:54:42 borkdude I have this spec in spec1:
Caused by: java.lang.IllegalArgumentException: no conversion to symbol
at clojure.core$symbol.invokeStatic(core.clj:596)
at clojure.core$symbol.invoke(core.clj:589)
at clojure.spec_alpha2$explicate_1$fn__904.invoke(spec_alpha2.clj:314)
I have to look into this more, I just briefly tried it
2019:01:19 14:19:39 alexmiller The key thing is that specs have to start off as forms, not function objects. So here the anonymous function is getting evaluated too early and I suspect you need to quote the (fn ... ). That may be fixable though in spec since I turned with-gen into a macro. I’ll take a look when I’m at a computer.
2019:01:19 20:08:46 borkdude thanks
2019:01:20 11:01:24 borkdude Quoting the fn didn’t help
(defn seqable-of
"every is not designed to deal with seqable?, this is a way around it"
[spec]
(s/with-gen (s/and seqable?
(s/or :empty empty?
:seq (s/and (s/conformer seq)
(s/every spec))))
#(s/gen (s/nilable (s/every spec :kind coll?)))))
(s/def ::seqable-of-map-entry (seqable-of ::map-entry))
with the message:
Caused by: java.lang.IllegalArgumentException: No method in multimethod 'create-spec' for dispatch value: speculative.specs/seqable-of
Here I would like to have to functions in the same namespace that return two different "partial views" of a resource along with other information in a map.
It seems that I can't due to the fact that two pieces of information are glued together: the unqualified key and the spec.
I feel like I miss kind of a scoped spec declaration or a function like this : (s/key :resource <spec>) that I can embed in the declaration: (s/keys :req-un [(s/key :resource <spec>) ::foo]).
But I feel also that I am maybe misusing the library.
Has someone bumped into this problem already?
Thanks
2019:01:19 19:58:37 misha @dam since you use :req-un in all 3 s/keys, you can pick another namespace for second ::resource:
2019:01:20 08:24:50 lambdam @misha thanks for this solution. I did that in a way for something a bit different and the problem I bumped into was that you cannot require the namespace containing the specs in another namespace and use it with a shortcut. Let's say the previous code was in a namespace a.
(ns b
(:require [myapp.lib.a :as a]))
(s/valid? ::a/foo <whatever>)
(s/valid? :quz/resource <whatever>) <-- I have to remember that this comes from namespace myapp.lib.a
and doing this
(ns b)
(s/valid? :quz/resource <whatever>)
the dependency graph of spec definitions is not right and the compiler will complain (it happened to me and I spent time renaming a lot of specs)
Thus, even though I can do it, I tend to feel that it's a clumsy solution.
2019:01:20 13:01:19 misha tying all specs' ns to file ns - is often not a good idea
2019:01:20 08:50:47 lambdam Another question that comes to my mind, how can I validate a map whose keys are string (typically data from an HTML form).
Doing this (s/def ::form-data (s/keys :req ["foo" "bar"])) is half of the spec. I'd like to qualify the content of the keys.
Again I feel like missing a syntax like this: (s/def ::form-data (s/keys :req [(s/key "foo" string?) (s/key "bar" int?)])).
2019:01:20 09:37:59 lmergen i keep running into an issue with managing the complexity of my specs, and was wondering whether i'm "doing it all wrong". specifically, of quite a few specs, i need to have multiple variations of the same "schema", depending on the context.
to give an example, a user should not need to have an ID before it is inserted into the database, but it does need an ID when it is.
this is a simple example, but this pattern repeats. since ideally, i want my code to be able to declare whether it expects a a "stored user" or not, i currently compose these things as follows:
on the surface this seemed like a good idea, but after a while of using this, i discovered downsides to this. most importantly: as soon as you have other specs that depend upon inserted users or not (e.g. a "company" vs "inserted company"), you repeat this pattern a lot, even when there is not necessarily a difference between a "company" and "inserted company".
as an alternative to this, i was thinking of making the state an explicit property i can multi-spec on. this makes the approach more data-driven. but then i wouldn't be able to directly say, "i expect an inserted user" in the code through specs anymore.
are there any alternatives to this ?
i hope my question is making a bit of sense 🙂
2019:01:20 16:52:10 ericnormand @lmergen I think Rich Hickey has noticed that problem, too
2019:01:20 16:52:29 ericnormand he talked about it at the conj and hinted at a big change to spec to make this easier
2019:01:20 16:52:52 lmergen yeah I saw that, sounds very promising!
2019:01:20 16:52:59 ericnormand basically, you define a map spec separately from the selection
2019:01:20 16:53:25 ericnormand which lets you separate out the definition of the entity from the context
2019:01:20 16:53:35 lmergen yes so you declare what you need rather than what something is
2019:01:20 16:54:03 ericnormand awesome
2019:01:20 16:54:05 lmergen ok thanks I kind of forgot about that
2019:01:20 16:54:18 ericnormand i have faced this same problem myself
2019:01:20 16:54:47 ericnormand proliferation of specs because, for instance, if you have all the CRUD operations, they each require slightly different data
2019:01:20 16:54:55 ericnormand though you'd like to think of them as operating on the same entity
2019:01:20 16:55:56 borkdude it’s the same problems that type systems without extensible records have
2019:01:20 16:56:50 lmergen yes
2019:01:20 16:57:06 lmergen so now spec becomes a category system :)
2019:01:20 16:57:31 borkdude what’s that
2019:01:20 16:57:53 lmergen sorry I meant category as in category theory
2019:01:20 16:58:05 lmergen rather than describing types you describe traits
2019:01:20 16:59:05 lmergen I am fairly sure I have just mentioned a few forbidden words
2019:01:20 16:59:26 borkdude — awkward silence —
2019:01:20 16:59:40 borkdude 😉
2019:01:20 16:59:46 lmergen haha
2019:01:20 17:03:06 alexmiller I think it’s most useful to think of it as oriented around value sets, as described by predicates
2019:01:20 17:04:24 lmergen I’ll be watching your progress alex, just found the new spec repo
2019:01:20 17:05:12 alexmiller maps are composites of their attributes and the key is that semantics are derived from the attributes, not from the composite itself
2019:01:20 17:05:34 lmergen so they become self describing ?
2019:01:20 17:07:38 alexmiller not sure that’s the important bit
2019:01:20 17:08:40 lmergen true
2019:01:20 17:10:00 alexmiller the new idea from Rich’s last keynote is that different functions can speak more precisely about which subset of the composite it cares about, and we can have better tools for describing what a function does with those inputs wrt to the output
2019:01:22 01:21:15 kenny Is there any way to get more info about why a spec generator is failing? This is the error message:
Nothing in the stacktrace points to my code. I guess I could go through and individually test all my specs for which failed but in this case the spec is huge and that would be very time consuming.
2019:01:22 01:22:23 gfredericks @kenny not an answer to your direct question, but that particular failure is usually caused by an s/and, where one of the non-initial predicates is not likely to happen by chance
2019:01:22 01:22:36 gfredericks (or all of them together, more accurately)
2019:01:22 01:22:39 kenny Right. There's a lot of those 😞
2019:01:22 01:24:47 gfredericks it's a long outstanding issue between spec and test.check; test.check's alphas have a feature in such-that that would let spec report what spec is causing it
2019:01:22 01:24:59 gfredericks but spec doesn't use that feature yet afaik
2019:01:22 01:27:26 seancorfield It doesn't help you now @kenny but when I'm writing specs, I pretty much always have a (comment ..) form with calls to s/exercise for each of my specs, and I make sure they generate as I'm writing them... for exactly this reason.
2019:01:22 01:31:57 kenny I have some decent guesses for the ones that fail but the feature @gfredericks mentioned would significantly improve this experience. Then you could get rid of all those comment blocks @seancorfield😉
2019:01:22 01:32:44 kenny Any idea if there's a jira issue for this?
2019:01:22 01:35:08 seancorfield I have a lot of Rich Comment Forms in my code 🙂
2019:01:22 01:36:13 seancorfield I find they're a good way to keep notes about my thinking as I develop, as well as containing expressions that produce test data and show usage of functions -- and specs 🙂
2019:01:22 13:12:10 lmergen what would be the recommend way to do a multi-spec dispatch for some nested map variable ?
e.g. let's say that i have this
{:entity/type :user
:entity/data
{:user/id 1234}}
and i want to multi-spec the :entity/data based on the :entity/type ... this seems pretty tricky ?
2019:01:22 13:13:41 lmergen because this is not possible, although it's something that i would want:
2019:01:22 13:45:36 jeroenvandijk If you control the design of the data, but i guess you dont, there is probably a better way more in line with clojure.spec
2019:01:22 14:31:26 lmergen 🤔 not sure whether this is brilliant or evil
2019:01:22 14:31:33 lmergen it works though
2019:01:22 14:34:08 lmergen how would you redesign the data ? lift the data inside :entity/data into the parent map so that multispec works ?
2019:01:22 14:34:08 lmergen how would you redesign the data ? lift the data inside :entity/data into the parent map so that multispec works ?
2019:01:22 21:39:14 jeroenvandijk yeah sorry for giving the evil hack 🙂 Changing the data will make it much cleaner. And using multispec for example gives you very nice specific validation errors e.g. when using it with expound
2019:01:22 22:54:00 lmergen yes thanks a lot, your comment was helpful!
2019:01:22 23:23:47 jeroenvandijk btw, not using a namespaced keyword, so :data instead of :entity/data could also be used as a fix instead of lifting the data inside. You would use s/keys with :req-un instead of :req
2019:01:22 14:55:01 lmergen actually that makes a lot of sense
2019:01:22 19:48:07 misha qualified keywords allow you to have a mix of "unrelated" keys in a flat map
2019:01:22 20:26:31 lmergen yes, it is obvious that I was doing this wrong
2019:01:22 20:26:44 lmergen makes a lot of sense
2019:01:23 12:02:20 victorb Hm, I'm a bit confused about the s/or. Trying to get spec to accept that a key can be either a-map or b-map, both of them are fine, so far got this:
2019:01:23 12:04:19 victorb @jeroenvandijk ooh, so simple. Thanks a lot for the fast help! 👍
2019:01:23 12:05:04 victorb ugh, should have been obvious if I just read the docs slower, missed the key+pred pairs. I'll blame it on too little coffee
2019:01:23 12:08:25 jeroenvandijk I had the same once, no worries
2019:01:23 12:09:14 jeroenvandijk It doesn't blow up either so hard to know you did something wrong (without coffee 😉 )
2019:01:24 06:26:56 dacopare I'm trying to write a spec for a map that satisfies two keys specs.
But generation obviously fails. Is there a way to do this that produces a good generator?
Perhaps using the and section in s/keys?
(s/keys :req [::x ::y (or ::secret (and ::user ::pwd))] :opt [::z])
2019:01:24 06:27:39 seancorfield s/merge
2019:01:24 06:27:56 seancorfield It’s designed to merge multiple s/keys specs.
2019:01:24 06:28:36 dacopare Thank you very much @seancorfield, it seems to be the perfect tool for the job.
2019:01:24 12:55:00 victorb How can I spec a map where the keywords can be anything (basically IDs in this case) but I do know what I want the values to be
2019:01:24 13:04:21 borkdude @victorbjelkholm429 what about (s/map-of any? #{:value1 :value2})
2019:01:24 13:06:26 victorb Thanks @aisamu and @borkdude. map-of any? seems to be what I was looking for.
2019:01:24 13:08:04 victorb Ended up with (s/map-of keyword? :my/other-spec)🙂
2019:01:24 13:15:44 fmjrey Better to change your spec before coding than changing it after coding!
2019:01:24 14:38:43 victorb @fmjrey funny you say that as I currently sit and trying to retrofit clojure.spec after getting the basic mvp features in place 🙂
2019:01:25 13:55:35 Ben Hammond I've just started playing with .
I can see that the :args are getting verified, but I cannot see how to get the :ret verified
2019:01:25 13:58:52 Ben Hammond Hmm I was expecting the return value to fail.
I can see the change in the doc string
(doc digits)
-------------------------
dev/digits
([just-an-int])
Takes just an int and returns the set of its digit characters.
Spec
args: (cat :just-an-int int?)
ret: (coll-of keyword? :min-count 1 :kind set?)
=> nil
2019:01:25 13:59:08 Ben Hammond what do I need to do to get the :ret spec verified?
2019:01:25 13:59:11 borkdude @ben.hammond ret is not checked during instrumentation
2019:01:25 13:59:20 borkdude it is during generative testing
2019:01:25 13:59:35 borkdude this is FAQ number 1 I guess
2019:01:25 14:00:11 Ben Hammond is that a feature?
or a bug?
2019:01:25 14:00:25 borkdude design decision
2019:01:25 14:01:04 Ben Hammond okay, so I have to manually call (spec/conform
that's not a biggie
Thanks
2019:01:25 14:01:54 borkdude There’s also orchestra which turns on ret checking in instrumentation
2019:01:25 14:02:51 Ben Hammond Well I was just trying to pace out my basic understanding
don't need to get too fancy at this point
2019:01:25 14:02:57 Ben Hammond thanks
2019:01:25 14:05:50 vemv is there a good predicate for checking if x can be used as a spec?
should be true for int?, (spec/spec ...), #{1 2 3}
2019:01:25 14:09:07 Ben Hammond
(doc spec/def)
-------------------------
clojure.spec.alpha/def
([k spec-form])
Macro
Given a namespace-qualified keyword or resolvable symbol k, and a
spec, spec-name, predicate or regex-op makes an entry in the
registry mapping k to the spec. Use nil to remove an entry in
the registry for k.
=> nil
2019:01:25 14:09:28 Ben Hammond any predicate should be fine
2019:01:25 14:09:51 Ben Hammond (which will include int? and hashsets)
2019:01:25 14:10:24 Ben Hammond or the keyword identifier of an existing spec
2019:01:25 14:10:43 Ben Hammond or an inline spec declaration
2019:01:25 14:12:02 Ben Hammond by 'regex-op` I think they mean combinations of (spec/+, (spec/cat, ,(spec/alt` etc..?
2019:01:25 14:13:26 Ben Hammond how do you tell if an arbritary function is a predicate? I'm not sure if you can distinguish function arity from the outside
2019:01:25 14:13:38 Ben Hammond other than by provoking ArityExceptions
2019:01:25 14:15:05 Ben Hammond I guess you could get java.lang.reflect to tell you
and there is probably a clojurey wrapping somewhere
2019:01:25 14:15:49 vemv not sure if we're talking about the same thing
2019:01:25 14:16:11 vemv (defn spec? "whether x can can be used as a spec" [x] ...)
2019:01:25 14:16:25 vemv ^ I'm seeking this
2019:01:25 14:16:41 borkdude @vemv you want a predicate that tells you if a thing fits inside (s/fdef x --> ? <--) ?
2019:01:25 14:17:24 borkdude spec already has a spec? predicate, but that doesn’t do the same
2019:01:25 14:17:36 vemv yeah, not s/spec?
2019:01:25 14:18:10 danielneal ifn?
2019:01:25 14:18:10 vemv seeking a filler for (s/def ::foo ->>> x <<<-)
2019:01:25 14:18:17 borkdude I guess fn? and keyword? and spec? maybe?
2019:01:26 17:29:00 borkdude I don’t understand why, because this has never been the case before with CLJ. With CLJS these are pretty common, because it has no direct linking.
2019:01:26 18:46:50 borkdude The cause is probably that spec-checking-fn calls with-instrument-disabled, which calls binding which expands in a call to hash-map, so there’s a loop.
2019:01:26 19:04:48 borkdude A possible solution would be to replace in the binding macro:
2019:01:28 09:58:30 borkdude I’ll probably give up on it for now. CLJS has plenty of similar issues like this (instrumenting apply, seq, etc.)
2019:01:28 09:59:44 borkdude I’m not sure what your solution accomplishes, but you could add it to the issue with some context
2019:01:28 10:01:53 jeroenvandijk From what I understand, hash-map calls hash-map itself when instrumented. So I thought of aliasing hash-map to the old version of hash-map during the validation itself. Just an idea, maybe it's shortsighted
2019:01:28 10:03:14 borkdude right. the spec-checking-fn replacing hash-map calls itself (because binding emits a call to hash-map), so it ends up in a loop.
2019:01:28 10:19:41 jeroenvandijk Maybe a complete clojure.core fix would be to have (stest/instrument x) to have x to be aliased to it's previous value so it cannot interfere with the instrumentation itself when x is used for the instrumentation.
2019:01:28 10:20:28 borkdude that would be the more fundamental solution yes
2019:01:28 10:21:10 borkdude it would also speed up core instrumentation probably
2019:01:28 17:35:56 butterguns Hi. I'm experimenting with Spec. I think I'm approaching it (incorrectly) from an OO background. Suppose I have a spec ::vehicle #{:car :truck :semi-truck :bicycle :motorbike}. It is part of a map (s/keys :req [::vehicle ::color]).
At some time later in processing, I have a function that requires 2-wheeled vehicles, so I might have ::two-wheeled-vehicle (s/and ::car has-two-wheels?).
My problem is I've already used the ::vehicle key for the more "generic" specification of a vehicle. I would need to create a new map with new keys (`(s/keys :req [::two-wheeled-vehicle ::color])`), and copy all the values over, in order to validate the stricter specification. Am I doing it wrong?
2019:01:29 15:44:31 jkrasnay I’m trying to use s/keys with a vector of keys, such that I can use the same vector of keys elsewhere:
2019:01:30 08:41:20 Josip Gracin Hi! clojure.spec.test.alpha/check silently ignores symbols which are not "checkable". This seems counter-intuitive to me because I'd expect it to fail when I call check on a symbol and the symbol does not have an associated spec (e.g. it is misspelled).
2019:01:30 08:41:47 Josip Gracin the definition of check looks like this:
2019:01:30 08:42:18 Josip Gracin it seems to me that the (filter (checkable-syms opts)) might better go to the no-arg variant
2019:01:30 08:42:45 Josip Gracin does this make sense?
2019:01:30 13:10:04 alexmiller File a jira
2019:01:30 13:11:45 Josip Gracin ok, tnx!
2019:01:30 15:15:41 mping Hi, is there any good way to xform spec’s :problems into something that could make sense for a human over a json api?
2019:01:30 16:07:15 bbrinck I suppose it depends on how technical the user is, but you could potentially use Expound here.
2019:01:30 16:10:24 bbrinck The default options might be too noisy (they include the specs), but this can be changed easily. You can also customize the messages in some cases. Let me know if you want help with configuring expound or just need some sample code for your use case.
2019:01:30 16:15:14 bbrinck Phrase is another option
2019:01:30 16:36:59 mping tks, I basically want to convert the spec’s :problems into something more tractable for a human
2019:01:30 16:37:43 mping it seems that i’d have to change *explain-out* right?
2019:01:30 16:38:23 bbrinck Yep, that’s what Expound does :)
2019:01:30 16:39:02 bbrinck You can just replace explain-out with the printer provided by Expound
2019:01:31 18:58:08 taylor s/gen takes an overrides map
2019:01:31 18:58:31 borkdude @mattmorten stest/check also takes an overrides map in case you need that
2019:01:31 18:58:33 taylor something like (s/gen ::person {::zip new-spec}) I think?
2019:01:31 19:00:22 butterguns That was even simpler than I hoped for. Thanks!
2019:01:31 20:45:05 butterguns I have a related question. I have a coll-of ::animal, where ::animal is a multi-spec. How do I provide overrides to force the ::animal that is generated?
2019:01:31 20:45:09 butterguns
(s/def ::animal-type #{:dog :cat})
(s/def ::cat-attribute #{"very catty"})
(s/def ::dog-attribute #{"dog dog"})
(defmulti animal-type ::animal-type)
(defmethod animal-type :cat [_] (s/keys :req [::cat-attribute ::animal-type]))
(defmethod animal-type :dog [_] (s/keys :req [::dog-attribute ::animal-type]))
(s/def ::animal (s/multi-spec animal-type ::animal-type))
(s/def ::animals (s/coll-of ::animal))
(s/def ::petshop (s/keys :req [::animals]))
(gen/sample (s/gen ::petshop) 1)
=>(#:{:animals [#:{:cat-attribute "very catty", :animal-type :cat}
#:{:dog-attribute "dog dog", :animal-type :dog}]})
;; How do I override s/gen to get a list of :cat's only?
(gen/sample (s/gen ::petshop {??}) 1)
2019:01:31 20:47:59 borkdude Narrator: and this was harder than he hoped for.
2019:01:31 20:49:50 borkdude so the spec for cat is (s/keys :req [::cat-attribute ::animal-type])?
2019:01:31 20:50:00 butterguns yep
2019:01:31 20:50:38 borkdude I don’t have experience with spec and multimethods, so I’ll shut up now
2019:01:31 20:51:47 alexmiller name the spec returned by the defmethod and override it?
2019:01:31 20:52:24 alexmiller or you’re looking for a particular branch of the multimethod to be poked?
2019:01:31 20:53:11 butterguns I just need cats!
2019:01:31 20:53:16 alexmiller override ::animal with the generator for the spec returned by (animal-type {::animal-type :cat}) ?
2019:01:31 20:53:42 borkdude is multi-spec relevant here?
2019:02:01 11:12:20 borkdude Is there something like s/undef in spec? I guess not?
2019:02:01 11:15:21 eval2020 IIRC there’s a ticket for that
2019:02:01 11:17:07 borkdude What I actually want is to define a an fdef that doesn’t get instrumented when calling (stest/instrument), but this may be too niche.
2019:02:01 11:20:11 borkdude this is already in spec
2019:02:01 11:21:45 borkdude but this doesn’t work for fdef?
2019:02:01 11:22:26 borkdude (s/def clojure.core/hash-map nil) works
2019:02:01 11:23:51 borkdude seems to work also in CLJS
2019:02:01 11:23:54 eval2020 would expect fdef’s to be put in the registry
2019:02:01 11:25:19 borkdude cool, I might be able to use this, thanks @eval2020!
2019:02:02 01:12:55 drone running into issues with circular references between specs (::a references ::b in it’s definition and vice versa). taking a step back to consider alternative ways to model the specs, but will spec2 support referring to undefined specs? As in:
(s/def ::a ::b)
(s/def ::b <spec>)
2019:02:02 03:34:25 alexmiller Yes, and I have done some more work there to defer lookups more pervasively
2019:02:02 02:38:14 kenny I'd be nice to use s/assert in production but if you s/assert a spec that contains a fspec, it will require test.check to check the value. Is is possible to disable this functionality in production but still use the s/assert check?
2019:02:02 03:29:55 seancorfield How would it check the spec then @kenny?
2019:02:02 03:30:44 seancorfield (and you've got the same problem if you use s/valid? or s/conform, right?)
2019:02:02 03:34:24 kenny fn? is all I need in prod.
2019:02:02 03:34:50 kenny It’d be nice to get Spec error messages without the overhead of generation.
2019:02:02 03:39:19 seancorfield If you want checks in production, put them in the code explicitly with s/valid? and/or s/conform.
2019:02:02 03:40:35 seancorfield Asserts in production are a code smell, IMO, since they throw an Error, not an Exception so you're basically going to "kill" your request/process.
2019:02:02 03:45:04 seancorfield If you want "Spec error messages" then you don't really want s/assert -- you want explicit code to check validity and call explain or something.
2019:02:04 15:12:34 daniel-tcgplayer Hey everyone, I'm experiencing something odd trying to implement spec into our project for the first time. I'm using fdef to define my function's spec, however it's not validating my arguments. I'm using clojure 1.10. Here's the code:
Calling this function (inc-num "a") throws a ClassCastException instead of the spec error. Any ideas?
2019:02:04 15:14:58 taylor did you s/instrument the function too?
2019:02:04 15:15:10 taylor that's what enables the :args checking at runtime
2019:02:04 15:16:05 daniel-tcgplayer I did not! Let me go ahead and do this, maybe I got ahead of myself in the documentation 🙂
2019:02:04 15:16:55 borkdude yes, writing an fdef does not turn on instrumentation automatically. also note when you re-define the fdef, you have to call instrument again to make it effective
2019:02:04 15:18:58 daniel-tcgplayer I see. So for all functions that I'm relaying on fdef to spec, I must also instrument those functions
2019:02:04 15:19:11 borkdude yes
2019:02:04 15:19:24 borkdude you can instrument all function at once by calling (stest/instrument)
2019:02:04 15:19:39 daniel-tcgplayer Per namespace?
2019:02:04 15:19:49 borkdude if you’re using component, you might want to hook it up to the start/stop cycle. no, globally
2019:02:04 15:20:41 taylor yep that'll instrument everything that's been loaded, across all namespaces. there's also unstrument to do the opposite
2019:02:04 15:21:33 daniel-tcgplayer Awesome! Thanks everyone for the quick feedback, I'm movin' again. I'll go through the documentation more thoroughly
2019:02:04 15:25:16 borkdude I’ve never used fspec. does spec really check the arity of such a function when you call a higher order function whose function argument is spec’ed with it?
2019:02:04 15:27:05 taylor @U04V15CAJ fdef uses fspec internally, so there's not much difference AFAIK. when you spec a function that takes another function as an argument, spec invokes the passed function ~20 times — a kind of mini-check — to see if it conforms
2019:02:04 15:27:35 borkdude spec invokes then function when?
2019:02:04 15:27:49 taylor whenever you call the higher-order function, if it's instrumented
2019:02:04 15:28:12 borkdude it doesn’t just call it when it’s called normally?
2019:02:04 15:29:00 taylor not sure I understand the question
2019:02:04 15:31:07 taylor so whenever you call f (instrumented), spec is going to do a lil mini-check of the function you pass to f to see if it conforms to the fspec, and if it does then of course f will call g again
2019:02:04 15:31:47 borkdude why is that sane behavior? I can see this being useful when doing generative testing
2019:02:04 15:34:12 taylor I think you need test.check on your classpath
2019:02:04 15:34:55 borkdude yeah, I know why this happens, I’m just surprised that it suddenly wants to do generative testing while it can just conform while running, like normal functions
2019:02:04 15:35:20 taylor there's no way to check an fspec without invoking the function though
2019:02:04 15:36:00 borkdude oh, now I see, because they are not replaced by an instrumented version… right.
2019:02:04 15:36:14 borkdude but still, it’s a little weird this.
2019:02:04 15:37:01 borkdude you would have to be very wary of using side effects in fspec’ed functions
2019:02:04 15:38:05 borkdude I haven’t used fspecs in speculative yet and this may be a reason I’m not going to..
2019:02:04 15:42:20 taylor I think the problem is more about speccing higher-order functions in general, than it is purely about fspec, b/c fdef is just a shorthand for s/def + s/fspec. another option is to just use fn? or ifn? when you're speccing HOFs, that way spec doesn't mini-check them
2019:02:04 15:42:39 borkdude yes, that’s what we’re doing now
2019:02:04 15:43:00 borkdude in most cases it’s sufficient, although it would maybe nice to be able to speak about function arity
2019:02:04 15:47:16 taylor one case where I can see value in fspecing args to HOFs is when you check the HOF, spec will pass it a dummy function that expects the right :args and returns values generated according to the :ret spec
2019:02:04 15:48:34 borkdude that would be useful, but I want this to be pluggable. so my fdef would not use fspecs, but I do want to plug them instead of ifn? during generative testing.
2019:02:04 15:49:18 borkdude you can do that by overriding spec generators
2019:02:04 15:50:14 borkdude but overriding ifn? would be a bit too global, since the args and ret could use ifn? with different properties
2019:02:04 15:50:29 borkdude so then you would have to give the argument list a named spec
2019:02:04 15:50:36 borkdude which I what I do sometimes
2019:02:05 14:55:09 drone chiming in that the generator requirement for HOFs also does not seem sane to me. should be able to instrument without assumption that you’re using generative testing
2019:02:05 14:55:44 borkdude I’d say so too. @alexmiller maybe should be an option to turn this off on spec2?
2019:02:05 15:02:58 alexmiller it’s a topic we’ll revisit
2019:02:04 20:16:32 hmaurer Is there a way quick way, without using an external lib, to generate a random value from a spec while limiting the max depth when the value is a map?
2019:02:04 20:17:00 hmaurer e.g. I have nested s/keys specs and I would like to generate a random value for the spec but limit the depth of the data structure
2019:02:04 20:17:20 hmaurer I know there are some libs on github that do this, but I am wondering if there is a short way to do it out of the box with specs
2019:02:04 20:18:28 taylor is it a recursive spec? if so, there's a *recursion-limit* binding you can use during generation. the other thing that comes to mind is passing in a small/fixed size arg to the sample function
2019:02:04 20:21:28 hmaurer @taylor yes it’s recursive, although not directly. The spec is modelling an AST
but I am getting “Couldn’t satisfy such-that predicate after 100 tries.”
2019:02:04 20:23:58 hmaurer any idea how I can make it work?
2019:02:04 20:25:03 taylor yeah that's b/c the generator for your s/and is only based on the first inner s/keys spec, and the generated values from that don't conform to the second multi-spec
2019:02:04 20:25:53 hmaurer @taylor ah. Any way around this?
2019:02:04 20:25:57 taylor you can wrap that s/and with s/with-gen to specify a custom generator, or override the generator elsewhere
2019:02:04 20:26:17 taylor some of the spec functions take an overrides map for this
2019:02:04 21:01:07 hmaurer is there a way to define a spec on a string without seq’ing it first?
2019:02:04 21:01:08 hmaurer a regex spec
2019:02:04 21:04:26 alexmiller no
2019:02:04 21:08:32 hmaurer @alexmiller is it a generally bad idea to use specs to parse strings?
2019:02:04 21:08:48 alexmiller it’s not recommended, but I literally can’t stop you :)
2019:02:04 21:08:48 alexmiller it’s not recommended, but I literally can’t stop you :)
2019:02:04 21:09:10 hmaurer 😄 what is the recommendation against it?
2019:02:04 21:31:29 souenzzo Do your own spec-inspired (and maybe backended by spec) library to specify string?!?!
2019:02:04 21:33:13 borkdude I recently found this lib: https://github.com/miner/strgen
It’s cool, but maybe comes with limitations
2019:02:04 21:33:27 hmaurer @U2J4FRT2T not a bad idea 😄 I would need to first learn more about how spec works though
2019:02:04 21:33:56 borkdude The nice thing about using spec for strings is that you get nice error messages and can generate examples.
For better error messages one could also use a parser combinator or parser generator.
2019:02:04 21:35:10 hmaurer @U04V15CAJ being able to generate examples is definitly useful
2019:02:04 21:38:31 alexmiller sorry, was on a call
2019:02:04 21:38:55 alexmiller the point is that spec regex were designed to spec sequential collections
2019:02:04 21:39:28 alexmiller strings are not sequential collections and the performance of the regex specs on them might be bad
2019:02:04 21:39:59 alexmiller and there are quite good options for parsing strings (namely, the regex built into the JDK and exposed by Clojure)
2019:02:04 22:32:28 seancorfield FWIW, we use test.chucks string regex generator quite a bit -- we're very happy with that, combined with regular string regex. I would not recommend trying to use spec's sequence regexes on a string.
2019:02:04 22:53:02 borkdude @U04V70XH6 good to hear. how does it compare to miner’s lib? I see his name in the README of test.chuck. Was his lib merged into test.chuck?
2019:02:04 22:53:40 borkdude @U052852ES I think this thread is worth reading since you asked me about a related problem you’re solving with spec
2019:02:04 23:12:52 souenzzo We can compile specs into regexp at macro time 👀
2019:02:05 00:35:49 seancorfield @U04V15CAJ No idea. First I've heard of Steve Miner's regex lib.
2019:02:05 07:55:14 thomas @U04V15CAJ I have seen it and I will have a look again, thank you.
2019:02:05 08:27:41 borkdude I meant the test.chuck regex thing
2019:02:05 09:25:39 Vincent Cantin Where do clojurists usually place their fdef?
- In the same namespace of their functions?
- In the same namespace with “.spec” prepended?
- In a separate, global /src/spec/ namespace tree?
- other?
2019:02:05 11:02:04 n2o I usually put it directly below the function definition, but i am not really happy with it to be honest.
2019:02:05 11:16:23 Vincent Cantin Why aren’t you fully satisfied with it?
2019:02:05 12:04:22 n2o the source code feels a bit messy. Simple clean functions, then a new definition below it just for specs... Just a personal impression. Some kind of choppy in my opinion
2019:02:05 12:05:06 n2o I tried to put them into a separate namespace but as you have to instrument them to really use the specs, the specs are like to be forgotten. So this does not seem to be a good solution
2019:02:05 12:05:48 n2o I then tried to put them below my source code, but in the same namespace (at the bottom of the file). I almost forgot all of them and was wondering, that I wrote specs for those functions when I scrolled down the file after some months. So this is also not a good solution
2019:02:05 12:07:23 n2o Currently I put them directly together, so I don't forget them
2019:02:05 17:47:11 seancorfield If I want the code to be usable without spec (i.e., on Clojure 1.8), I put fdefs in a separate namespace (see clojure.java.jdbc for example). Otherwise I put the fdef immediately above the function it refers to.
2019:02:06 09:10:53 n2o Ok, in my case I do not need to be downwards-compatible. But yes, then you don't have a different choice.
2019:02:05 09:34:11 borkdude @vincent.cantin The answer is: up to you. There is no “best way” regarding this. Personally, I co-locate it with the defn.
2019:02:05 13:53:04 abdullahibra Hello guys
2019:02:05 13:53:19 abdullahibra how could i check the values of map only using clojure.spec, i know how can i do this for map keys names + values, but i'm in situation which i don't know what key names are but i know what types of values should be ?
2019:02:05 13:53:52 borkdude e.g. for checking string values: (s/map-of any? string?)
2019:02:05 13:53:56 abdullahibra should handle the values using s/coll-of
2019:02:05 13:54:03 abdullahibra ah great
2019:02:05 13:54:15 abdullahibra @borkdude thanks
2019:02:05 13:56:25 abdullahibra is clojure.spec good alternative to type system in language like Haskell, ocaml ?
2019:02:05 20:25:27 hmaurer most definitely not
2019:02:05 20:25:29 hmaurer 😄
2019:02:05 20:25:48 hmaurer it’s very different and by no mean a substitute for static typing (as much as I like spec)
2019:02:05 13:57:48 borkdude it has different applications, everything’s runtime, so you get no compile time guarantees, macros being the exception
2019:02:05 13:58:07 borkdude but it’s also more flexible than a type system. different trade offs.
2019:02:05 14:06:38 manutter51 Also spec is still evolving, there are going to be changes, especially concerning how maps are spec’ed.
2019:02:05 14:22:08 borkdude would it make sense to co-develop the CLJS version of spec2 in the same repo and pull it outside CLJS itself, so CLJS never is behind on spec?
2019:02:05 14:22:21 borkdude like test.check
2019:02:05 14:26:12 borkdude am I right that you can’t use anonymous functions in specs anymore in spec2 so they would always have to be defined top-level? what about with-gen?
2019:02:05 14:26:48 borkdude thanks for the link @victorbjelkholm429, useful information
2019:02:05 14:29:22 alexmiller I do not think you should be tracking spec 2 changes in cljs yet - still in flux
2019:02:05 14:30:07 alexmiller you can use anonymous functions
2019:02:05 14:30:27 alexmiller what about with-gen?
2019:02:05 14:31:26 borkdude can?
2019:02:05 14:31:52 alexmiller ?
2019:02:05 14:32:24 borkdude I’m reading about “symbolic specs”, it seems that you cannot use anonymous functions in there?
2019:02:05 14:32:30 alexmiller you can
2019:02:05 14:32:50 borkdude
Symbolic specs consist only of:
Spec forms (lists/seqs), with spec op in function position, composed of other symbolic specs
Qualified keywords (names that can be looked up in the registry)
Qualified symbols (predicate function references)
Sets of constant values (an enumeration)
2019:02:05 14:33:06 alexmiller spec op can be fn
2019:02:05 14:33:35 alexmiller so those are valid spec forms
2019:02:05 14:33:52 borkdude ah ok. I was confused, because of feedback you had earlier, I thought it wasn’t allowed anymore, but then I misunderstood
2019:02:05 19:54:04 taylor I really like the music notes at the end of each post! There's some good lesser-known stuff on Zeppelin's In Through the Out Door too 🙂 are you a fan of The Meters?
2019:02:05 19:54:57 alexmiller yep and yep
2019:02:05 19:56:35 alexmiller been listening to the first Meters album a lot lately actually :)
2019:02:05 14:34:56 alexmiller it has a note about fn in the middle there
2019:02:05 14:35:41 alexmiller if you want to use anonymous functions at the top level with the api functions, you need to wrap them in s/spec
2019:02:05 14:37:13 borkdude alright. I don’t know how much in flux things are, but if you want me to give it a try again with speculative, let me know.
2019:02:05 14:39:28 alexmiller I’ve got at least one thing to fix still from Sean’s feedback
2019:02:05 14:39:52 alexmiller and we’re working through best direction for some of the things you saw last time still
2019:02:05 14:40:14 alexmiller so probably not worth doing anything yet
2019:02:05 23:05:43 drone maybe using spec in an unintended way, but I ran into a situation where there were two spec forms in an s/and, the result of the first conformed the input and the second spec form in the s/and was a predicate expecting the original input, not the modified/conformed form. it feels like validation should be independent of conforming
2019:02:05 23:06:56 drone and I don’t understand the purpose of conforming. is it intended to be like schema’s coercion? or is it to produce some intermediate form useful for spec-checking?
2019:02:05 23:46:44 alexmiller Re and, we have long considered having both “flowing” and nonflowing variants of and
2019:02:05 23:46:54 alexmiller And that might still happen in spec 2
2019:02:05 23:47:41 alexmiller The purpose of confirming is not coercion - it’s to tell you which choices were made during validation for optional specs or components
2019:02:05 23:48:10 alexmiller That is, why was it valid?
2019:02:05 23:58:14 drone Sounds good. The flowing was definitely surprising.
2019:02:05 23:58:28 drone Thanks, got it
2019:02:05 23:58:39 drone A lot of design decisions
2019:02:06 03:22:20 flyboarder So this is a spec error I ran into today and Im here thinking there must be a better way to get to the root of the problem
2019:02:06 03:22:36 flyboarder
2019:02:06 03:34:50 taylor what’s on line 28 in bootstrap/feature.clj? is that your project?
2019:02:06 03:37:09 taylor it’s probably the destructuring of the first arg, maybe a keyword is being passed as the first arg
2019:02:06 03:37:25 flyboarder ^ I was able to solve this the problem was
(spec/explain-data ::feature-gate feature) when I needed
(spec/explain-data ::feature-gate [feature])
2019:02:06 20:27:36 borkdude @alexmiller do you plan to support this for spec-alpha2 as well? https://dev.clojure.org/jira/browse/CLJ-2060
I noticed it does not yet work in CLJS.
2019:02:06 20:29:53 alexmiller should work, haven’t looked at why yet, but wasn’t intentionally broken
2019:02:06 21:19:30 seancorfield FYI, I just ran out entire test suite against the latest spec2 and there's just one failure (for s/valid? on a large data structure for a very complex, nested spec) -- which was the unexplained failure I had before. The generator problems I mentioned before are fixed by the latest work.
2019:02:06 21:28:29 seancorfield @alexmiller What in spec (or spec2) causes this predicate to be used? (or (nil? %) (sequential? %))
2019:02:06 21:29:57 seancorfield (it's not in our code so I assume it's what underlies s/cat or something?)
2019:02:06 21:48:05 alexmiller I think that’s checked for all regex specs
2019:02:06 21:48:30 alexmiller yeah
2019:02:06 21:48:54 alexmiller I assume you see that fail?
2019:02:06 21:51:38 seancorfield I've tracked it down a bit further...
(def s [ ... sequence of hash maps ...])
(count s) ;;=> 6
(s/explain ::ba-decline (take 3 s)) ;;=> Success!
(s/explain ::ba-decline (drop 3 s)) ;;=> Success!
(s/explain (s/cat :one ::ba-decline :two ::ba-decline) s)
;; fails, claiming that the first element of s does not satify: (or (nil? %) (sequential? %)) in: [0] at: [:one]
2019:02:06 21:52:18 seancorfield (this works with spec1)
2019:02:06 21:53:00 seancorfield Am I missing something obvious about combining regex specs?
2019:02:06 21:53:56 seancorfield The ::ba-decline spec is an s/cat of three items.
2019:02:06 21:54:58 seancorfield I'll see if I can create a minimal example that fails, based on this (unfortunately there are a lot of complex specs behind this).
2019:02:06 22:03:23 seancorfield @alexmiller
(s/def ::x string?)
(s/def ::y int?)
(s/def ::z keyword?)
(s/def ::a (s/keys :req-un [::x]))
(s/def ::b (s/keys :req-un [::y]))
(s/def ::c (s/keys :req-un [::z]))
(s/def ::abc (s/cat :a ::a :b ::b :c ::c))
(def a {:x "x"})
(def b {:y 42})
(def c {:z :bar})
(s/explain ::abc [a b c])
(s/explain (s/cat :one ::abc :two ::abc) [a b c a b c])
user=> (s/explain (s/cat :one ::abc :two ::abc) [[a b c] [a b c]])
Success!
certainly is suspicious :)
2019:02:06 23:08:17 seancorfield It's almost like regex specs don't unroll anymore! :rolling_on_the_floor_laughing:
2019:02:06 23:14:43 alexmiller I may have actually introduced that by introducing the delayed resolution of aliased specs
2019:02:06 23:17:07 alexmiller basically regex specs in the registry are shielded by the keyword (which is not a regex spec), so the code doesn’t believe they can be combined
2019:02:06 23:18:05 alexmiller so this may fight with the last changes - I can’t remember now if that was from forward references or from something about gens?
2019:02:06 23:18:55 seancorfield This was broken before you fixed either of those I believe. Certainly broken before the gensub fix.
2019:02:06 23:19:30 alexmiller well good to know then :)
2019:02:06 23:19:35 seancorfield Although, with a quick clj -Sdeps ... it's enough to test that repro case against any version of spec2 🙂
2019:02:06 22:16:42 alexmiller I’ll take a look, seems buggy to me
2019:02:06 22:17:10 alexmiller There is a lot gnarly code in that form/object transition. I could easily have broken something subtle
2019:02:06 22:17:33 alexmiller Thanks for the repro
2019:02:06 22:18:52 seancorfield I was lucky that the first simple repro I tried still broke 🙂
2019:02:06 22:19:13 alexmiller Hey, that’s a good sign
2019:02:06 22:21:02 seancorfield At this point, it really is a pretty minimal set of changes in our code to go from spec1 to spec2. It's frustrating that we can no longer use comp and partial to construct predicates and have to resort to #(..) or (fn [x] ..) but, fortunately, that didn't affect much code.
2019:02:06 22:23:05 seancorfield We have a few places we're having to use s/spec* now so the "helper macros" mentioned in your latest journal will help clean that up when they drop in GitHub.
2019:02:06 22:54:10 alexmiller yeah, that has further solidified today and I think will be very useful
2019:02:06 22:31:58 borkdude I still have at least 3 bugs with spec alpha 2 (see issue link above)
2019:02:06 22:36:15 seancorfield Interesting. I never knew you could define a spec to nil to make it go away...
2019:02:06 22:37:05 borkdude It’s a fairly recent addition.
2019:02:06 22:52:32 borkdude Make that 2. The (s/def spec nil) was a mistake on my part, I had to call unstrument first (which failed because ::s/invalid is not a valid any? which I ran into)
2019:02:06 22:52:42 alexmiller oh good :)
2019:02:06 22:52:43 borkdude undefining still works:
2019:02:07 12:30:23 borkdude I thought I had to make upgrading existing spec libraries less painful is to have something like a reader conditional that can dispatch on the version of spec.
E.g.:
2019:02:07 16:23:26 dominicm doesn't work for cljs though
2019:02:07 16:23:34 borkdude that’s the big deal, CLJS
2019:02:07 16:24:11 borkdude we could use a goog-define for it in CLJS, but then still you cannot do anything on the namespace declaration level
2019:02:07 16:25:21 dominicm it's a shame that it doesn't work in cljs given the work that was done to make require work there.
2019:02:07 16:25:43 borkdude that’s only for self-hosted AFAIK
2019:02:07 16:25:48 borkdude and REPL usage
2019:02:07 16:26:49 borkdude anyway, it would be nice to have a way have libraries support both versions. I bet it will be 2 branches and versions with “-spec2” suffixes for a while
2019:02:07 16:27:55 dominicm yeah, I think you're right
2019:02:07 16:28:31 dominicm you could use a macro, and perform the require voodoo at the clojure level perhaps 🤔
2019:02:07 16:28:36 dominicm but still not great
2019:02:07 16:29:04 borkdude oh right, I think I saw someone “abuse” a data reader for that recently. Maybe it was @U050PJ2EU?
2019:02:07 16:29:04 dominicm it would work in shadow-cljs, which has the ability to specify custom :keywords which run
2019:02:07 13:49:16 borkdude will spec-alpha2 be released at the same time as clojure 1.11 as one bundle?
2019:02:07 13:54:35 alexmiller unknown
2019:02:07 13:59:36 borkdude I have yet to test the latest SHA. Will update you, probably in the weekend.
2019:02:07 14:05:03 alexmiller no worries
2019:02:07 14:05:08 alexmiller good catches
2019:02:07 18:55:04 joefromct can anyone point out something i may have missed… if i try to generate from a s/keys spec un-req i get all blank maps… but if i use the same with req i can generate data. Do you need a custom generator for anything un-req ?
2019:02:07 18:56:31 alexmiller do you mean req-un ?
2019:02:07 19:09:16 alexmiller s/keys is a valid spec that will validate all registered keys in the map
2019:02:07 19:09:25 joefromct yeah i guess i did come across in the guide the mention of (s/keys) was valid and i guess the rule to fn’s are “extra keys no-big-deal”
2019:02:07 19:09:30 alexmiller but of course it doesn’t know how to gen
2019:02:07 19:10:16 alexmiller just use this mnemonic … “I req-un the map should have a key like this”
2019:02:07 19:59:40 jimbob ….nice
2019:02:07 20:00:57 jimbob whats the idiomatic way to mandate presence of keys?
related: whats also the idiomatic way to mandate the lack of extraneous / undefined keys?
2019:02:07 20:01:40 jimbob i suppose for spec you can do something like (s/and (contains % #{mandated-keys})
2019:02:07 20:02:03 borkdude mandate the presence of keys: req-un
not allow extra keys: this is not according to the philosophy of spec/Clojure I think
2019:02:07 20:02:42 jimbob interesting. thanks
2019:02:07 20:03:25 taylor there are some code snippets around for disallowing "extra" keys though @ben.borders even though it's counter to the design philosophy
2019:02:07 20:03:48 borkdude you can always hack around the design philosophy of Clojure 😛
2019:02:07 20:04:10 jimbob right, i figured.. seems like i should probably try to allign more with the philosophy
2019:02:07 20:04:21 jimbob i’m sure there are valid reasons for that.
2019:02:07 20:04:27 jimbob would just like to read about it i suppose
2019:02:07 20:04:31 alexmiller Rich did a whole talk about it
2019:02:07 20:06:06 alexmiller grep for “code for growth”
2019:02:07 20:06:20 jimbob thank you
2019:02:07 20:06:45 taylor I think one of the metaphors is "if you're expecting a truck to deliver your TV, you shouldn't care what else is on the truck" or something like that
2019:02:07 20:06:49 borkdude There have been some conversations about what spec is and isn’t suited for. E.g. it isn’t designed for coercion or “closed world” assumptions (not allowing extra keys), e.g. using it as API boundary protection
2019:02:07 20:07:27 alexmiller I think the greater point is actually that you shouldn’t design your systems that way, and if you don’t, spec will mesh well :)
2019:02:07 20:08:08 jimbob right.. we don’t now.. we allow our maps to have whatever values they have, and we grab the ones we care about
2019:02:07 20:08:17 jimbob which tends to be most or all of them.
2019:02:07 20:08:34 borkdude well, sometimes I don’t want to have crap in my jsonb field, but there’s always select-keys, etc.
2019:02:07 20:55:41 seancorfield > I think the greater point is actually that you shouldn’t design your systems that way, and if you don’t, spec will mesh well :)
It's fine for a (REST) API to be passed a bunch of parameters it doesn't care about. It can just ignore them. Then spec works well for parameter validation.
2019:02:10 13:14:10 Jp Soares I started to use REST API with compojure-api 2.0.0 that can coerce data with spec and generate swagger docs from it. I thought it would be the trend for specs, but reading this conversation maybe not.
It's better to use compojure-api with schema to follow the clojure philosophy in REST?
2019:02:10 18:24:29 seancorfield I don't understand your question : what do you mean "clojure philosophy in REST"?
2019:02:07 20:57:41 seancorfield > well, sometimes I don’t want to have crap in my jsonb field, but there’s always select-keys, etc.
And you can pull the "known" keys out of a spec fairly easily (usually!) so that you can narrow your set of fields down to just what's in the spec (we do this around database insertion where we have a spec for the row representation of data).
2019:02:07 21:00:11 joefromct how would a spec look for the output (https://github.com/clojure/clojure/blob/master/src/clj/clojure/xml.clj#L78) from clojure.xml\parse look?
Below is what i’m guessing at so far, but the ::content references ::element and vice versa… not sure how to handle this. (I’m not worried about namespaces at the moment):
I think its tricky for me because it’s a hierarchy.
2019:02:07 23:45:51 butterguns ...only gives me alphanumeric, for 10,000 iterations. Why is that?
2019:02:07 23:46:04 butterguns No symbols / unicode
2019:02:07 23:47:27 seancorfield That's probably all the string? generator is coded to produce.
2019:02:07 23:48:03 seancorfield test.chuck has a regex string generator that can produce a much wider range of character values, if you need to test that.
2019:02:07 23:49:22 seancorfield (my editor does not render them all correctly 😞)
2019:02:07 23:49:56 butterguns ... scrolling down to "Lets set up some Generators", he uses gen/string instead of string? and it produces the full range
2019:02:07 23:50:06 seancorfield Ah, there you go!
2019:02:07 23:50:20 butterguns Would be nice not to have to override all my string? specs with gen/string however 🙂
2019:02:07 23:52:18 seancorfield I guess it depends on how important it is that you exercise your code with more exotic strings?
2019:02:07 23:53:25 butterguns Yeah, I guess so. It just surprised me. Testing against exotic strings strikes me as exactly within generative testing's wheelhouse
2019:02:07 23:54:38 seancorfield Well... it is but this is a case where you need to opt into it.
2019:02:07 23:57:23 seancorfield It really depends on what parts of your data processing you need to stress test. If you only pass a string through your system and into the DB, you don't really care whether it's alphanumeric or all poop emojis.
2019:02:07 23:58:06 seancorfield If you have specific string processing that does some sort of parsing, it might well be useful to stress test it with really exotic strings.
2019:02:07 23:58:36 seancorfield Not everything is important to test to the same level.
2019:02:07 23:59:36 seancorfield It might be more useful, for example, to use a string generator that only ever produces an empty string, a short readable string, or a giant long string.
2019:02:08 00:01:19 butterguns That's a good point. I think this perhaps exposes a weakness in my understanding of generative testing. My brain says "what will this really test that can't be better accomplished with targeted unit testing"
2019:02:08 00:01:37 butterguns wacky strings is often my go-to answer
2019:02:08 00:02:03 butterguns But this is a larger question, somewhat off topic
2019:02:08 00:04:57 seancorfield My experience so far leads me to think that the power of generative testing is more in the combinations of data it uses to exercise code rather than the specific values of an individual item.
2019:02:08 00:08:12 butterguns Can you elaborate? To that point: I feel like that is what makes gen testing so difficult. E.g. I may have a "Command" that has two parameters. But only a very narrow combination of those two parameters actually make sense. Otherwise the "Command" is just gibberish.
2019:02:08 00:09:40 butterguns And when I start using (gen/elements ["very" "small" "number" "of" "valid" "inputs"]) it sorta doesn't feel like generative testing anymore
2019:02:08 00:15:02 seancorfield If you have interdependent parameters, that's certainly a bit trickier to test. You'd need to write a spec for the possible combinations that were valid, and make sure it's generatable.
2019:02:08 00:17:47 seancorfield An example from one of our apps. We have a spec for the set/sequence of valid interactions a user (an admin in this case) can have with a sequence of data. Given that (complex!) spec, we can generate arbitrary but valid combinations of operations the user could run. We use those generated combinations of operations to drive an HtmlUnit test to verify that a) they are actually possible and don't produce errors and b) that the end state for the transformed sequence of data is still valid (using its own set of specs).
2019:02:08 00:18:35 seancorfield Automated UAT of a web app, via generative testing 🙂
2019:02:08 00:23:07 butterguns Ah, ok this makes a lot of sense! I think I'm getting hung up on the idea that I have to check the result of applying these operations, in literal sense. Like, "after a generated operation, I should inspect the operation and foo should == bar". It sounds like you're saying "after a generated operation, make sure everything conforms to spec, plus no errors thrown, and then just move on"
2019:02:08 00:36:49 seancorfield It's really going to depend on your specific system under test. In another part of our system, we have certain pairs of functions that are inverses of each other, so for those we'll do full-on property-based testing to ensure that, for arbitrary "valid input", calling (= (inverse-fn (some-fn some-input)) some-input).
2019:02:08 00:38:19 seancorfield And then in other situations, you might want to specifically test handling of bad data so you might write a spec for some of the things that your input cannot be, and then run tests to ensure that for arbitrary "bad input" you get some sort of appropriate failure response (i.e., that given bad input you don't get a success response).
2019:02:08 00:38:57 seancorfield (that might help you detect combinations of bad input that you unexpectedly allow through your app -- a gap in your validation)
2019:02:08 01:27:19 ericnormand john hughes (creator of quickcheck and now property-based testing consultant) always talks about how prop-based testing is exploratory
2019:02:08 01:27:41 ericnormand you model the system you are checking, and often the tests don’t pass
2019:02:08 01:28:02 ericnormand you investigate and find that your test was missing something
2019:02:08 01:28:13 ericnormand so you modify it and run it again
2019:02:08 01:28:28 ericnormand it fails again, it was missing something else
2019:02:08 01:29:19 ericnormand my point is that property based testing always requires work to model the system
2019:02:08 01:29:37 ericnormand there is no “one way” to test
2019:02:08 01:30:50 ericnormand i think property based testing is a good name. you’re looking for algebraic properties to test for
2019:02:08 01:31:15 ericnormand @seancorfield’s example of inverses is a good property
2019:02:08 01:31:31 ericnormand another is idempotence
2019:02:08 01:31:48 seancorfield Yup, you often start with a property you believe should hold and sometimes you discover it doesn't -- so either the property is wrong or your system doesn't preserve when it should 🙂
2019:02:08 01:32:04 ericnormand commutativity, associativity, identity, zero
2019:02:08 01:32:17 ericnormand all good properties to start with
2019:02:08 01:32:35 ericnormand then you can get creative
2019:02:08 01:32:55 ericnormand as in “it’s commutative under these conditions”
2019:02:08 01:33:17 ericnormand or it’s commutative under this comparison operator
2019:02:08 01:33:29 ericnormand (instead of =)
2019:02:08 01:35:19 ericnormand as for generators, i have made use of three
2019:02:08 01:35:36 seancorfield This output value should always exceed this input value. The output should always be an ordered sequence (regardless of its actual contents). The output value(s) should always be between these bounds (possibly based on input values). Etc.
2019:02:08 01:35:49 ericnormand always valid data, junk data, and almost valid data
2019:02:08 01:35:58 ericnormand the almost valid is the hardest to model
2019:02:08 01:36:50 ericnormand for me at least
2019:02:08 01:37:14 ericnormand it’s almost as if you need to start with valid data and break it
2019:02:08 01:37:21 ericnormand in a random way
2019:02:08 01:37:57 ericnormand those are to test error conditions as sean was saying
2019:02:08 01:38:45 ericnormand does such a thing exist in clojure?
2019:02:08 01:39:30 seancorfield In theory, you could build such a system (to mutate code). Easier to mutate input data and see if your system breaks in unexpected ways 🙂
2019:02:08 01:39:40 ericnormand yes
2019:02:08 01:40:15 ericnormand if d doesn’t pass spec, f should throw an exception
2019:02:08 01:40:41 ericnormand it’s hard to test that thoroughly
2019:02:08 01:41:23 ericnormand to feel confident you’ve found all of the ways you could do the data wrong
2019:02:08 01:41:54 ericnormand it would be interesting to make a generator that could mutate any other value randomly
2019:02:08 01:42:08 ericnormand add or subtract 1 from numbers
2019:02:08 01:42:21 ericnormand add a char to a string
2019:02:08 01:42:37 ericnormand drop an element from a list
2019:02:08 15:28:06 alexmiller I fixed it, then broke it again :) I’ve reverted my last changes
2019:02:08 15:38:29 borkdude it works now, except for some error messages that look strange (see issue). I think we’ve now seen the first time that all speculative specs can be instrumented. 🙂
2019:02:08 15:41:19 borkdude I have to port respeced to spec-alpha2 before I can run the test suite, I’ll do that sometime soon
2019:02:08 15:46:55 alexmiller I’m not getting what’s weird about the errors
2019:02:08 15:50:11 borkdude so it’s clear which of my own specs the argument is violating
2019:02:08 15:50:25 alexmiller so this bit: spec: :speculative.specs/map
2019:02:08 15:50:31 borkdude I guess so.
2019:02:08 15:51:27 borkdude Maybe it would also be good for expound to test this behavior, since it relies on things like this (@bbrinck)
2019:02:08 15:52:07 alexmiller yeah, I’m sure there are some subtle things like this that have broken, particularly in regex world
2019:02:08 15:52:54 borkdude I don’t have specific demands for this, but expound probably has. Now’s the chance to get it fixed 🙂
2019:02:08 15:54:02 borkdude Are spec objects that are defined in the registry backward compatible with spec1? I don’t mean how you are defining them, but the actual result of defining a spec?
2019:02:08 15:54:19 alexmiller no
2019:02:08 15:54:21 bbrinck @borkdude Good idea! I will start a spec2 branch soon for expound to see what works and what needs changes.
2019:02:08 15:54:45 alexmiller for the moment they are the same protocol, but that protocol is in a different ns so not compatible
2019:02:08 15:55:24 alexmiller migration/release is another whole thing - we’ve talked about it at length but not going to worry about it for now till we’re much closer
2019:02:08 15:55:30 borkdude @alexmiller if that were true, I think there would be a more smooth upgrade path. spec2 would just register it in both the old and new registry and old consumers would still work
2019:02:08 15:55:38 borkdude ah ok, makes sense.
2019:02:08 15:56:08 bbrinck I’m wondering if expound may need to depend on spec1 and spec2 at the same time. Even if expound has different namespaces that use spec1 and spec2, presumably a lib could upgrade to spec2 but still use a lib that uses spec1.
2019:02:08 15:57:04 alexmiller that should be fine (presuming you have a version of clojure aware of both registries, etc)
2019:02:08 15:57:56 alexmiller the clojure integration aspects will require a clojure release - lots of open questions about exactly what that will cover still tbd
2019:02:08 15:58:09 bbrinck Yeah
2019:02:08 15:59:30 borkdude I’ve thought about supporting two versions of the lib, one with a -spec2 suffix in the version. Could work.
2019:02:08 16:00:27 bbrinck That makes sense, as you think about migration/release it would be useful for lib authors to come up with some practices as well, informed by your decisions. I suspect many libs will be in the situation where they want to be spec2 compat but must assume that some deps use spec1.
2019:02:08 16:00:29 borkdude but then again, there’s only one or two places right now where I needed to change something? the finicky difference is the namespace declarations
2019:02:08 16:01:13 borkdude anyway, now it probably not the time
> but not going to worry about it for now till we’re much closer
🙂
2019:02:08 16:01:38 bbrinck If all spec-libs require both spec1 and spec2, then at least the incremental cost of using any spec-lib won’t be that much :)
2019:02:08 16:04:32 borkdude And then we have CLJS in the mix as well, so if you write CLJC there is that to consider too
2019:02:08 18:59:37 gklijs Is there a cljs way to statisfy inst? I used it in a cljc spec and ran into trouble reading the java Instant, then just switched to number, but there might be a nicer way?
2019:02:08 19:14:35 lilactown are you asking how to make a value that is an inst? AFAIK it's just a (js/Date.)
2019:02:08 19:32:24 gklijs I should explain in more detail. I have a list of images in de back-end (Clojure) with just an id of the image and the time it was uploaded for now. I want to spec and use this list in the front-end (Clojurescript). I already have some (de)serialize code based on the spec to not repeat the keys everywhere. So I think I could get away with transforming the Clojure Inst to a string in such a way it's easy for Clojurescript to make it into a js/Date. Which gives me my answer by using a DateTimeFormatter.ISO_INSTANT to format the Instant to string, it should be possible to use Instant.parse(x) to get back the Instant while at the same time in Clojurescript I can use #inst x to get the js/Date.
2019:02:08 19:36:20 lilactown if I understand correctly, what you are trying to figure out is how to transfer the instant over the wire?
2019:02:08 19:36:25 lilactown from the back-end to the front-end?
2019:02:08 19:37:41 lilactown FWIW, you can't call #inst x where x is a var or some other binding; #inst is a literal, so you'd have to write the date literally #inst "1980-01-01"
2019:02:08 19:38:36 lilactown what I would suggest is using something like EDN or Transit to communicate between your back end and front-end, so that they can transfer the inst literally without having to format to a string and then re-parse the string
2019:02:08 19:57:15 gklijs I already use EDN, but that goes wrong, because there is no Instant object in cljs. You where right about the literal, I need (js/Date. x) to create it.
2019:02:08 19:57:33 lilactown are you reading the EDN in?
2019:02:08 19:57:49 gklijs yes
2019:02:08 19:58:06 lilactown you might just need to provide the correct arguments to the reader
2019:02:08 20:03:43 gklijs Thanks, might work better. I actually use reader/read-string now..
2019:02:08 19:40:42 ballpark Hey everyone... question about using s/merge with s/multi-spec: https://stackoverflow.com/q/54599149/59439
I put it on SO for easier findability
2019:02:08 22:12:35 borkdude @alexmiller I now ported my testing library to spec-alpha2, this works.
Now I’m running the tests of speculative and I ran into a bug which I can reproduce as follows:
user=> (require '[clojure.spec-alpha2.gen :as gen])
nil
user=> (gen/sample (s/gen any?))
Execution error (IllegalArgumentException) at clojure.spec-alpha2.protocols/eval189$fn$G (protocols.clj:11).
No implementation of method: :gen* of protocol: #'clojure.spec-alpha2.protocols/Spec found for class: clojure.core$any_QMARK_
2019:02:08 22:13:27 borkdude I can probably workaround this by using the test.check generator directly, but just fyi.
2019:02:08 22:36:28 borkdude This does work: (gen/sample (gen/gen-for-pred string?))
should work since it’s a qualified symbol referencing a predicate function?
2019:02:08 23:20:33 borkdude It might as well be an accidental breaking change. I’ll wonder what Alex thinks. I’m off to bed now.
2019:02:08 23:41:29 seancorfield Given that (s/valid? any? :foo) etc all reject any? (and string? etc), I think it's pretty clear it's deliberate @borkdude
2019:02:08 23:42:59 seancorfield any? isn't a spec, it's a predicate. You can define a spec in terms of a predicate. So (s/def ::foo any?) is valid and then (s/exercise ::foo) will be valid but (s/exercise any?) is not.
2019:02:08 23:43:39 seancorfield And then (s/spec any?) produces a spec from a predicate. So (s/exercise (s/spec any?)) is valid.
2019:02:08 23:44:33 seancorfield The line between specs and predicates is very blurred in spec1.
2019:02:09 00:43:05 alexmiller Sean is correct. s/gen takes a spec object. A predicate symbol or function is not a spec object. Invoking s/spec with a predicate will expand into a qualified symbolic spec and pass it through spec*, yielding a spec object
2019:02:09 00:43:54 alexmiller So (s/spec any?) or you can do the work of the spec expander and do (s/spec* `any?)
2019:02:09 02:07:29 butterguns Quick question: any way to supply a seed to gen/sample?
2019:02:09 02:09:08 butterguns I want to do some performance testing / benchmarking. Hence getting a predictable sample is critical
2019:02:09 02:28:28 seancorfield @mattmorten It sounds like you really need to provide your own overridden generators that produce the same sequence over and over again?
2019:02:09 02:36:10 butterguns Hmmm. That would involve a lot of duplication. In my test ns, I have 20+ lines of generator code to produce a complex data-structure in my app. I'm very happy with it. I'd rather say "give me seed 1 of this generator" than go back and write duplicate generators for each of my (many, very nested) attributes that produce a constant value
2019:02:09 05:11:04 flyboarder Why doesn’t spec/assert let me set my own message, seems like an optional 3rd arg would be really convenient
2019:02:09 07:33:27 borkdude @seancorfield @alexmiller alright, thanks. Now that this is clear I’ll change those
2019:02:09 10:00:37 borkdude without s/with-gen it works, with it doesn’t
2019:02:11 05:07:59 alexmiller fixed in latest
2019:02:09 10:03:32 borkdude It also works when not using s/or + s/with-gen so it seems s/or specific
2019:02:09 10:24:09 borkdude I bumped into another one, but this might be related, so I’ll wait for this one to be resolved
2019:02:09 13:42:18 alexmiller You have vector in the spec, not vector?
2019:02:09 13:42:38 alexmiller Not sure that’s the cause but seems like a thing to fix first
2019:02:09 15:37:21 borkdude Still doesn’t work with vector?.
2019:02:10 11:24:10 borkdude This works in spec1 and 2: (gen/sample (s/gen (s/every number? :kind vector?)))
This works in spec1 but doesn’t work in spec2. Should it? (gen/sample (s/gen (s/every number? :kind coll?)))
2019:02:10 14:14:39 alexmiller I think so? Not sure why it wouldn’t
2019:02:10 16:22:08 borkdude Well it doesn’t yet. I posted it in the issue
2019:02:11 05:05:55 alexmiller I did find and fix a bug in the gen from a :kind, but this particular example should and does fail in both versions. coll? can generate any kind of collection, including maps, which will not pass with something like:
2019:02:11 05:06:03 alexmiller
user=> (gen/sample (s/gen (s/every number? :kind coll?)))
Error printing return value (IllegalArgumentException) at clojure.lang.RT/seqFrom (RT.java:553).
Don't know how to create ISeq from: java.lang.Double
2019:02:11 05:06:25 alexmiller that’s with spec 1, but I see same on spec 2 now. You might sometimes get lucky and get enough non-maps for this to work occasionally.
2019:02:10 11:39:14 borkdude Apart from this issue and the or-spec issue, the speculative tests pass with spec-alpha2!
2019:02:10 12:23:23 borkdude The specs are now backward compatible with spec1. So if spec2 and spec1 would have the same namespace names, it could just be a drop-in replacement.
2019:02:10 12:54:03 borkdude It just occurred to me that libs can maintain compatibility with spec1 and spec2 by doing this:
(:require [clojure.spec.alpha :as s1] [clojure.spec-alpha2 :as s2])
and then define specs for each version of spec:
(s1/def :foo number?) (s2/def :foo number?)
2019:02:10 13:20:01 borkdude @jpsoares106 FWIW yada is not moving to spec for this reason. Schema will remain to be used to API boundary validation/coercion.
2019:02:10 18:27:46 seancorfield @borkdude what is "this reason" in your comment above? We use spec extensively for "API boundary validation / coercion" and it works great.
2019:02:10 18:38:34 seancorfield That doesn't answer the question -- WHY?
2019:02:10 18:40:49 borkdude Malcolm referred to this post on SO where it says that clojure.spec is not indented for coercion:
https://stackoverflow.com/a/49056441/6264
I’m not sure what I said anymore, lost track.
2019:02:10 18:40:51 seancorfield (And, yes, I read a lot of the follow up discussion there -- none of it seems substantive. It's all opinion)
2019:02:10 18:41:32 borkdude This opinion comes from the clojure.core team. But if it works for you, nobody will stop you I guess 🙂
2019:02:10 18:42:16 seancorfield I think it's misrepresenting what Alex said.
2019:02:10 18:43:59 seancorfield And it specifically omits Alex's justification, which was that it forces all clients of the spec to use the coercion. Which in this case is EXACTLY what you want here.
2019:02:10 18:44:30 borkdude What do you mean by “it”, the SO article?
2019:02:10 18:44:56 seancorfield Dave's answer there yes
2019:02:10 18:49:19 borkdude I haven’t tried using spec in this way. What I want at the API level is not having junk to come into my system which makes it all the way into the database in some jsonb field. Also I want coercion at the API level (e.g. query params are always strings and some have to be numbers, array, etc.).
All I know is that yada does this for me using Schema and I haven’t tried to make this work with spec. The main author of yada doesn’t want to use spec for this anymore. That’s all I know 🙂.
2019:02:10 18:50:53 borkdude I wish there was a clear and more elaborate article on this subject so I had more clarity on this myself.
2019:02:10 18:53:54 seancorfield When I'm not just on my phone I'll write more about this. We were an early adopter of spec and use it very heavily in production for a lot of different things. I've probably spent more time discussing this aspect with Alex than anyone else outside of Cognitect 🙃
2019:02:10 18:54:27 borkdude Please do!
2019:02:10 18:57:45 borkdude > We do this for parameters in our REST API, for long, double, Boolean, date, etc – we have two specs for each: one that is a spec for the target type in the domain model (which in these cases is just defined as the appropriate built-in predicate), and one that is a spec for the API level (which accepts either the target type or a string that can be coerced to the target type). Then we use the appropriate spec at the appropriate “level” in our application.
How do you deal with “extra” data coming into your API? How do you deal with preventing junk going into your persistence layer?
2019:02:10 18:59:17 lilactown select-keys? seems like that’s something spec won’t save you from
2019:02:10 18:59:58 borkdude yeah, a nested select keys actually which is what Schema and other tools are good at
2019:02:10 19:01:41 borkdude e.g. EQL
2019:02:10 19:06:08 lilactown curious to know how sean handles that too. We’ve been using spec for our services, but just relying on whatever reitit’s integration with spec-tools does
2019:02:10 19:16:43 ikitommi spec-tools allows one to drop all extra keys while doing coercion.
2019:02:10 19:16:55 ikitommi it’s automatically on in reitit & compojure-api
2019:02:10 19:20:39 seancorfield Regarding that quote from me: we have different specs for different layers. We have a spec for the persistence layer and we have code around the persistence layer that derives the set of keys (i.e., the set of columns) from the specs and uses that to narrow the keys to just those the database will accept. I.e., we are open for extensions, as far as the maps are concerned, until we have to close the set for storing it in a system that isn't open 🙂
2019:02:10 19:24:03 seancorfield At the API boundary, we have specs that are specific to the REST parameters -- strings as inputs (of course) and validation that does the minimal coercion in order to validate those arguments.
2019:02:10 19:25:19 borkdude It would be useful to have a function that worked generically on a spec that selected just the data that the spec describes, without changing the spec. Is that public?
2019:02:10 19:27:36 seancorfield Given that a REST API requires coercion, you have a limited number of choices here. You must do some coercion somewhere. You do it before, during, or after the validation step. You can't validate without some coercion. There isn't much point in doing the coercion twice -- and you don't want different code inside validation and outside it. You have the choice of doing all that coercion upfront and then validating the result, or you can do it in place via spec as part of the validation. Anyone criticizing combining that in spec should also be criticizing Schema for the same thing, IMO.
2019:02:10 19:31:51 seancorfield The objections to doing coercion in a spec are based on a number of things. Alex has repeatedly pointed out that if you do coercion in a spec, you are forcing that coercion on all clients of that spec -- which is a concern if you're building reusable specs and I agree that you shouldn't do coercion in a general, reusable spec.
2019:02:10 19:33:10 borkdude To be clear, what’s the problem of forcing coercion on a user of the spec?
2019:02:10 19:33:49 ikitommi Alex talks about using conform to do coercion.
2019:02:10 19:33:56 ikitommi it’s always on.
2019:02:10 19:35:06 seancorfield Right. So you only want to use it in a context where you need that coercion in order to do the validation.
2019:02:10 19:40:25 seancorfield I'm with Alex that spec shouldn't be used for JSON transformation/parsing 🙂 Our REST API mostly has simple string input that gets coerced as part of the validation that those strings are valid numbers, dates, booleans -- but where we have JSON input, we use a JSON library to do that aspect of parsing/coercion.
2019:02:10 19:42:41 ikitommi sure, the json strings get parsed with Jsonista/Cheshire and then the values are coerced. I would argue that “dropping keys that are not part of the spec” happens in the coercion part. Or you do it manually, which is IMO not a good idea.
2019:02:10 19:43:15 seancorfield We disagree on that, but that's fine.
2019:02:10 19:43:57 ikitommi hopefully spec will add support for coercion. the current ways of doing it (conform or form walking) are kinda hacks and need to go.
2019:02:10 19:45:34 seancorfield I doubt it, given how much and how often Alex et al have argued against coercion in spec 🙂
2019:02:10 19:46:49 borkdude Clearly there’s a tension between some users of spec and the authors of spec on this subject (coercion/stripping). I believe a good article/guide on http://clojure.org about this topic and the recommended way to go about it would be cool.
2019:02:10 19:53:56 ikitommi I guess the upcoming select could be used to strip out data not defined in it?
2019:02:10 19:54:35 seancorfield No, it's for selecting which keys to check in a given context.
2019:02:10 19:56:08 seancorfield So that it can decomplect the overall shape of the data spec from the conformance needed in specific situations.
2019:02:10 20:56:09 ikitommi @borkdude you asked about a function just to strip out just keys, just remembered that there is a existing transformer doing only that. So, this works (but requires spec-tools):
2019:02:11 14:58:24 alexmiller you’re getting the more specific error messages from the definition of bounded-string
2019:02:11 14:58:33 mpenet you can also map the op to something useful in your context for other means I guess
2019:02:11 14:59:04 mpenet user/json-object
2019:02:11 14:59:06 mpenet etc
2019:02:11 14:59:07 alexmiller many of the spec ops have internal constraints that will generate explain problems
2019:02:11 14:59:18 alexmiller so this is the same as other spec ops
2019:02:11 14:59:43 alexmiller like regexes check whether the input is nil? or sequential?
2019:02:11 14:59:49 alexmiller keys checks for map?
2019:02:11 14:59:50 alexmiller etc
2019:02:11 15:00:59 alexmiller it may be that some further adjustments should be made in the explain generator (just deferring to the definition spec right now)
2019:02:11 15:02:28 alexmiller I didn’t highlight it, but inst-in, int-in, and double-in are all derived specs and I reimplemented all of those in terms of defop
2019:02:11 15:02:47 alexmiller they are effectively all parameterized compound specs
2019:02:11 15:02:50 borkdude This gives more or less the same result when it comes to error messages:
2019:02:11 15:14:39 borkdude @mpenet
> it’s basically a parameterized spec, not 2 different specs
Not sure what you mean by this. If you call (bounded-string 1 10) and (bounded-string 1 20) you will get two different spec objects
2019:02:11 15:15:41 mpenet yeah but the form share quite a bit, I guess the facts it's 2 distinct spec objects is an impl detail
2019:02:11 15:16:10 mpenet as I said I don't know more than what I read on the blog post + intuition of what's the final intent
2019:02:11 15:19:17 mpenet for lib authors it's nicer to read (s/sorted-coll-of x comparator) than (s/and (s/coll-of x) #(..))
2019:02:11 15:20:28 borkdude when would a lib author read this? not trying to argue for the sake of arguing, just want to get it clear for myself 🙂
2019:02:11 15:21:28 borkdude I haven’t used s/form and friends myself yet, so I haven’t needed this feature much (probably out of ignorance)
2019:02:11 15:21:30 mpenet ex: spec-tools when trying to understand specs to build json schemas
2019:02:11 15:22:17 mpenet here we have specs that are used to validate json params also, one of which is some kind of derivative of coll-of, it makes dealing with it much nicer too
2019:02:11 15:22:52 borkdude but compared to a macro you don’t see a benefit there yet? I mean when reading and writing the literal code?
2019:02:11 15:23:05 mpenet not sure what you mean
2019:02:11 15:23:40 mpenet more readable s/form is a plus for me, ability to build your own s/coll-of like ops is good too
2019:02:11 15:24:05 borkdude I mean, when you define a macro you get to write same code:
2019:02:11 15:24:17 borkdude when are you using s/form?
2019:02:11 15:24:29 borkdude maybe this is used by explain logic?
2019:02:11 15:26:03 mpenet anytime you want to understand what's behind a spec
2019:02:11 15:26:05 mpenet programatically
2019:02:11 15:26:12 borkdude > ability to build your own s/coll-of like ops is good too
this is limited with s/defop since you cannot pass in any spec argument. e.g. (my-coll-of (s/nilable ::my-spec)) won’t work?
2019:02:11 15:26:32 mpenet no idea
2019:02:11 15:26:38 mpenet as I said I read only the blog post
2019:02:11 15:27:55 borkdude that’s what left me wondering about defmacro vs s/defop
2019:02:11 15:29:51 borkdude maybe not a huge limitation since you can always def the “anonymous” spec first
2019:02:11 15:44:24 alexmiller int-in, inst-in, double-in are all good examples where this is useful too
2019:02:11 15:44:38 alexmiller they are all compound parameterized specs
2019:02:11 15:45:40 borkdude > where this is useful
What exactly are you referring to?
2019:02:11 15:46:07 borkdude the result of s/form?
2019:02:11 15:47:11 alexmiller I just mean general cases where s/defop is better
2019:02:11 15:47:53 alexmiller symbolic specs form a language, defined by the ops
2019:02:11 15:48:31 alexmiller using a macro that expands to a compound spec is fine - you’re using an initial step to produce something in the language
2019:02:11 15:49:21 alexmiller using defop lets you create new ops in the language for the special case where the op can be defined in terms of other spec ops (and is not parameterized by another spec)
2019:02:11 15:50:39 alexmiller and if you need that, you can drop down another level and do the same thing the provided ops are doing - implement the create-spec multimethod to return a Spec protocol instance
2019:02:11 15:50:54 alexmiller (but a lot more code is required)
2019:02:11 15:51:01 borkdude is nilable an op? then why can (s/nilable ::foo) not be passed as an argument to defop, but ::foo can?
sorry, bit confused about “symbolic specs”
2019:02:11 15:51:14 alexmiller nilable is an op
2019:02:11 15:51:59 alexmiller as I said above, defop is not designed to be parameterized by other symbolic specs
2019:02:11 15:52:14 borkdude so passing ::foo just accidentally works?
2019:02:11 15:52:31 alexmiller what’s the example?
2019:02:11 15:52:49 borkdude (seqable-of ::foo) vs. (seqable-of (s/nilable ::foo))
2019:02:11 15:54:07 alexmiller it really goes to the evaluation model. spec names (fq keywords) are a little special in that they eval to themselves and also they are the only thing other than a spec object explicitly handled in the spec api functions.
2019:02:11 15:56:03 alexmiller in this case, ::foo evaluates to a valid symbolic spec (where another spec form evaluates to a spec object, which is not a symbolic spec)
2019:02:11 15:57:22 alexmiller so it will work, but you’ve created a narrow constraint on how it can be used
2019:02:11 15:57:54 borkdude right, so it’s something that happens to work, but not really the common use case for defop
2019:02:11 15:58:02 alexmiller yeah, I hadn’t really thought about that
2019:02:11 15:58:47 alexmiller you’re also not going to get proper explain path-ing with it as a spec created by defop is considered to be a single op
2019:02:11 15:59:04 alexmiller so if the sub-spec fails, you won’t get the parent spec in the path
2019:02:11 16:01:39 borkdude ok. to conclude: defop is not designed to support spec arguments. If you want that, either write a macro and accept less helpful error messages and s/form output, or “drop down another level and do the same thing the provided ops are doing - implement the create-spec multimethod to return a Spec protocol instance” which requires more code
2019:02:11 16:03:11 alexmiller yes
2019:02:11 16:03:33 alexmiller although I think in the macro case, the errors are almost exactly the same
2019:02:11 16:03:47 alexmiller so I would maybe quibble with that part
2019:02:11 16:08:35 borkdude not going to publish this anywhere, so I think for now it’s clear 😉
2019:02:11 16:15:03 borkdude I notice that regular pre-defined predicates are also not supported in defop:
2019:02:11 16:31:23 alexmiller the definition in defop is not going to evaluated - it should be a valid symbolic spec but where the parameterized values are substituted (defop is literally going through and replacing the args with their values)
2019:02:11 16:47:20 borkdude Made a typo. This works:
2019:02:11 16:53:22 seancorfield For specs parameterized by other specs, you can do something like
(defn domain-keywords
"Given a spec, return a new spec that can conform a keyword or string to
a keyword that is constrained by the given spec."
[spec]
(s/spec* `(s/with-gen (s/and ::keyword ~spec)
(fn [] (g/fmap name (s/gen ~spec))))))
(that and bounded-string above come from the World Singles Networks' codebase)
2019:02:11 16:54:49 borkdude @seancorfield what benefit does that have over writing domain-keywords as a macro?
2019:02:11 16:56:15 alexmiller well, it’s a function so you can apply it, so usual benefits of function over macro
2019:02:11 16:57:23 alexmiller (with the caveat that the spec arg needs to be a form, not an object, here)
2019:02:11 16:57:44 seancorfield It was a function in the spec1 world (without (s/spec* ..) and the quote/unquote, so we made it a function in the spec2 world. Minimal change (and it still composes and applies etc).
2019:02:11 16:58:38 borkdude @seancorfield I had a similar thing with seqable-of. Function in spec1, but when I turned it into a function in spec2 using s/spec*, I could not provide specs like (s/nilable ::foo) because I got an error, so then I made it a macro.
2019:02:11 16:58:39 seancorfield I plan on writing up a (probably long) blog post on our conversion from spec1 to spec2 when Alex tells me spec2 is stable enough for that to be widely valuable 🙂
2019:02:11 16:58:48 alexmiller you can’t use this with in s/def though (like (s/def ::x (domain-keywords ...)))
2019:02:11 16:59:05 seancorfield Right. And we use s/defop for those sorts of things.
2019:02:11 16:59:06 alexmiller but you could with the functional entry point s/register which takes an object (which is what domain-keywords returns)
2019:02:11 17:00:11 alexmiller @seancorfield btw, I spent some time working on the regex thing and I need to undo the changes I made to support forward references in regexes
2019:02:11 17:00:25 alexmiller which will fix the nesting issue, but re-break forward references
2019:02:11 17:00:42 seancorfield @borkdude Mostly, we've found changing our defn spec builders over to s/defop has been all we've needed in the most common cases. A few have needed s/spec* instead.
2019:02:11 17:00:46 borkdude > Right. And we use s/defop for those sorts of things.
Sorry, referring to what?
2019:02:11 17:01:30 seancorfield @alexmiller That's fine -- the forward reference issue only affected one spec in our entire code base so I can just move it below the sub-specs 🙂
2019:02:11 17:01:31 alexmiller Forward refs in regex is solvable via different means but I need to talk to Rich before I commit to a path on that and he’s out today
2019:02:11 17:01:55 seancorfield @borkdude "those sorts of things" = "use this with in s/def"
2019:02:11 17:02:09 borkdude a’ight
2019:02:11 17:04:21 borkdude I can’t write a very long blogpost about transitioning to spec2. All I had to do is report bugs, which were all fixed, wrap a bunch of predicates in s/spec, refactor one predicate to #(not (sequential %)) instead of (complement sequential?) and turn a private function into a macro.
2019:02:11 17:05:40 borkdude I think I might write a tweet about it instead.
2019:02:11 17:06:07 alexmiller :)
2019:02:11 17:06:30 alexmiller @seancorfield reverted the fwd reference fix, which should fix the nesting issue (but break that fwd reference)
2019:02:11 17:08:51 borkdude did forward referencing ever work? didn’t know
2019:02:11 17:10:33 alexmiller yes, in many cases
2019:02:11 17:11:10 alexmiller in general, specs delay lookup of named specs until use
2019:02:11 17:11:23 alexmiller I fixed several places where that wasn’t being done
2019:02:11 17:13:05 alexmiller but changes in how regexes are implemented mean that we effectively broke it for them
2019:02:11 17:14:54 alexmiller regex impls used to not be specs but would get spec-ized when needed. in spec 2, regexes actually are spec instances (thanks to metadata protocol implementation!) which simplifies the code in many places, but removed the point where this delay naturally happened before
2019:02:11 17:15:18 alexmiller fixing it is … tedious
2019:02:11 17:15:38 borkdude (s/declare ::foo)😉
2019:02:11 17:15:51 alexmiller yeah, no thanks :)
2019:02:11 18:08:35 seancorfield @alexmiller Good to know. I'll run a full test suite with the latest spec2 shortly.
2019:02:11 19:47:46 seancorfield @alexmiller Confirming that our full test suite runs on the latest spec2, with that one forward reference regex spec moved after the specs that it refers to.
2019:02:11 19:49:35 alexmiller 👍
2019:02:11 19:49:40 borkdude speculative still works as well
2019:02:12 13:56:36 guy If you’re going to be using spec in a project is it better to use spec-2 now instead?
2019:02:12 14:07:16 alexmiller no
2019:02:12 14:07:29 alexmiller we have not yet done any releases of spec 2
2019:02:12 21:30:47 guy Thanks!
2019:02:12 17:05:14 seancorfield (if you're using CLI / deps.edn, you can at least start testing against spec2 to see what code changes you might need to make at some future point @guy)
2019:02:12 21:25:29 aisamu We usually create aliases (`alias`) and then use ::
2019:02:12 21:30:32 misha 1) assuming alias :datomic.client.protocol = pro: ::pro/response is fine, but ::pro.response/body just does not work, (as keyword ns is just a string with no "structure" inside, afaik). So you'd need to make an alias for :datomic.client.protocol.response as well. And if you don't have datomic.client.protocol.response and datomic.client.protocol namespaces as files, thats 2 lines for create-ns, and 2 for alias, all just to save 5 words :(
2) afaik, clojurescript does not have alias function (or I did not find it yesterday)
2019:02:13 06:07:04 caleb.macdonaldblack Is there a way to create an anonymous spec? So instead of defining it, I can just create one and pass it into spec functions?
2019:02:13 06:13:43 seancorfield @caleb.macdonaldblack I suspect the answer is "yes" but it's different between spec1 and spec2...
2019:02:13 06:14:17 caleb.macdonaldblack spec2 as in the next version of spec?
2019:02:13 06:14:23 seancorfield In spec1, you can mostly use a predicate interchangeably with a spec. In spec2, you can construct spec objects on the fly.
2019:02:13 06:14:50 caleb.macdonaldblack Well I'm now excited for spec2
2019:02:13 06:15:19 seancorfield Hahaha... well, there are no releases yet, but if you're using deps.edn, you can test against it.
2019:02:13 06:17:39 caleb.macdonaldblack I'm using leiningen. I may as well wait until an official release. I'm glad to see spec moving in that direction
2019:02:13 06:19:06 seancorfield Have you seen Rich's talk from Conj? (Maybe Not)
2019:02:13 06:19:20 seancorfield He talks about future direction for spec...
2019:02:13 06:23:51 seancorfield Specifically he talks about how s/keys complects the shape of data and the actual checks that are needed in any given context.
2019:02:13 06:51:19 alexmiller in both spec 1 and 2 you can create a spec object and pass it to any of the spec api functions
2019:02:13 06:51:41 alexmiller with the caveat that s/keys relies on having registered key specs to rely upon
2019:02:13 06:52:50 alexmiller there may be more support in spec 2 for map selection specs with anonymous key specs
2019:02:13 06:53:22 alexmiller I am working in that area right now
2019:02:13 07:17:34 caleb.macdonaldblack Ah okay i didnt know that
2019:02:13 14:25:37 rascio Hi here! I'm playing with clojure specs, I'm struggling in trying to use generators for a spec with regex:
(s/def ::entity (s/and string? #(re-matches #"DEST:\d+")))
Am I wrong in defining something? Or the regex can't be generated by the default lib?
2019:02:13 14:27:41 borkdude @manuelrascioni you’re missing a %:
2019:02:13 14:30:29 borkdude Your spec will generate, but the likelihood that it generates strings that will satisfy the predicate is extremely small. You probably want to provide a generator with the spec, using s/with-gen
2019:02:13 14:39:34 rascio @borkdude ah...thank you! It seems I still have to train my eye to check for this kind of mistakes...I will check the docs for the with-gen thank you for the hint!
2019:02:13 14:47:49 rascio great! thank you, it is very helpful!
2019:02:13 14:50:27 rascio just to check if I understood well, the fmap is used to "customize" the value generated by a generator, and return a generator, right?
2019:02:13 14:54:46 borkdude fmap returns a new generator which transforms the values generated by the mapped-over generator using a function
2019:02:13 15:26:23 borkdude I recently ran orchestra which tests ret-specs with speculative on a body of code. The only ret spec which wasn’t correct was for an fdef for which I had not used generative testing.
2019:02:13 15:28:54 borkdude it was e.g. re-find, re-matches, etc. for which I had not considered that it could also return nils, as in
(re-find #"(a)?(b)" "b")
["b" nil "b"]
I wonder how I could have written a generator for this.
2019:02:13 15:29:13 borkdude I would have to generate regexes and strings that would sometimes match, sometimes not
2019:02:13 16:13:24 borkdude This fun experiment is able to generate regexes that seem to not terminate when executed…
2019:02:13 20:25:47 seancorfield As written, your spec accepts (only) numbers -- and it is generating numbers that your spec accepts. That's the correct behavior.
2019:02:13 20:27:14 seancorfield Conforming numbers to strings as part of a spec feels very wrong to me (and you know what an advocate I am for certain types of coercion in specs! 🙂)
2019:02:13 20:28:23 borkdude this was just an example, not something I’m doing for real
2019:02:13 20:29:36 borkdude the thing I was doing for real was the regex.pattern above, where I want to generate strings, but describe those strings in terms of spec
2019:02:13 20:30:38 seancorfield Generators must produce values that are acceptable to your spec.
2019:02:13 20:31:23 seancorfield (and we've had repeated cautions from @alexmiller not to use spec regex for string parsing/generation stuff 🙂 )
2019:02:13 20:33:03 borkdude for parsing yes, because of performance, there are better tools, but for generation, I currently don’t know a better tool 😛
2019:02:13 20:35:07 seancorfield Why not use test.chucks regex string generator?
2019:02:13 20:35:16 seancorfield (or did I miss your rationale for not using that?)
2019:02:13 20:35:41 borkdude I want to generate regexes, not strings that are matched by a given regex
2019:02:13 20:38:19 borkdude once I have that, I can use test.chuck to generate strings from the generated regexes. and then I can use stest/check to test re-find, etc.
2019:02:13 20:52:11 alexmiller why not make a regex for regexes?
2019:02:13 22:38:20 aisamu Can't tell if serious
2019:02:13 23:08:08 alexmiller I'm not sure either
2019:02:13 20:52:19 alexmiller then use test.chuck on it
2019:02:13 20:58:04 borkdude regex language cannot be expressed with a regex, I think you need a CFG tool like spec
At least I’m now finding return values that I didn’t account for in an early version of the spec, e.g.:
["ba" "a" "a" "" "" nil nil "" "" ""]
2019:02:13 21:46:28 borkdude afk now
2019:02:14 10:01:00 borkdude Seems to work now. I’m generating regexes using a simplified regex spec (that exists solely for generating) and I’m generating strings that match it using test.check. I’m using these combinations to test re-find, etc. Thanks for the suggestions all 🙂
2019:02:14 10:04:49 borkdude I think I’ll just replace the generator on CLJS with a simpler one
2019:02:14 11:33:37 gfredericks @borkdude there's a PR for that 🙂
2019:02:14 11:33:51 gfredericks I'm not sure if it works though
2019:02:14 11:34:40 borkdude PR seems out of date. Even if it’s incomplete I think it’s better than nothing? @wilkerlucio
2019:02:14 11:36:35 gfredericks you're suggesting I just merge it?
I'm not sure what you mean by "out of date"
2019:02:14 11:37:02 borkdude that the PR has merge conflicts
2019:02:14 11:41:16 wilkerlucio I wouldn't merge it, the range of things it works is quite narrow at this point, I'm using something simpler to generate strings, if we want to get that working in cljs (which would be awesome) that code needs better testing and impl, not good to merge as is IMO.
2019:02:14 11:42:20 wilkerlucio what I'm using for cljs these days is a much simpler string generator that knows just some basic patterns (numbers, letters, alphanum):
2019:02:14 11:35:57 borkdude I just realized that core.match could play nice with clojure spec conform results (in general, not related to this regex discussion)?
https://stackoverflow.com/a/54687183/6264
2019:02:14 11:36:05 borkdude (probably realized a little late)
2019:02:14 13:02:32 joshkh i'm probably missing something obvious -- how can i define a spec so that its value conforms to one of a collection of specs? for example, a person can have a pet that's either a mammal or a fish, neither of which share a common attribute.
2019:02:14 13:04:26 borkdude you need to tag the alternatives with a key
2019:02:14 13:04:32 joshkh ah!
2019:02:14 13:04:50 joshkh i saw that in cat/alt, didn't realise it applied to /or. thanks as usual, borkdude.
2019:02:14 13:40:22 andy.fingerhut borkdude, are you basically looking for bugs in the JVM's and/or JavaScript's regex library using spec and generative testing?
2019:02:14 13:42:02 andy.fingerhut I wouldn't be surprised if you find bugs in those, and/or test.chuck's generate-a-string-matching-a-regex-from-a-regex code, or all of the above. Just so you realize that the time scale for fixing the JVM and/or JavaScript's regex library might be a bit long 🙂 Those libraries must be challenging to maintain.
2019:02:14 13:45:31 borkdude The situation is like this: someone used Orchestra with speculative specs. Orchestra checks ret specs at runtime (in contrast to spec). It turned out the ret specs of regex functions were incomplete and I could have found this if I had used generative testing (which I have for almost all spec’ed functions except these). Now I have almost fixed this generative testing, but I ran into the CLJS limitation of test.chuck as a last issue.
2019:02:14 13:46:23 borkdude I can bypass this by generating simpler strings for CLJS. It will still find the case I forgot to spec.
2019:02:14 13:48:01 borkdude I’m not using it for anything more serious than this, just wanted to see how for I could go with it.
2019:02:14 14:16:44 mpenet any planned changes (or options) about fn as argument triggering gen, something like wrapping their arguments with asserts instead so that they fail at invoke time ?
2019:02:14 14:17:04 mpenet I think there was a jira issue about this
2019:02:14 14:21:57 alexmiller will revisit, haven't yet
2019:02:14 17:02:10 djtango Is there a way to provide a custom generator for a spec distant to the spec's definition/local to the test context? The docs for instrument say that the :replace only relates to fn-specs and :gen option is only for stubbed vars
2019:02:14 17:03:47 borkdude @djtango do you want to test with stest/check or instrument?
2019:02:14 17:07:03 djtango I think right now want to manually generate some inputs using gen/sample
2019:02:14 17:07:19 djtango though could also do stest/check - was it stest/check that lets you also patch in your own generators?
Overriding the generator on a spec that’s made with with-gen has no effect apparently?
2019:02:16 14:49:19 alexmiller Oh, that’s true based on the impl for sure
2019:02:16 20:10:30 don.dwoske I finally watched the Maybe Not talk and have some comments. Agree totally based on our history of modeling a biomedical research domain that schema/select is a great idea. A context is needed in order to know the required/optionality. For us information gets added to entities over time, what is an optional field in one stage in the pipeline, may be required in a future stage.
However, I think something additional is missing. Our predicates also depend on the context. The allowable values for a key depend on the context, so we want to attach narrowing predicates at that 'select' time, not just the required/optionality part.
Hopefully a simple example: if we define a Car entity, with a make, the make is validated against a CV list that has lots of makes in it. However, if we are working within a particular dealership context - a dealership which only sells Jeeps and Land Rovers, we want to add a new predicate at that select time which not only requires the make be present, but that it also be one of those two values, and not just any kind of make. Not only is the shape gaining requirements in some contexts, the allowable values are as well.
The predicates added at select time are always and- they reduce the set of allowable values and cannot make it more general - thus, there is no breaking change to the original s/def contract for the key. Rich was close to this idea during his talk and applied it to shapes, but not to predicates. Hopefully, I've described all this well enough, and I'm not missing something obvious, but my Google/Slack-fu didn't turn up anything relevant on this. Thanks.
2019:02:17 12:30:48 wilkerlucio @U4NQYBCU8 I trully believe in this idea of information getting adding over selections over time, maybe you will be interesting to check a library that I maintain that pushes this idea futher, and provides a system where by defining inputs and outputs as maps (with schema/select like definitions), the system can transition from some input to some output, given there is some available path in the system. this is the lib: https://github.com/wilkerlucio/pathom and if you like you can see I talk I gave about it (also in this conj :)) https://www.youtube.com/watch?v=yyVKf2U8YVg
2019:02:19 22:38:56 johanatan You can get an idea how to use it by reading the tests (including spec generators) and source code.
2019:02:19 22:39:34 johanatan The strong point of this lib is very user-friendly validation failure error messages.
2019:02:16 21:06:38 alexmiller you can s/and additional predicates any time you like
2019:02:18 22:44:57 don.dwoske But that changes things in a global context... an earlier stage context doesn't want to 'see' the additional s/and for that def.
2019:02:18 23:00:33 alexmiller I’m saying you can s/and additions predicates into a new spec, either at the point of use or registered
2019:02:18 23:16:06 don.dwoske But isn't the point of schema/select being able to reuse the same schema in different select contexts, so that registering new specs is not necessary. e.g. I want to use the same schema in two contexts 1) makes allow only land rovers and jeeps and 2) makes allow fords and chryslers ... what I'm doing there is wanting to add predicates at the select stage. Am I missing something here?
2019:02:18 23:20:42 don.dwoske It's not always a linear accretion and narrowing of information - there are forks where one part of the system has different rules for validating cars than another part (e.g. one dealership has different rules than another) but the schema for a car is always the same.
2019:02:19 18:51:43 butterguns Would something like this work in spec2?
(s/def ::model string?)
(s/def ::car (s/keys [::model]))
;; Would "select" the ::model key AND apply additional rules
(s/select ::car [(s/and ::model #(is-made-by-jeep? %))])
(s/select ::car [(s/and ::model #(is-made-by-ford? %))])
2019:02:19 18:52:05 butterguns (pseudo code of course)
2019:02:19 19:01:35 alexmiller not currently
2019:02:19 19:04:46 butterguns I'm not sure I understand when you say "you can s/and additional predicates any time you like". I want to add additional predicates at "selection-time", like above
2019:02:19 19:05:44 alexmiller I get what you're asking
2019:02:19 19:06:00 alexmiller I don't have an answer for you right now
2019:02:19 19:08:31 butterguns No problem! Was just spit-balling answers for @U4NQYBCU8
2019:02:20 18:44:57 don.dwoske Thanks. Just want to be understood and acknowledged, no answers needed... I think you get it. @UFQAPAUU8 example is approx. the thing we want.
However, another thing to make clear is that selecting for required/optionality is orthogonal to adding predicates to keys on the schema within a context. A key may still be optional in a spec context - but if the key/value happens to be there, I want to add predicates to that key's spec. Some psuedo-stuff :
(s/def ::id int?)
(s/def ::make string?)
(s/def ::model string?)
(s/def ::color string?)
(s/def ::car (s/schema [[::id ::make ::model ::color]]))
(favorite-car car) needs ::make, ::model
(favorite-car) checks (::make is 'jeep') and (::model is 'wrangler')
(favorite-car) checks ::color is 'red' or 'black'
For favorite-car, make and model are required, color is still optional. Predicates in the context are added to both required and optional fields to narrow the specification.
2019:02:17 21:23:07 hmaurer Speaking of schema/select, what’s the current state of this? Is alpha code already available?
2019:02:18 07:47:51 jaihindhreddy Where f and g are functions I've written, and i is any integer i.e, for any integer i, this property must hold. Can I express this with spec, or should I drop down to test.check?
2019:02:18 13:36:39 misha > I’ve been working on the new select functionality for spec 2 this week
♥
2019:02:18 14:18:36 gfredericks @jaihindh.reddy I think if i is not an arg to target then you can't easily do that with spec; you'd have to maybe write another function target* that also takes i, and you'd only use it for testing, which is a bit weird
2019:02:19 12:56:55 alexmiller You need to force eval somehow so macro around it, use eval, etc. this is an area where there are more and different answers in spec 2
2019:02:19 12:57:34 sonnyto thanks. i used def-impl directly . is this a problem?
2019:02:19 13:53:40 alexmiller it's fine (although won't work in spec 2)
2019:02:19 13:54:09 alexmiller there is a new function in spec 2 called register that is similar
2019:02:19 16:07:38 borkdude Is metadata and docstrings on specs considered for spec2? E.g. I’d like to conditionally instrument them based on metadata
2019:02:19 16:16:53 alexmiller yes
2019:02:19 18:29:36 borkdude bananadance
2019:02:19 18:33:26 alexmiller it was in scope for spec 1 we just never got to it :)
2019:02:20 01:02:03 jsa-aerial I have question that more experienced people here will likely think obvious. Say I have a map with some keys, two of which need to have the same value. For example m, {::id v1 ::nm v2 ...}, and I want to enforce (= (m ::id) (m ::nm)). I haven't seen this sort of example, but presumably it is covered by spec?? Thanks in advance for any insight!
2019:02:20 01:30:17 jsa-aerial Hmmmm, this doesn't give errors, but it doesn't work either:
2019:02:20 01:31:19 jsa-aerial
2019:02:20 01:40:18 jsa-aerial Hmmmm, looks like id-eq-nm? needs to use (m :sid) and (m :snm)
2019:02:20 15:42:10 kvlt Hey all. I'm writing specs for a few controller functions. They take in request maps and hand off those maps to have work done.
The issue I'm experiencing is that a collection inside of that (`body-params`) can vary by quite a bit depending on which endpoint is called and I don't like the idea of writing a really loose spec that can work with all of the say, user endpoints. As such:
(s/def ::body-params (s/keys :opt [::username ::email ::....]))
and would prefer to have specs setup for each call:
(s/def ::body-params (s/keys :req [::email] :opt [:...])).
Without putting each endpoint into a new file. How do I go about handling this?
2019:02:20 17:24:27 johanatan Isn’t it possible to have more than one ns form per file?
2019:02:20 15:47:48 borkdude I think this is a problem which spec2 solves
2019:02:20 15:48:16 borkdude you can watch the latest Rich Hickey talk on youtube if you want to know more
2019:02:20 15:48:47 borkdude not that this is solving your problem right now… sorry 🙂
2019:02:20 15:52:06 kvlt Yeah, I saw the talk. I was just kinda hoping that there was something I could do naow
2019:02:20 16:07:01 butterguns Could a multi-spec be used for this?
2019:02:20 16:15:58 butterguns You'll have to do a step first that assigns the request-type key to the map, depending on the controller that was callled
2019:02:20 21:01:24 borkdude @guy as far as I know orchestra only changes instrumentation (it includes ret + fn checks). you still define specs with normal spec, so everything should still work. orchestra uses different namespaces, so it doesn’t patch/replace spec itself (I think)
2019:02:20 21:02:33 borkdude @jeaye might be able to confirm or correct this
2019:02:20 21:07:28 guy Thanks!
2019:02:20 21:07:49 guy Also i’m a little confused about how to use :fn
2019:02:20 21:08:49 guy Am i supposed to try and link the :args and :ret using :fn ?
As in when a certain input is something, then the return should be something, using fn to validate that?
2019:02:20 21:09:22 jeaye Yep, you still use spec for everything. Just use orchestra to enable instrumentation and consider using defn-spec to clean up your fns.
2019:02:20 21:12:19 guy ok thanks 🙂
2019:02:20 21:09:43 borkdude @guy with fn you can do additional checking on arg + ret and dependencies between them
2019:02:20 21:11:40 jeaye :fn is optional, so you'll likely know when you need it.
2019:02:20 21:11:44 borkdude @guy e.g. in the group-by spec I linked I did an additional check that group-by should not produce more elements than the input collection had
2019:02:20 21:13:51 guy 🤔
2019:02:20 21:13:52 guy ok
2019:02:20 21:15:01 guy I guess it might be an issue with my understanding.
Lets say you have a function that takes some args and returns true or false.
Would you ever want to use :fn to sort of specify that when the args are nil, you return false
2019:02:20 21:15:29 guy I’m not sure if thats a bad example, but when i check an instrumented function, i want the args and return to make sense
2019:02:20 21:16:00 guy or maybe i’m thinking about it the wrong way :shrug:
2019:02:20 21:16:56 borkdude I think in the case of simple predicates fn specs might not be the most helpful. The clojure spec guide might provide you with more helpful ones.
2019:02:20 21:17:24 guy I’ll read the guide again and check that speculative lib too. Thanks 🙂
2019:02:20 22:45:32 drone ❤️defn-spec
2019:02:20 22:47:47 drone we’ve rolled our own for defrecord, but would like to see standard spec-enhanced forms be a thing. but I have a feeling they’re most useful for those of us using spec as a poor man’s type system, which is probably something Hickey wants to discourage
2019:02:20 23:02:19 alexmiller can you explain what you're talking about?
2019:02:20 23:03:07 alexmiller what is "spec-enhanced forms"?
2019:02:21 00:00:00 drone just like defn-spec adds inlined specs for function return values and parameters, defrecord-spec adds inlined specs for fields, along with adding specs to the built-in record construction functions
2019:02:21 00:00:35 drone eh, s/inlined/inplace
2019:02:21 00:01:01 alexmiller Ah. Rich is considering a spec-enhanced defn.
2019:02:21 00:12:24 drone defn and defrecord are the obvious ones to gain spec-enhanced forms. specs for protocol implementations could also be nice (in defrecord/deftype or extend-type). default specs for protocols (in defprotocol)? protocol related stuff I haven’t thought enough about. but have noticed it feels like a “spec gap”
2019:02:21 00:52:56 alexmiller Protocols can’t be spec’ed
2019:02:21 01:02:35 flyboarder @mrevelle if you keep your protocol an implementation detail and dont expose it as your public api it’s not a problem
2019:02:21 01:03:46 flyboarder for example (my-func) could be a spec’d function and (-my-func) could be a protocol function
2019:02:21 01:04:00 flyboarder then (my-func) calls (-my-func) internally
2019:02:21 01:04:22 flyboarder This is what we did with hoplon
2019:02:21 01:42:55 alexmiller These days we consider that a best practice in the core team
2019:02:21 03:35:40 drone protocols could be spec’d
2019:02:21 03:40:58 drone @flyboarder yeah, I suppose. not a fan of the redundancy and lack of documented expectations that specs may provide. it can also be useful to allow others to implement your protocols (see loom and ubergraph)
2019:02:21 03:58:40 dorab What am I doing wrong here?
2019:02:21 03:58:47 dorab
user> (require '[clojure.spec.alpha :as s])
nil
user> (require '[clojure.spec.test.alpha :as stest])
nil
user> (require '[clojure.spec.gen.alpha :as sgen])
nil
user> (s/def ::one-arg-fn (s/fspec :args (s/cat :x any?)))
:user/one-arg-fn
user> (defn map-vals
"Map the function f over all the values of the associative collection coll."
[f coll]
(reduce-kv (fn [m k v]
(assoc m k (f v)))
(empty coll)
coll))
#'user/map-vals
user> (s/fdef map-vals :args (s/cat :f ::one-arg-fn :coll associative?))
user/map-vals
user> (def xmap {:a " foo" :b "bar "})
#'user/xmap
user> (stest/instrument)
[user/map-vals]
user> (map-vals clojure.string/trim xmap)
Execution error - invalid arguments to user/map-vals at (REPL:65).
(nil) - failed: (apply fn) at: [:f] spec: :user/one-arg-fn
user>
2019:02:21 04:10:06 alexmiller when you use an instrumented fspec, it will actually generate values according to the fspec args spec and invoke the function you pass with them
2019:02:21 04:10:19 alexmiller here you declared that the fspec function takes an any? arg
2019:02:21 04:10:32 alexmiller the generator generated nil and invoked clojure.string/trim with it
2019:02:21 04:10:40 alexmiller which throws
2019:02:21 04:12:03 dorab Ah. Thanks. So, I should use a s/with-gen in the spec definition of ::one-arg-fn?
2019:02:21 04:12:05 alexmiller so it's effectively telling you that when passing clojure.string/trim to map-vals, you have passed a function that will not take an any?
2019:02:21 04:12:24 alexmiller you can
2019:02:21 04:12:38 alexmiller or instead of fspec, many people find it easier to just use ifn?
2019:02:21 04:13:00 alexmiller this generative test on instrument of fspec is a feature that many people find surprising (or wrong)
2019:02:21 04:13:15 alexmiller and something we're going to revisit in spec 2
2019:02:21 04:14:25 dorab OK. Thanks for the explanation. What would be the recommended way in spec1 to write a spec for a single arity function?
2019:02:21 04:17:17 dorab Or, is the recommendation to just use ifn? as you mention above?
2019:02:21 04:28:25 alexmiller I would just do that
2019:02:21 04:28:48 alexmiller with current state, I don't find that I gain anything but headaches with fspecs
2019:02:21 04:32:38 dorab Thanks.
2019:02:21 05:57:13 gklijs What would be the best way to prepare for spec 2. I now sometimes use the :opt is it useful to rewrite those to :req and make the values also allow nil?
2019:02:21 06:27:29 seancorfield @gklijs s/keys will stay as-is, as far as I know, so you will be able to move to s/schema and s/select piecemeal over time as you need it.
2019:02:21 06:28:43 seancorfield I have our codebase running against spec2 with minimal changes (by depending on the latest git SHA of clojure/spec-alpha2 -- since we use CLI/`deps.edn`) but it's still very much a moving target with no formal release (and still some bugs being worked out).
2019:02:21 06:30:05 seancorfield The main thing you may trip over is if you have used a predicate in a context that is really expecting a spec -- spec1 allows that but spec2 does not. Also, some constructs are now stricter about what constitutes a "spec".
2019:02:21 06:31:10 alexmiller the fate of s/keys is not yet determined
2019:02:21 06:31:35 seancorfield Oooh... so it may go away before spec2 is "released"?
2019:02:21 06:32:22 alexmiller maybe, but really don't know yet
2019:02:21 06:32:22 seancorfield That would make the migration from spec1 to spec2 a lot more work!
2019:02:21 06:32:37 alexmiller at the moment I'd say it's more likely to still be there
2019:02:21 06:33:28 alexmiller the idea behind schema has shifted a lot since Rich's talk, not sure if it will survive in the form described then
2019:02:21 06:33:44 alexmiller he's deep in the hammock :)
2019:02:21 06:33:59 seancorfield Interesting... I look forward to seeing how this evolves then...
2019:02:21 06:34:16 seancorfield And, yeah, I feel that hammock... I'm still deep in thought about next.jdbc🙂
2019:02:21 06:34:20 alexmiller @gklijs I would not rewrite opts to nilable reqs - I think that's the wrong direction
2019:02:21 06:34:44 alexmiller really, unless you're a "riding the bleeding edge" guy like Sean, I would just wait
2019:02:21 06:35:54 alexmiller select will have both required and optional selections
2019:02:21 06:36:29 alexmiller although the semantics around optional stuff is a little different
2019:02:21 06:39:17 gklijs Ok, I'll leave it for now then. I now use it as part of (de)serialisation, with optional added defaults on a subset of what can be specced, also using the spec's to generate the cljs functions to edit the data. I probably need to write some sort of migration at a point, but that should be trivial. Might use another namespace to spec the existing data in spec2 at some point.
2019:02:21 15:26:48 guy If you have two specs but you want to have a shared id between the two, is that possible in a generator? But also the id’s are generated as well. 👀
2019:02:21 15:29:58 borkdude example please?
2019:02:21 15:31:11 misha you'll need custom generator, which probably will have to generate examples for both specs at the same time. Another funky option is to override default id gen to generate predefined small set of ids, and then filter out examples generated by "default" gens which happen to share an id
2019:02:21 15:31:11 misha you'll need custom generator, which probably will have to generate examples for both specs at the same time. Another funky option is to override default id gen to generate predefined small set of ids, and then filter out examples generated by "default" gens which happen to share an id
2019:02:21 15:32:21 guy thanks! that makes sense
2019:02:21 15:34:08 misha first option would look like:
1) gen set of ids,
2) gen set of obj1 with default gen
3) gen set of obj2 with default gen
4) override ids in pairs of obj1 obj2 with ids from 1)
2019:02:21 15:35:10 misha this will allow you to leverage default spec gens w/o any modification, and just build higher level one
2019:02:21 15:37:12 misha writing a gen for entire family of interdependent objs is hard -_-
2019:02:22 01:35:40 jsa-aerial So, what is the status of CLJ-2320? In my case I can limp along with s/merge, but this is (according to the issue itself) a 'major' priority but does not appear to have had any activity since last June. OTOH, it doesn't have any votes either. I don't have an account so can't log in to vote anyway...
2019:02:22 01:37:44 jsa-aerial Also, why would what is effectively a 'logic system' use short circuiting semantics for an 'and' operator???
2019:02:22 03:22:13 seancorfield @jsa-aerial Anyone can create a JIRA account, I believe...
2019:02:22 03:23:12 seancorfield I think s/and really has to be short-circuiting since some predicates may blow up if they don't have a guard predicate ahead of them...?
2019:02:22 16:07:20 jsa-aerial @U04V70XH6 Sounds like a likely explanation, but in this context that 'should not happen'. Put another way, that should be up the user's predicates. If a 'guard' operator is wanted - call it 'guard' or some such.
2019:02:22 08:09:06 Christian Johansen Is there any way to associate a docstring with a spec?
2019:02:22 08:10:29 Josip Gracin not in spec1, AFAIK
2019:02:22 08:10:42 Christian Johansen do you know if it's planned for 2?
2019:02:22 08:12:12 Josip Gracin I should probably let Alex answer that, but I believe it will probably be in spec2
2019:02:22 08:15:27 Josip Gracin my belief is based on previous discussions of this on slack and mailing lists
2019:02:22 08:26:58 Christian Johansen ok, cool. I haven't followed either of those closely, so crossing my fingers 🙂
2019:02:23 16:23:23 adc17 Is there a way of following progress on the s/schema/`s/select` functionality that Rich mentioned in his talk? I can't see an issue tracking this on Jira (I also can't see spec in the clojure-contrib libraries)?
2019:02:23 16:48:43 alexmiller clojure-contrib is very old and deprecated
2019:02:24 21:04:20 eoliphant Hi, trying to muddle through a modeling scenario in the context of spec. In one of my domains I work with grants. so a grant has a unique :grant/number that’s a string of some format “GRT-999”, etc., string? + regex and I’m good. But i also have to deal with the fact that various ‘issuers’ of grant’s have their own formats. Issuer A, uses “GRANT-999", B uses “G_999”,etc. So, I’m grappling with somehow providing ‘context’ to the spec. Or are these better modeled as different things entirely :issuer-a/grant-number, :issuer-b/grant-number?
2019:02:24 21:09:22 borkdude I can imagine that a grant has a :grant/issuer and based on that value you can verify if the :grant/number is valid, possibly using s/and or maybe using a multi-spec?
2019:02:25 07:31:23 misha
2019:02:25 07:34:04 misha @eoliphant ^^^ this is a bit simplified, as you probably will not have issuers as keyword, but it can be expanded with mapping of issuer to number format.
2019:02:25 07:36:06 misha you'd also need to customize generators a bit, if you need them.
2019:02:25 10:49:24 Audrius what if to use Spec with regular (non namespaced) keywords? should it work OK?
2019:02:25 10:49:39 Audrius is there a too t generate spec from data structure?
2019:02:25 10:51:02 borkdude @masta I believe spec-tools has such a tool
2019:02:25 14:08:18 eoliphant Thanks @misha, @borkdude. One other bit of fun is that at the end of the day, the number of issuers/formats could be large and might end up in a db
2019:02:25 14:35:15 rickmoynihan Is it possible to turn clojure/core.specs off during macroexpansion? So you can see the failing expansion?
2019:02:25 14:39:23 rickmoynihan I’m assuming there’s a reason this couldn’t also be a set!able var? Not that restarting my REPL is that difficult! 🙂
2019:02:25 14:40:41 alexmiller well it's in a system property as stuff is potentially checked while you're loading before you have a chance to set it
2019:02:25 14:41:22 alexmiller it could probably also be a dynvar (although that would need to be checked on every macroexpansion, so probably some perf impacts)
2019:02:25 14:46:44 Audrius
(spec/fdef data->graph
:args ::data
:ret ::graph)
why this does not work? I mean it will take for ever to start...
2019:02:25 14:47:40 alexmiller "not work" == ?
2019:02:25 14:51:24 Audrius process starts for ever
2019:02:25 14:54:39 alexmiller is data->graph a macro or a function?
2019:02:25 14:55:09 alexmiller if a function, it shouldn't do anything until you instrument
2019:02:25 14:55:58 Audrius it is a function...
2019:02:25 15:09:42 Audrius still running 🤔 maybe it is too complex for Spec to handle...
2019:02:25 15:10:21 Audrius and runs for ever to start...
2019:02:25 15:23:53 guy Have you tried (exercise ::data 3) to see if outputs anything?
2019:02:25 15:29:51 Audrius also runs for ever...
2019:02:25 15:31:31 Audrius Found it...
2019:02:25 15:31:45 Audrius (spec/def ::data (clojure.spec.alpha/def is recursive or something 😄
2019:02:25 19:52:06 jrwdunham Hi here! I'm hoping someone can help me out with what is probably a basic question about clojure spec
2019:02:25 19:52:12 jrwdunham I have this function:
2019:02:25 19:52:13 jrwdunham
(defn starts-with-any?
"Return truthy if s starts with any character in chrs; otherwise nil."
[s chrs]
(seq (filter (fn [c] (string/starts-with? s (str c))) chrs)))
2019:02:25 19:54:57 jrwdunham it's failing on a nil ret, but shouldn't it pass on the first branch of s/or?
2019:02:25 19:55:28 borkdude it’s failing on the path [:fn :nil-case], so that would be the first branch
2019:02:25 19:55:47 jrwdunham oh right, and it needs :ret
2019:02:25 19:55:50 jrwdunham d'oh
2019:02:25 19:56:01 alexmiller yeah
2019:02:25 19:56:26 jrwdunham haha, thanks. sorry for the typo question
2019:02:25 19:57:14 alexmiller duckie
2019:02:25 22:53:52 robertfw Is there a recommended/succinct way to reuse a string regex spec in keyword form? That is, I have a spec that uses a regex to define a string format (in this case, it's an ID format). In some places, that value is used as a keyword in a map. I could break out the specifics, but was wondering if there is an easy way to do something in the spirit of (s/def ::id-as-keyword (as-keyword ::id-spec))
2019:02:26 12:12:15 Audrius how to make spec for list of maps? '({}{}{}) ?
2019:02:26 12:12:59 mpenet (s/coll-of map?)
2019:02:26 12:13:24 mpenet but most of the time you want something more precise than map?
2019:02:26 12:13:54 Audrius but how to make that the maps also conforms to a spec?
2019:02:26 12:15:20 mpenet replace map? with a namespaced keyword that points to a spec
2019:02:26 12:15:35 mpenet or a spec directly
2019:02:26 15:53:24 jsa-aerial Any reason why (s/explain-data a-spec a-value) does exactly what is expected (and is correct) and given the exact same input (s/valid? a-spec a-value) fails with a class cast exception? This is on 1.9
2019:02:26 15:54:46 borkdude do you have the spec and example data? did you test in clj 1.10?
2019:02:26 16:45:45 jsa-aerial @favila doesn't look to be that (might be peripherally related). I think the issue is related to a misuse of s/merge in my case. I have a set of predicates def'd to specs. None of these are map/key related (like with s/keys) so none involve a map. Then I use s/merge (because s/and short circuits...) on these. Now, s/merge is happy to do this. s/explain-data is happy to do 'the right thing' and check all predicates of the merged spec. But s/valid? has a bug as well, but instead of 'doing the right thing' it just blows up. I put 'right thing' in quotes because I suppose s/merge should not allow this in the first place. All of this is a shame - s/and should not be short circuiting in the first place. This is a logic system after all. If you are concerned about guarding flow through to other predicates then your predicates are what's broken. I suppose you could have a guard operator for that sort of case, but IMO, that would still be bogus. ¯\(ツ)/¯
2019:02:26 16:46:36 favila why would you want s/and not to short-circuit?
2019:02:26 16:48:28 jsa-aerial Because you are saying that the spec should satisfy all predicates. Like in propositional logic. Not programming languages
2019:02:26 16:49:22 jsa-aerial The reason this is really useful is because you would like to catch all the errors in one go - not get one report to user. fix that one, get the next, report to user, etc. That is really annoying and just wrong
2019:02:26 18:51:21 favila but s/and accepts specs and conformers
2019:02:26 18:51:33 favila not just predicates
2019:02:26 18:51:45 favila well, nm just conforming is a problem
2019:02:26 18:52:09 favila so you would have to drop conformers, or introduce some trickery about evaluation
2019:02:26 18:52:56 favila even if all were evaluated order would still have to matter
2019:02:26 18:55:00 seancorfield @jsa-aerial s/and flows data through all the specs so it must be short-circuiting -- it has an inherent order and you can't rely on being able to apply subsequent specs if an earlier spec fails. (and I feel we've had this conversation before)
2019:02:26 19:13:21 jsa-aerial It is what it is - I think if you wanted that 'guarded flow` capability that another name would have been far better. I don't really care TBH, I just think it was a poor choice.
2019:02:26 19:04:44 misha Yeah, each next spec in s/and receives conformed value from prior one. So if one step is invalid – next would just blow up with random noise instead of useful explain data.
2019:02:26 19:09:09 misha otherwise how would you s/conform to s/and spec? conform only last step? none? Most of the s/and specs I saw – check fields first, and then some relation between them next (e.g. this id and that id are the same.), which is basically: 1st step is conforming, last is not
2019:02:26 19:09:56 favila if s/and were restricted to predicates, and predicates were written defensively (which they should be IMO), then I think what @jsa-aerial proposes would be fine; order could be used for making generators, but all could be checked in parallel
2019:02:26 19:10:43 misha how do you restrict to predicates only?
2019:02:26 19:10:53 favila I mean if conformers were not legal
2019:02:26 19:11:05 favila bascially, you would have to drop conformers as a feature of s/and
2019:02:26 19:11:12 misha well, this excludes s/keys opieop
2019:02:26 19:11:17 misha or anything containing s/or
2019:02:26 19:12:05 favila I don't follow?
2019:02:26 19:13:05 favila I'm just talking about s/and. Maybe for clarity I should say this would not be s/and anymore. s/nonconforming-and or s/parallel-and
2019:02:26 19:13:56 favila s/all maybe
2019:02:26 19:14:20 misha I'd call it s/every or something. But "flowing" behavior was surprising to me back then, I agree
2019:02:26 19:15:23 misha but, how would such s/every be more useful than clojure.core/and?
2019:02:26 19:15:43 favila every predicate would be tested unconditionally
2019:02:26 19:15:53 favila I should say, could be
2019:02:26 19:16:08 borkdude is this for form validation or something?
2019:02:26 19:18:14 jsa-aerial Not necessarily, could be anything. You just want to be able to report all findable problems at once - not piecemeal
2019:02:26 19:16:41 favila but I haven't found spec very useful for things like that
2019:02:26 19:18:57 jsa-aerial Actually it works amazingly good when coupled with phrase
2019:02:26 19:16:50 misha what extra would it provide?
2019:02:26 19:16:59 favila report all errors instead of just the first one
2019:02:26 19:17:01 borkdude maybe build your tool on top of spec, but not use s/and to collect all the errors. just call s/explain for each field for example
2019:02:26 19:17:08 misha you can't exercise it, can you? because it involves conforming
2019:02:26 19:17:45 favila conforming and excercising seem orthogonal?
2019:02:26 19:18:14 misha exercise gives you pairs of [generated conformed], I think?
2019:02:26 19:19:17 favila the conformed of s/every would always be identity
2019:02:26 19:19:27 favila or whatever, s/every-pred
2019:02:26 19:19:33 misha I think you'll be able to implement such s/every on top of spec2
2019:02:26 19:19:35 favila no conforming
2019:02:26 19:20:01 favila I'm more curious what the explain-data would look like
2019:02:26 19:20:43 misha ok, what if one of the items in s/every would be spec with s/and inside? it'll propagate conformed value, I think
2019:02:26 19:23:30 misha @seancorfield I don't see "not arbitrary predicates" there. It says "Qualified symbols (predicate function references)" are ok
2019:02:26 19:23:47 seancorfield @misha See the code I just posted above.
2019:02:26 19:24:41 misha what if you defn it first, instead of partial?
2019:02:26 19:25:17 favila probably it needs an fspec
2019:02:26 19:25:23 favila (just guessing)
2019:02:26 19:25:27 favila which sounds great to me
2019:02:26 19:25:49 borkdude
(s/def ::foo (s/and #(int? %) #(> 10 %)))
works in spec-alpha2
2019:02:26 19:26:02 misha I think it should be orthogonal: fdef and predicate fn
2019:02:26 19:26:22 seancorfield Yes, you can def a predicate from (partial > 10) and then it falls into the set of things that can be treated as a symbol spec. But the point I'm making is that s/and isn't intended to operate on predicates, it's for specs.
2019:02:26 19:26:51 misha in spec2? or in both 1 and 2?
2019:02:26 19:26:58 seancorfield We got our entire codebase up and running on Spec2 and we had several places where we had "mixed" specs and predicates and those were rejected and we had to rewrite things somewhat.
2019:02:26 19:27:15 favila so a def-ed predicate is a spec
2019:02:26 19:27:24 misha most of my use cases for s/and were s/keys + fn
2019:02:26 19:31:25 jsa-aerial If you have s/keys you can use s/merge which does satisfy all semantics
2019:02:26 19:27:37 favila well symbolic spec, not spec object
2019:02:26 19:27:47 borkdude @favila see above. anonymous functions are allowed, but you have to write them with fn or #(… %)
2019:02:26 19:28:17 favila that doesn't jive with the doc @seancorfield linked
2019:02:26 19:28:35 borkdude To be honest, I found it confusing too, but that’s how it works now
2019:02:26 19:28:59 misha also, my impression was, that any predicate is sort of first class in spec by design
2019:02:26 19:29:19 seancorfield It's an important (but subtle) distinction. Spec1 doesn't enforce it but Spec2 does.
2019:02:26 19:29:50 favila what about michiel's counter-example?
2019:02:26 19:30:13 seancorfield What counter-example?
2019:02:26 19:30:22 borkdude (fn [x] int?) is a “symbolic spec”, it’s just what they call it now
2019:02:26 19:30:39 favila "fn" is a spec form?
2019:02:26 19:30:40 borkdude but just simple symbol isn’t, like int?
2019:02:26 19:30:40 misha I see how spec2 could disallow fns returning fns as predicates (partial), but forbidding anonymous fns would be strange (and doing it only for few spec forms – just awkward)
2019:02:26 19:31:24 borkdude I think this will be a source of much confusion.
2019:02:26 19:31:25 seancorfield int? is acceptable as a symbolic spec.
2019:02:26 19:31:58 favila > Spec forms (lists/seqs), with spec op in function position, composed of other symbolic specs
> Qualified keywords (names that can be looked up in the registry)
> Qualified symbols (predicate function references)
> Sets of constant values (an enumeration)
2019:02:26 19:32:16 favila unless "fn" is a spec form, I don't see how anonymous inline fns would be allowed
2019:02:26 19:32:33 favila even then that seems iffy
2019:02:26 19:32:47 misha @jsa-aerial s/merge is for "map-validating specs", it's not for things like "also check that value in [:foo :bar] is the same as in [:baz]"
but s/and - is exactly for that
2019:02:26 19:33:22 jsa-aerial Yes, I know that - just saying it does satisfy all semantics
2019:02:26 19:33:46 misha which "all semantics"?
2019:02:26 19:34:01 seancorfield He means "not short-circuiting"
2019:02:26 19:34:05 jsa-aerial If you merge a bunch of specs it check them all
2019:02:26 19:34:34 misha I meant, it is (s/and keys-spec custom-pred) or (s/and regex-spec custom-pred)
2019:02:26 19:36:17 misha keys-spec could be (s/merge keys-spec1 keys-spec2 ...), yes. but important part is a custom-pred, which is usually (for me) goes last, and does not conform anything, but often requires conformed input, in case there are branches with different "postvalidation" semantics required
2019:02:27 14:58:41 Audrius Can I have spec for a map entry that it has keyword, but corresponds to a different spec?
I have such map {::value "" ::other-map {::value {::bla 1}}} so my ::value sometimes is string sometimes is map...
2019:02:27 14:58:41 Audrius Can I have spec for a map entry that it has keyword, but corresponds to a different spec?
I have such map {::value "" ::other-map {::value {::bla 1}}} so my ::value sometimes is string sometimes is map...
2019:02:27 15:36:18 favila My spec isn't going to help in your case
2019:02:27 15:36:31 favila If you can, you should change your map to use a different keyword
2019:02:27 15:36:55 favila or, it's possible you should be specing a higher-level language
2019:02:27 15:32:33 valerauko why would you want to do that?
2019:02:27 15:35:56 Audrius I amended my question...
2019:02:27 15:33:18 favila
2019:02:27 15:34:12 favila @vale Real World Data (tm) has fields whose type may change depending on what map it is in
2019:02:27 15:34:48 favila I appreciate and understand the philosophical stance of spec here, but it does cause friction
2019:02:27 15:38:56 favila The case I always seem to run in to is a field which has a baseline type (e.g. it should be number?) wherever it appears, but in certain kinds of maps (defined with s/keys) it must be some subset of its allowed values (e.g. even?)
2019:02:27 16:01:37 valerauko you could namespace those specific use cases
2019:02:27 16:02:06 valerauko like :map-with-strings/name and :map-with-whatevers/name
2019:02:27 16:02:21 valerauko and then :key-un to ignore the namespace
2019:02:27 16:06:38 valerauko that is imo the most spec-friendly way to say "a value as in outer-map and value as in inner-map"
2019:02:27 16:10:10 favila yeah, so basically put a transformation in front of all these functions
2019:02:27 16:10:31 favila like I said, friction
2019:02:27 16:11:35 favila @audrius if you control the keys and the shape of your data, you should use different keys (even just namespaces) as @vale suggests
2019:02:27 16:12:25 favila the spec philosophy of "one key one type" is a worthy goal
2019:02:27 16:12:27 valerauko no, you don't have to have different keys in your data
2019:02:27 16:12:40 valerauko just spec them in different namespaces
2019:02:27 16:12:54 favila this forces the concrete maps to use un-namespaced keys
2019:02:27 16:14:10 valerauko why would you have two names in the same namespace refer to multiple different things
2019:02:27 16:14:56 valerauko you're right i don't know your use case
2019:02:27 16:14:58 favila it's pretty easy to encounter data shaped like this
2019:02:27 16:15:03 favila e.g. datomic transaction maps
2019:02:27 16:15:12 valerauko i'll let smarter people argue the point
2019:02:27 16:18:04 lilactown An example would be like we are trying to spec a user’s medical records. They have a coverage type:
:coverage/type that can be #{:medical :dental :vision}
and then in a certain context, you want to specify it must be :medical along with some other facts about the user’s data
2019:02:27 16:18:21 lilactown right?
2019:02:27 16:18:32 favila that is exactly the case I encounter constantly
2019:02:27 16:18:42 favila but it is different from the OP's case which is more extreme
2019:02:27 16:19:00 favila here's an extreme example for a datomic transaction map: {:db/id (string, or tempid-record, or entity-id-long, or keyword-ident, or tuple[entity-id-long or keyword-ident, indexable-datomic-valuetype)}
2019:02:27 16:19:47 favila and suppose for a particular function you want to specify that no tempids are legal for :db/id
2019:02:27 16:19:58 favila you can't respec :db/id contextually
2019:02:27 16:21:12 favila you can add a predicate, but you have the burden of maintaining the predicate, a less useful s/merge, and custom generators
2019:02:27 16:21:56 favila or, you can make a new field and remap key names on the way in
2019:02:27 16:22:11 favila (i.e. transform your data for typing reasons)
2019:02:27 16:25:00 lilactown yeah. atm we have a specific medical-coverage? predicate that we combine with others. it’s not the greatest
2019:02:27 16:36:36 favila That gist I linked earlier may help you
2019:02:27 16:36:50 favila it was written exactly for the use case you described
2019:02:27 16:38:54 favila (keys+ :req [:coverage/type] :conf {:coverage/type #{:medical}}) would produce an s/keys spec where :coverage/type was checked both against its natural spec and any override you have in :conf, and :conf is the one used for generators
2019:02:27 16:40:28 lilactown that's awesome! thanks, I'll check it out
2019:02:27 16:54:43 drone for things related (but not the same) to this, we use different records with specs. the records may have overlapping keys, and records of different kinds can be determined by their record (really, class) type
2019:02:27 21:07:25 hiredman that sort of sounds like a multi spec
2019:02:27 21:07:43 hiredman keyed on :coverage/type
2019:02:27 21:53:20 lilactown the coverage type might be 1 part of many keys
2019:02:27 21:56:34 lilactown ex: {:coverage/type :medical :coverage/code 1234 :coverage/status :active} answers "does user have XYZ program", for example
2019:02:27 23:23:40 drone multi-spec uses a multi-method to retrieve the correct spec based on whatever dispatch function you choose. so you could dispatch on the value of :coverage/type and return different specs for each of your coverage record “types”
2019:02:27 23:24:08 favila that's not the goal here
2019:02:27 23:24:22 drone there seem to be at least two different things being discussed
2019:02:27 23:24:27 drone what is the goal
2019:02:27 23:25:41 favila Will and I were discussing the situation where we want to spec a map where an entry is only allowed some subset of what is normally allowed
2019:02:27 23:26:06 favila :coverage/type doesn't alter the type of the map (what multispec would allow)
2019:02:27 23:26:45 favila this is "my function takes coverage maps but only :medical ones"
2019:02:27 23:27:44 favila there's also an open vs closed tradeoff here
2019:02:27 23:27:54 favila multispec can only be open
2019:02:27 23:30:41 drone > spec a map where an entry is only allowed some subset of what is normally allowed
you mean a coverage map tagged as :medical can only contain some subset of all coverage map keys?
2019:02:27 23:31:04 favila no
2019:02:27 23:31:49 favila we mean there's a map type "coverage", one of its keys is coverage/type which can be #{:medical :dental} this function only deals with :coverage/type = :medical
2019:02:27 23:33:01 drone and spec’ing the function with medical-coverage? (mentioned above) is too much work?
2019:02:27 23:34:10 drone given this data model, I’m not seeing how else you’re proposing to check this aside from looking at the tag?
2019:02:27 23:35:32 favila you need to override the generator too
2019:02:27 23:35:44 favila you need to chain a bunch of one-off predicates if you have more subset constraints
2019:02:27 23:36:02 favila it would be nice to override in s/keys directly
2019:02:27 23:36:11 favila I wrote something hacky that does this
2019:02:27 23:36:41 favila (keys+ :req [:coverage/type] :conf {:coverage/type #{:medical}}) is what it looks like
2019:02:27 23:37:00 favila the spec in the :conf map is used for conforming and generation
2019:02:27 23:37:14 favila (oh, also you can't s/merge s/and)
2019:02:27 23:38:51 favila another alternative is key rewriting
2019:02:27 23:39:51 favila :coverage/type -> :coverage/type=medical, then you can (s/keys :conf [:coverage/type=medical), but that just shifts the pain around
2019:02:27 23:41:34 drone so, why not just have specs to represent each of the kinds of coverage: medical-coverage, dental-coverage, ... and then a type that represents the common, generic coverage?
2019:02:27 23:41:56 drone s/type/spec
2019:02:27 23:42:05 favila the constraint is, :coverage/type is the concrete key
2019:02:27 23:42:18 favila when you have a coverage map, it must have :coverage/type as the key
2019:02:27 23:42:27 favila but different functions want different values for that key
2019:02:27 23:43:56 favila so you can either chain predicates to your s/key with s/and (and lose niceties like s/merge, automatically correct generators, fewer predicates), or you can rewrite your key so that type and value match (and end up with key rewriting layers everywhere)
2019:02:27 23:44:21 favila what you cant do is say "for this keys spec, only allow some of what would normally be in :coverage/type"
2019:02:27 23:52:08 lilactown @mrevelle the point is that I don't actually care about the coverage type. what I do care about is answering some higher-level question about the data
2019:02:27 23:52:39 lilactown like I might want a spec that is called :user/super-cool-program
2019:02:27 23:53:24 drone @favila ah, thank you. I am tracking now
2019:02:27 23:53:28 lilactown and their coverage map should look like:
2019:02:27 23:53:46 favila S/keys blurs the lines between a spec and a predicate somewhat
2019:02:27 23:54:07 lilactown the code, type and status tell me that the user has "Super Cool Program"
2019:02:27 23:54:55 favila It’s a fancy composable predicate system for map-shaped data
2019:02:27 23:55:56 favila But you either obey its “spec key = type” constraint or you lose all the niceties it provides
2019:02:28 00:03:45 drone maybe spec is trying to do too much. it’s acting as a specification for: contracts, test value generators, and data coercions
2019:02:28 00:06:16 drone when you make use of chaining predicates to solve your problem, you can express the contract you’d like. but at the cost of support for the other areas
2019:02:28 00:08:09 drone I use spec only for contracts, so hadn’t experienced the pain
2019:02:28 00:31:26 drone it’s ironic that the argument for spec over any kind of static type system leans significantly on expressiveness. but with that expressiveness (i.e., arbitrary predicates) comes tradeoffs that make difficult to accomplish some of the other goals of spec
2019:02:28 02:05:35 misha @lilactown I think your :user/super-cool-program (or something as specific as such fields combination) should be just predicate with a custom generator.
2019:02:28 02:09:41 misha @favila what is the downside (and is there any?) of always having conformed datomic's :db/id tagged, like [:lookup-ref [:user/name "foo"]] or [:perm-int 12345]. Where do you actually need to constantly check against e.g. not-a-temp-id, so you'd be forced to come up with funky qualifying s/and, etc.?
2019:02:28 02:11:16 favila A function may only work with lookupable db ids
2019:02:28 02:11:18 misha ofc, I'd love to always get conformed {:db/id 1234} out of queries/pulls. But is it a high price?
2019:02:28 02:12:02 misha that means you'd need to conform first, and have function working with conformed map, right?
2019:02:28 02:12:21 misha ... or conform db/id within fn
2019:02:28 02:13:16 misha do you actually have an example of such fn? can't think of one from the top of my head
2019:02:28 02:13:19 favila Typing the result of pull is easy
2019:02:28 02:13:25 favila It’s consistent
2019:02:28 02:14:17 favila Typing a map intended for a tx is harder because you’re typing a particular instance of a dsl
2019:02:28 02:15:36 favila But that’s the issue right there
2019:02:28 02:15:47 misha how is it different, given you specced db/id as s/or?
oh wait. db/refs -_-
2019:02:28 02:16:08 favila :db/id or :my/ref-attr has different types
2019:02:28 02:16:37 misha yeah, db/refs are pain. even in pull btw.
2019:02:28 02:16:39 favila Depending on whether it is produced by a pull or embedded in the map of a TX Map dsl
2019:02:28 02:17:45 misha yeah, does this mean entire app's specs will know about datomic (app uses datomic and the :user/friends is a :db.type/ref)? or is there another way?
2019:02:28 02:18:43 favila Well that’s exactly the point
2019:02:28 02:19:28 favila Some functions deal with :db/id (just to pick one attr) in a pull result context
2019:02:28 02:19:51 favila I those causes they want to assert such values passing through them are always Kong’s
2019:02:28 02:19:56 favila Longs
2019:02:28 02:20:01 alexmiller you don't have to spec every attribute
2019:02:28 02:21:15 favila In other map contexts or parts of the app the same attr means something different (eg it’s meant for a tx later, or it can be anything d/entid can resolve)
2019:02:28 02:25:03 misha Alex, true. I need to think through what I will not get if I omit :db.type/refs from spec registry, e.g. generative testing or validation -wise
2019:02:28 08:32:42 Josip Gracin Is there any reason the following two definitions would not be equivalent? I'm having some problems with the one using partial.
2019:02:28 08:54:59 borkdude @josip.gracin Works for me:
2019:02:28 08:55:51 borkdude it could be that after you change the protocol, you have to re-define the spec. to be sure, just reload your program entirely
2019:02:28 08:57:52 Josip Gracin thanks! I just needed a sanity check before I go further into debugging. I have a situation where those two behave differently.
2019:02:28 10:13:35 misha might be related, if you use spec2
2019:02:28 10:55:44 Josip Gracin @misha thanks! I'm using spec1 for this.
2019:02:28 10:56:19 borkdude spec2 isn’t really finished, so not recommended to use yet I think
2019:03:01 12:08:33 djtango is there a way to provide a generator to a spec you want to use at runtime for validation, but have test.check only as a dev/test dependency?
2019:03:01 12:09:54 djtango can you augment a spec in the registry in a separate namespace?
2019:03:01 12:13:00 gfredericks @djtango the whole point of spec's proxy namespace for generators is to allow that
2019:03:01 12:13:31 gfredericks you reference/build the generator using that namespace, and as long as you don't use it, t.c isn't needed
2019:03:01 12:15:05 djtango hmm I must be doing something wrong
2019:03:01 12:15:48 gfredericks I think any function calls in that ns have to be delayed inside a (fn [] ...) so that they don't actually happen
2019:03:01 12:16:16 gfredericks which is part of why the spec api requires you to supply a 0-arg-function-that-returns-a-generator
2019:03:01 12:28:15 borkdude I could have moved this code to the test code and use a generator overrride
2019:03:02 00:17:45 ag has anyone had this problem? It seems instrument doesn’t see my fdefs and they simply are ignored. fdef and the function and instrument - all in Clojurescript
2019:03:02 00:18:34 ag I’m trying to use orchestra’s instrument - doesn’t work
2019:03:02 00:25:12 seancorfield @ag You've required the namespace containing the functions and the fdefs before running instrument?
2019:03:02 00:26:31 ag hmm… when I eval instrument it returns vector with speced symbols, but it doesn’t fail the spec
2019:03:02 00:27:03 ag I’m trying to check if wrong data would cause it to throw, but it doesn’t
2019:03:02 00:27:10 seancorfield (I'm not sure how instrument works in cljs so you may want to ask for help in #clojurescript if this seems to be a cljs-specific problem?)
2019:03:02 00:27:29 seancorfield Hard to offer more advice without seeing your code...
2019:03:02 00:28:14 seancorfield Also, have you tried spec's instrument directly, to eliminate an issue with Orchestra?
2019:03:02 00:28:47 ag yeah, neither working ;(
2019:03:02 00:29:34 seancorfield I would try it in a Clojure REPL to see if you can confirm the basics work as you expect. If they do, try that exact same example in cljs if you can.
2019:03:02 00:33:17 ag the basics… work… if I simply create a function and call it in the repl, but when it’s within the app - it doesn’t work
2019:03:02 00:33:31 ag probably has something to do with re-frame
2019:03:02 00:38:21 ag yeah, seems I’m right… if I call fdefed function directly - it works. But doesn’t automatically validate when used in the app
2019:03:02 00:38:47 ag and the fn is definitely being called
2019:03:02 04:27:38 seancorfield @ag Ah, I suspect the code holds onto the original function value and doesn't see the instrumented version...?
2019:03:02 11:55:09 metametadata Hi. Is there any library which allows enriching the error message of the function-based spec?
Default:
(s/def ::alright (lib/with-explanation alright? (fn [x] (str x " is not alright!")))
(s/explain ::alright 123)
; Output: 123 - failed: alright? (123 is not alright!) spec: :user/alright
2019:03:02 12:00:25 borkdude maybe using the explain-printer stuff? expound would be a library to look at for an example of this
2019:03:02 12:50:04 metametadata setting custom :reason does the trick for me:
(defn with-reason-fn
"Will use (reason-fn [x]) to set :reason in explain data for the specified spec."
[spec reason-fn]
{:pre [(s/spec? spec) (ifn? reason-fn)]}
(reify s/Spec
(explain*
[_ path via in x]
(let [data (s/explain* spec path via in x)]
(map #(assoc % :reason (reason-fn x)) data)))
; Do not modify the rest of methods
(conform* [_ x] (s/conform* spec x))
(unform* [_ y] (s/unform* spec y))
(gen* [_ overrides path rmap] (s/gen* spec overrides path rmap))
(with-gen* [_ gfn] (s/with-gen* spec gfn))
(describe* [_] (s/describe* spec))))
(s/def ::alright (with-reason-fn (s/spec alright?) (fn [x] (str x " is not alright"))))
(s/explain ::alright 123)
; Output: 123 - failed: 123 is not alright spec: :user/alright
2019:03:02 14:34:57 Josip Gracin I'm trying to figure out why the following code is not working (using Clojure 1.10.0). The test fails. However, if I move the s/def definition of :n/repository after the extend-type line, it passes.
2019:03:02 14:37:01 borkdude what happens if you don’t use partial but #(satisfies? % Repository)? I’m not sure if that solved it, just curious
2019:03:02 14:38:59 Josip Gracin then it works. 🙂 you remembered my original question.
2019:03:02 14:39:22 Josip Gracin #(satisfies? Repository %)
2019:03:02 14:39:31 borkdude sorry yes
2019:03:02 14:40:00 Josip Gracin strange, isn't it?
2019:03:02 14:40:16 borkdude there was a discussion about this in the #beginners channel I believe
2019:03:02 14:41:14 borkdude partial just binds the value of the var too early to see re-definitions, I think it’s related to that, although I’m not sure how it works with protocols
2019:03:02 14:41:45 Josip Gracin yeah, I'm suspecting some interplay between protocol var, auto-generated interface and closures.
2019:03:02 14:42:07 Josip Gracin is extend-type redefining something?
2019:03:02 14:42:14 Josip Gracin I'll check
2019:03:02 14:42:57 borkdude I think a protocol is a var that gets redefined everytime you extend a type:
2019:03:04 16:43:09 jsa-aerial Is there a preferred idiom for the use of 'world information' in predicates and specs? This can occur in data validation. To clarify some, you have a data value, which you can check the 'shape/type' of via predicate and spec involving that value. Once that checks OK, a further validation requirement would be to see if what the value represents exists (in a DB or directory or ...) and is 'properly configured'. Those checks require looking outside the value so a predicate doing these checks needs more than the value. Maybe that is access to an in memory cache or a DB key or whatever. I can think of several ways to do this, but they all feel a bit off. So, is there an idom / preferred way to have that information available in predicates and specs?
2019:03:04 16:44:21 borkdude @jsa-aerial My gut feeling tells me that it’s not a great idea to do side effects in specs
2019:03:04 16:52:58 alexmiller specs are not a great match for these kinds of validations
2019:03:04 16:54:36 alexmiller in general, I've seen better success with dynamically generated static specs (like running some code to register a spec with a set of allowed values pulled from a db) than with dynamic specs (which somehow close over state required to dynamically check validity in some way)
2019:03:04 16:55:18 jsa-aerial I see - I can understand that. Basically this is 'out of scope'.
2019:03:04 16:55:28 alexmiller pushing the limits :)
2019:03:04 16:56:09 alexmiller might be easier to implement as an explicit validation implemented outside of spec
2019:03:04 16:57:03 jsa-aerial Yeah, that would be one of the ways I've considered
2019:03:04 16:56:35 jsa-aerial I will say there isn't any side effecting here, but it does use / need 'outside/ world' access
2019:03:04 16:57:38 borkdude maybe something like this would work though:
2019:03:04 16:58:24 borkdude depending on how you use spec -- this won’t work for e.g. fdefs
2019:03:04 17:02:53 jsa-aerial Some other ways: augment the value first with the 'key' for outside data so that the 'value' is wrapped and contains the necessary information. That's not too bad. If you have a map, augment with the 'outside key' and explicitly check the map contents (as opposed to using spec'd keys). Again, not too bad but kind of not using spec idiomatically...
2019:03:04 17:04:24 jsa-aerial Rather worse: use a dynamic var to hold the outside 'key'... This actually feels a bit dirty.
2019:03:04 17:11:14 alexmiller it is
2019:03:04 20:02:52 seancorfield It's also worth mentioning that, strictly speaking, reading values from a DB is side-effecting because if you repeat the call, you will not necessarily get the same result. It's not mutating anything directly, but it is not a pure operation either. I think a lot of people tend to think that SELECT * FROM whatever is "readonly" and therefore not subject to side-effects...
2019:03:04 22:42:12 jsa-aerial If nothing changes (anywhere), I don't see how it is 'side effecting' to read. The only way that would make sense is if the database were to change out from underneath you. In general that could happen, but in this case nothing of the sort is happening. Even more to the point, if it is already read and cached, nothing associated with the operations at hand is effecting anything anywhere.
2019:03:04 23:01:25 seancorfield That's why I said "strictly speaking" -- in the general case, where a database is a mutable thing that other processes or other threads could be updating, when you do a live read from the DB, you may get different results from different calls over time. Thus, the function doing the reading behaves as if it has side-effects because it is not pure. I was specifically excluding the case where the data is read once and cached.
2019:03:04 20:03:25 seancorfield (unless you're using Datomic where you can get the entire DB as an immutable value for whatever scope of request processing you want)
2019:03:04 21:03:35 ag how do I write fdef args for a function that takes: [[a b] _]?
2019:03:04 21:05:32 alexmiller there are several ways to potentially spec "[a b]" - what have you tried?
2019:03:04 21:05:58 alexmiller could be an s/tuple, s/coll-of with :count 2, a nested s/cat
2019:03:04 21:14:31 ag awesome! thank you @alexmiller and @borkdude!
2019:03:04 21:18:12 borkdude in spec2 this will be called (s/nest ..)
2019:03:04 22:44:46 robertfw apologies if this isn't the right place to ask - I've recently started using the replacement instrument function from jeaye/orchestra. I'm getting an error while generating one of my specs, the error is
2019:03:04 22:44:56 robertfw er, one moment while i clean that formatting
2019:03:04 22:45:38 robertfw I'm wondering if I'm missing something as I would expect to get a more detailed/specific error about what spec failed
2019:03:04 22:45:51 robertfw whereas the error seems to be something internal to orchestra
2019:03:04 22:58:58 seancorfield @robertfrederickwarner That looks to me like it expected a string but was passed (seq s) -- i.e., a sequence of characters instead.
2019:03:04 23:00:35 robertfw Yup - that's the problem within the spec. My confusion is more that the error from orchestra isn't what I'd expect to see - looking at https://github.com/jeaye/orchestra/blob/2019.02.06-1/src/clj/orchestra/spec/test.cljc @ line 115 (the line from the error above), I'd expect to see an error prefixed with "Call to..." followed by details about the spec that failed
2019:03:04 23:01:51 robertfw The problem itself - passing a sequence of chars instead of a string - is occurring within a generator for a spec so I am wondering if that is causing problems for the usual reporting
2019:03:04 23:02:53 robertfw I saw some discussion of orchestra here earlier so thought I would throw it out incase anyone here recognized something obvious, I'll dig around some more but may open a bug to see what @jeaye thinks
2019:03:04 23:51:30 robertfw did some further digging and tried out the standard instrument. Orchestra was hiding the error location a little bit but otherwise returns similar to the standard error
2019:03:04 23:52:20 jeaye @robertfrederickwarner So the issue exists in the upstream spec instrumentation as well?
2019:03:04 23:52:57 robertfw The upstream error I got was
2019:03:05 00:04:31 robertfw We have an ID value that uses a luhn check digit. The error was being caused by line 27 - where it is using subs, it was using drop-last which ended up returning the sequence of characters
2019:03:05 00:05:11 robertfw Is the expected behaviour to see what spec failed? e.g., hone in on luhn-check-digit being the failing fdef?
2019:03:05 00:21:06 jeaye What is regex-spec?
2019:03:05 00:21:21 jeaye Seems like that would be the problem to me.
2019:03:05 00:30:03 robertfw ah sorry I missed including that. It is...
2019:03:05 00:35:26 robertfw I do find that there are quite a few exception messages that could be a little more helpful. For example, right now I'm hunting down a problem with a misbehaving generator, and all I have to go off of is
Execution error (ExceptionInfo) at clojure.test.check.generators/such-that-helper (generators.cljc:320).
Couldn't satisfy such-that predicate after 100 tries.
2019:03:05 00:36:38 robertfw It's made a little more manageable through workflow (e.g., I know what spec I'm working on), but it would still be very helpful to get more details on what went wrong to help debugging
2019:03:05 00:47:22 robertfw (That's a general spec/test.check comment, not so much orchestra)
2019:03:05 00:48:12 jeaye I feel like you have a lot going on there, so it's not quite a minimal repro case at all.
2019:03:05 00:49:15 jeaye To figure out the source of that string sequence issue, keep removing code until the issue goes away. 🙂 If it goes away, add that code back and remove some other code until it goes away again. Keep repeating until the only code left is necessary for the reproduction.
2019:03:05 00:49:56 jeaye Namely, that spec error likely has nothing to do with generators or your custom macros or even all of that business logic around luhn digits.
2019:03:05 00:55:53 robertfw As mentioned earlier - I already fixed the actual issue with the code, my question is more about debugging flow and finding the culprit. In this case the error I was getting back was not very helpful in narrowing down where to look, and I was wondering if there was something I was doing wrong to impact that.
2019:03:05 01:04:02 jeaye Oh, that's my mistake. I came in last and must've missed that. I think the one thing I'd say we should keep in mind is that specs themselves should be simple enough to not really need debugging. To me, specs are for debugging, so it's awful to need to debug the debugging code. If you're stuck on weird issues with your specs, perhaps your approach can be simplified.
2019:03:05 01:05:35 jeaye Note, I also never use generators, since Orchestra + good unit and functional test coverage is everything I care about. However, especially given that this functionality (ret + fn instrumentation) has been removed from Clojure core and a lot of the spec press has been around generative testing, I may not be among the majority.
2019:03:05 01:09:13 robertfw We're still fairly fresh with it - we have one other clojure project in our repos but it is pre-spec. The generative testing has been useful so far, it's already caught a few issues that we probably wouldn't have spotted with our usual approach. But yeah.. it's definitely tricky going at times with more complex specs, especially for relatively new clojure devs like myself
2019:03:05 01:09:59 jeaye We have thousands of lines of specs and hundreds of spec'd functions in our code (at my company). I don't recall the last time I spent a noticeable amount of time debugging an issue with one of my specs. This is likely because they're all very simple. "This thing requires these keys. This is a number, this is a non-blank string, this matches this regex." Not too much more than that, for our entire front-end and back-end.
2019:03:05 01:11:17 robertfw The bulk of ours are like that and haven't given us any issue. Todays fun was caused by the more complicated spec for the luhned ID, and some specs defining incoming requests on our API that have some internal consistency properties
2019:03:05 01:11:59 jeaye Gotcha.
2019:03:05 01:12:16 robertfw Thanks for taking a look! I appreciate it. Orchestra was a great find for us, having the :ret and :fn checking on instrument is really simplifying our workflow
2019:03:05 01:12:30 jeaye I'm happy you're enjoying it. 🙂
2019:03:05 01:12:42 robertfw We were wondering why that wasn't part of the standard instrument
2019:03:05 01:12:51 robertfw It seems like a natural thing to do
2019:03:05 01:14:26 jeaye It was and then it was removed. My understanding is that the Clojure team didn't like to emphasize that usage of spec and noted the performance implications. To me, that is the usage of spec and the performance implications are negligible compared to the safety benefits.
2019:03:05 01:15:47 jeaye I wouldn't want to misrepresent them and I haven't actually spoken with them about it. Would love to sit down with Rich and talk it out, but who wouldn't love to sit down with Rich and talk anyway?
2019:03:05 01:17:18 robertfw Interesting. Yeah, no kidding! Definitely a brain to pick
2019:03:05 01:51:53 seancorfield It's because instrument is for checking calls pass the right data -- the :args key -- and check is for checking the functions themselves behave correctly (given conforming :args, does the output of the function satisfy :ret and :fn).
2019:03:05 01:52:11 seancorfield Orchestra complects those two, very different, types of checks.
2019:03:05 01:52:48 seancorfield (which is fine as long as folks understand that is what's going on and they are comfortable making that choice to go against how spec is designed)
2019:03:05 01:57:18 jeaye There's the official opinion. 🙂 Thanks, Sean. Fortunately, people now have the choice.
2019:03:05 03:16:35 seancorfield I wouldn't say "official", coming from me -- I'm just repeating what I understand the Cognitect folks to have said about function specs 🙂
2019:03:05 04:48:51 alexmiller yeah, that
2019:03:05 04:49:28 alexmiller ret specs are probably going to change a lot in spec 2 as well
2019:03:05 05:21:29 jeaye Hopefully not in a way which is incompatible with this form of instrumentation, I hope, Alex.
2019:03:05 05:21:59 jeaye Either upstream or via a soft fork such as Orchestra.
2019:03:05 05:24:02 seancorfield @jeaye At work we have a branch of our code running on Spec2 -- there were several substantive changes (beside the "obvious" renaming of namespaces in :require clauses). They weren't big changes, but they were breaking changes.
2019:03:05 05:24:53 jeaye Good to know, Sean. Thanks for the info.
2019:03:05 05:28:05 jeaye I'll wait and see how things are, when the dust settles. The libs, spec and spec2, aren't so large as to be very difficult to either work into this case or replace with something which does. There's superb thought work going into spec2, no doubt, and I have no interest in competing with that. I just want to make sure my team and other Orchestra users will be able to take advantage of the new spec features along with the instrumentation.
Could someone offer a good way of thinking about the above ^^^?
I.e. it is possible to refer to 'non-existent' spec inside map spec (through keys) but it's not possible at the "top level" s/def.
I guess my confusion is coming from the fact that semantically those uses seem equivalent to me--both use ns-qualified keyword as a way to refer to spec--one fails the other doesn't.
2019:03:05 13:05:27 alexmiller Aliased specs are not as delayed as they should be in this case, which is a known issue. I have been working on fixing it in spec 2.
2019:03:26 17:18:22 rickmoynihan ahh might be an old spec version…
2019:03:26 17:24:44 rickmoynihan Yeah I had an old spec; can confirm the issue is gone when using latest spec1 😌
2019:03:26 21:11:03 ag when defining a fdef spec for a function that is in different namespace, I can use fully qualified name. How can I alias fully-qualified part?
e.g:
- fn implementation of foo is in core.common
- spec is in core.specs
- I can do (s/fdef core.common/foo bla-bla-bla)
I cannot require core.common in core.specs, to avoid circular reference since specs are used in core.common. But because I’m using qualified name of foo - instrumentation still works.
But how can I use alias for core.common, so I don’t have to repeatedly write it fully? clojure.core/alias doesn’t work, because core.common cannot be imported in this namespace
2019:03:26 21:21:53 didibus I don't think there's a way.
2019:03:26 21:23:13 alexmiller currently, you can't do this (other than by creating it as a namespace, which will then allow you to alias it).
2019:03:26 21:25:28 ag (╯°□°）╯︵ ┻━┻ I’m sad and dissatisfied. It is totally entire Cognitect’s fault.
2019:03:26 21:26:43 lilactown it’s beginning to feel like we need something like alias but that doesn’t require the namespace to be loaded
2019:03:26 21:27:32 ag yeah… please have it… or I’m gonna… about to.. leave Clojure and do…. I dunno something else… woodworking or papier mache figurines to sell them on farmer’s market
2019:03:26 21:27:40 lilactown esp. in ClojureScript where you can’t even ad-hoc create the namespace
2019:03:26 21:44:44 alexmiller yeah, this is something we've talked about for a while and there is a ticket placeholder for it
2019:03:26 21:45:06 alexmiller we would like to have a lighter-weight aliasing mechanism (lighter than "creating namespaces") and I expect that could be used in both clj and cljs
2019:03:26 21:45:40 alexmiller hoping to do some work on this in 1.11
2019:03:26 21:46:07 alexmiller I feel your pain :)
2019:03:27 11:09:18 zclj Do anyone know of a library that can produce specs given a json schema?
2019:03:27 12:05:46 ikitommi @zclj haven’t heard, but would have use for that too. there is an issue in spec-tools, but no work going on: https://github.com/metosin/spec-tools/issues/154. Happy to help if someone starts doing that.
2019:03:27 12:31:45 zclj @ikitommi thanks for the info. I will do some experiments on my specific case and if it is generic enough I would be happy to contribute to the issue in spec-tools
2019:03:27 13:05:07 ikitommi Is there a way to force a fdef to validate always the inputs? Schema had the :always-validate metadata for this.
2019:03:27 13:08:48 ikitommi e.g.
(require '[schema.core :as s])
(s/defn ^:always-validate interceptor-x
[opts :- {:string? s/Bool}]
...)
(interceptor-x {})
; Syntax error (ExceptionInfo) compiling at (test.cljc:150:1).
; Input to interceptor-x does not match schema:
;
; [(named {:string? missing-required-key} opts)]
2019:03:27 13:07:44 ikitommi Something like stest/instrument but for non-test use.
2019:03:27 13:11:12 rickmoynihan Not that I know of.
You can use a :pre condition on a function that calls s/valid?, though you’ll not get an s/explain-* style error if you do.
2019:03:27 13:12:32 rickmoynihan If you actually want that though, why can’t you just call
2019:03:27 13:14:09 rickmoynihan I’d probably just call s/valid? and raise an s/explain-data if the check failed explicitly inside the function though if I wanted that — or however you want to report the error.
2019:03:27 13:14:58 ikitommi That would work, but that would not contribute to the function documentation.
2019:03:27 13:15:25 rickmoynihan It would if it was both an fdef and that 🙂
2019:03:27 13:16:18 ikitommi calling clojure.spec.test.alpha from non-test code also doesn’t sound like a good idea.
2019:03:27 13:20:07 ikitommi I’ll write an issue out of that.
2019:03:27 14:48:15 ikitommi I would still like a support functional way of doing that: have a function that takes options and returns the component (that has a spec). And the function args should be validated too.
2019:03:27 14:21:59 drone @ikitommi I’m not sure it addresses your exact issue, but orchestra’s instrument enables checking of :ret and :fn
2019:03:27 14:24:10 drone this has been better for how we use spec; where we enable instrumentation during development to try and get type-checking-like feedback while writing code
2019:03:27 14:29:03 alexmiller @ikitommi it is intentional that this functionality doesn't exist for functions as the goal is to have no overhead for function specs at runtime
2019:03:27 14:29:56 alexmiller but I'll leave it open for now as maybe it would be useful to have an opt-in feature for that
2019:03:27 14:31:21 ikitommi opt-in would be great.
2019:03:27 14:34:50 ikitommi @mrevelle orchestra is great, but it also has the dynamic *instrument-enabled* which can disable the instrumentation.
2019:03:27 23:42:16 jeaye We just use something like the following in our code:
2019:03:28 19:31:37 borkdude @ikitommi If you want to always validate, also in prod, why not use s/valid? for this?
2019:03:28 19:32:06 borkdude
(s/valid? (:args (s/get-spec `my-fn)) args)
something like that
2019:03:28 19:34:34 borkdude or if you don’t like using s/get-spec you can write the args spec separately and use that in fdef, you’ll still get the docstring
2019:03:28 19:39:31 drone we use s/valid? in asserts for things that should never happen and both s/valid? and s/conform in control flow (`if`, condp, match, etc). not advocating to do that, but sharing a related need we felt and what we’re trying out to address it
2019:03:28 19:48:38 borkdude s/assert also works, but we have assertions turned off in prod
2019:03:28 19:48:49 borkdude so s/valid? always works
2019:03:28 19:49:25 borkdude and I bet you can wrap this in a macro like (def-checking-fn name & body)
2019:03:28 20:26:25 ikitommi @borkdude thanks! looks just right. In my case, it's a framework calling the specced functions at runtime, so I can use apply & args to make a generic functional proxy.
2019:03:28 20:32:17 ikitommi ... or just extract the function spec if that exist and validate by the framework. indeed.
2019:03:28 20:29:46 ikitommi that said, tempted to add an s/defn with identical syntax as Schema somewhere. There are several ones (all bit different) out there already. The Lisp Curse?
2019:03:28 20:39:03 drone I think the only public one that’s maintained is orchestra’s defn-spec
2019:03:28 21:05:58 alexmiller fwiw, I think we’re probably going to add something with spec 2 (or maybe Clojure 1.11)
2019:03:28 21:18:31 metametadata I like https://github.com/Provisdom/defn-spec syntax as it's friendly out-of-the-box to Cursive IDE (specs are defined inside defn metadata). but maybe orchestra's macro is also supported by Cursive
2019:03:28 21:49:58 butterguns The other day, I asked a question about speccing Java object fields
https://clojurians.slack.com/archives/C1B1BB2Q3/p1553612761304000
I thought I'd check back in, and say that I got intimate with s/conformer! My solution is to use a conformer to convert the Java object into a clojure map of {<fieldname-keyword> <field-value>}, and then just use s/keys
2019:03:28 21:58:32 alexmiller note that this spec won't work backwards with unform though
2019:03:28 22:00:35 alexmiller in general, this kind of thing is frowned upon as it bakes transformation into the spec and doesn't give registry consumers the option of whether to do the transformation. I would probably recommend to instead actually convert the objects to Clojure data, then use simple data specs on it.
2019:03:28 22:01:49 butterguns So, doing the transformation in code, instead of within a spec?
2019:03:28 22:02:08 alexmiller yeah
2019:03:28 22:02:53 alexmiller conformer was really imagined primarily as a tool for spec op creators, not for api users
2019:03:28 22:03:18 butterguns OK, that makes sense. Thanks for the advice. I'm making progress
2019:03:28 22:02:05 alexmiller well the big thing likely here is to integrate the requires/provides semantics via select, and also to introduce a way to talk about returns in terms of args more descriptively. the :ret/:fn as it stands now is likely to change
2019:03:29 01:20:40 kenny @borkdude @mrevelle I've run into problems using s/valid? in production. If my specs use fspecs, it will throw in prod because no test.check. We work around this by never passing a function to a function that uses s/valid? but seems a bit dangerous. Would be nice to be able to disable fspec generative checking in certain production.
2019:03:29 08:18:00 ikitommi Is there a way to extract the fdef specs, given one has a function at hand? e.g. need to get the symbol out of the function to be able to look up for it’s spec. Functions are quite opaque in clojure (or I’m just doing it wrong)
2019:03:29 08:21:22 ikitommi something like this, but for real (and without eval):
2019:03:29 12:05:12 alexmiller You can demunge the class name
2019:03:29 12:05:36 alexmiller But won’t always work
2019:03:29 12:20:16 alexmiller In general, it’s better if you can start from a symbol than start from a function object
2019:03:29 13:14:42 ikitommi Thanks. In my case, I just don’t have the symbols available, just functions. Knowing the arity of the functions at runtime would be useful too to be able to fail fast in case of invalid arity of the (framework) functions (middleware, interceptors etc). Any change of getting the function symbol & arity queryable for runtime?
2019:03:29 13:47:45 alexmiller function objects are just function objects - the fact that a particular var is referring to them is in some ways incidental.
2019:03:29 13:48:25 alexmiller same for arity, although that one in particular is one that affects code evolution over time (it's common for arities to expand but still be backwards compatible)
2019:03:29 13:48:57 alexmiller I know these seem like obvious things to have on a function, but I think the implications are probably a lot more far reaching and subtle than you expect
2019:03:29 13:50:34 alexmiller there are a lot of good reasons to fight this kind of introspection and instead try to drive from symbols or vars
2019:03:29 13:50:53 alexmiller so, in short, no, not eagerly looking to make these changes
2019:03:29 14:04:55 drone @kenny we don’t use fdef because of this. And I agree, disabling the test generator req would help. I think it was mentioned as possibly coming in spec2
2019:03:29 14:35:48 stathissideris a bit of a philosophical question: I know spec uses qualified keywords, but do people normally go for the :com.my-company.invoice/total style of keywords or the :invoice/total style?
2019:03:29 14:48:33 alexmiller if you are writing a public library, then I think you endeavor to start your qualifier with something you control (by trademark, domain name, etc).
2019:03:29 14:48:52 alexmiller if you are writing a private app, then you should make qualifiers "sufficiently unique"
2019:03:29 14:49:26 alexmiller whether that means including company, department, product, whatever is relevant
2019:03:29 15:22:09 stathissideris @alexmiller thanks, that’s a balanced view which I think makes sense
2019:03:29 18:08:29 dominicm I'm starting to regret using particularly unique keywords that include the company name and sub project as a prefix for an application.
I'm considering adding aliases to human written edn because of it.
2019:03:29 19:10:56 carocad hey guys, does somebody knows what is the url of the api docs for spec-alpha2 ? The link seems to be broken in https://github.com/clojure/spec-alpha2😕
2019:03:29 19:12:27 alexmiller I'm not sure I've ever built it
2019:03:29 19:12:46 alexmiller the doc builder works off of a release and we haven't released a version of it yet
2019:03:29 19:15:16 alexmiller in other words, the link is right, the target is absent :)
2019:03:29 19:15:29 alexmiller I can try to build it but I'm probably not going to have time today
2019:03:29 20:23:30 seancorfield I know what I'll be doing this weekend (while my wife's in China)... 🙂
2019:03:31 14:09:34 eggsyntax Looking good, thanks Alex! Couple of thoughts/questions on that document:
- on first encounter, it seems confusing (and complecting 😉 ) to treat schemas sometimes as though they're specs (`s/def`, same registry) and sometimes not (can't be used for valid? etc). Why not either a) let them act as specs (maybe a schema acts like (s/select my-schema), ie all optional, when treated as a spec?) or b) make them fully separate (separate registry, can't use s/def to register them)?
- What do you mean by 'SPI'? 'Serial-parallel interface'? 🤔
- In the "get-movie-times" example, is there a way to say that address is optional, but if present it must include zip? I know I've encountered use cases like that in practice.
2019:03:31 20:27:50 seancorfield Yes, omit address from the vector but still provide the map for the sub-select
2019:04:01 03:35:27 alexmiller @U077BEWNQ - schemas as specs would be pretty useless since map specs are open (any map would validate) so we decided to make them different. There are some real subtle questions in the idea of wanting to have both for different purposes though and I haven't had a chance to talk through it with Rich yet. might change.
- SPI = service provider interface, for those building new kinds of specs/schemas, vs API which is what consumers use. probably should have explained that more clearly.
2019:03:30 17:28:17 didibus How would you spec the following: [:keyword1 '(:one :or) '(:more :lists) :keyword2 '(:one :or) '(:more :lists)] where each pair of :keywordN '(:one :or) '(:more :lists) is optional?
2019:03:30 17:33:43 didibus Nevermind, it does work. I had a typo.
2019:03:30 17:34:56 didibus I have another question though. It seems that conform evaluates the forms?
2019:03:30 17:35:38 didibus Basically, my above spec is for a macro. And I expect the lists to be code forms. Then I call conform to get it conformed, but it seems to evaluate the forms.
2019:03:30 17:38:06 didibus Ah yes, nevermind again
2019:03:30 21:34:03 seancorfield @alexmiller I'm trying the latest Spec2 and running into a new failure
(s/defop length-limited-string
"Given n return a spec for a string that is less then n characters
long."
[n]
(s/and string? #(>= n (count %))))
;=> #'ws.billing.specs/length-limited-string
(s/def :wsbilling/braintree-id (s/nilable (length-limited-string 64)))
;=> :wsbilling/braintree-id
(s/def :wsbilling/risk-id :wsbilling/braintree-id)
;=> :wsbilling/risk-id
(s/get-spec :wsbilling/braintree-id)
;=> #object[clojure.spec_alpha2.impl$nilable_impl$reify__12249 0x689396e2 "
This worked fine on the last version I tried, and that last s/get-spec call returned a spec, as expected, not nil.
2019:03:30 21:34:50 seancorfield If I change :wsbilling/risk-id to
2019:03:30 21:39:02 seancorfield Unfortunately we have a lot of specs that are defined as aliases of other specs. I happened to spot this one because we have a test that verifies every key in a related spec has a spec defined for it...
2019:03:30 21:40:16 seancorfield Ah, it looks like that spec exercises and conforms just fine, it's only get-spec that "fails" by returning nil.
2019:03:30 21:52:21 seancorfield OK, it looks like we used to call s/spec in these cases with Spec1, but that stopped working in Spec2 with the API split into symbolic specs and spec values, so I changed it to s/get-spec and checked it returned non-`nil`. But what I probably should have done was used s/form instead which seems to do the right thing in Spec2. It still seems surprising that s/get-spec returns nil for the case above tho'...
2019:03:30 22:54:37 alexmiller I know why you’re seeing that
2019:03:30 23:02:06 seancorfield Is it a bug or should I just not be doing it that way?
2019:03:30 23:02:35 seancorfield (I have almost everything passing again on Spec2 -- still trying to track down one failure in one of our newer apps)
2019:03:30 23:07:28 seancorfield OK, everything passes. The only change I had to make was that get-spec issue (in three places).
2019:03:30 23:07:47 seancorfield I'll take a break for a bit and then start looking at schema/`select`...
2019:03:31 01:50:11 alexmiller Bug
2019:03:31 03:55:18 seancorfield Need a JIRA for it @alexmiller?
2019:03:31 09:06:45 ikitommi @danielcompton awesome! It seems to use schema directly, have you though about copying just the needed parts? Takes 1sec to load just the schema.core to the repl and most of it is not needed here, I guess?
2019:03:31 13:01:42 y.khmelevskii @danielcompton as I can see defn-spec is slightly similar to https://github.com/gnl/ghostwheel, right?
2019:03:31 16:45:03 danielcompton @ikitommi yeah I've pretty much copied most of the impl from schema into defn-spec, there was just a little bit left in the macros namespace and I wanted to get a release out over the weekend
2019:03:31 16:45:15 danielcompton But that's on the list to do, shouldn't be much more work there
2019:04:01 07:34:17 tangrammer hi spec people!!
I’m trying to temporarily redefine the spec global registry in the same way as with-redefsdoes …
My intention is to reduce possible valid values, eg: instead of only checking that value is a string, validate that is exactly foo value
so far, this is the code that i’ve reached…
🤔 what do you think about using with-redefs to redefine specs?
2019:04:01 12:09:22 alexmiller Why?
2019:04:01 12:29:16 tangrammer the final idea is to apply authorisation
2019:04:01 12:32:58 tangrammer basically if certain path values of the speced data are “allowed”, the full data could be viewed by a user or not
2019:04:01 12:33:33 tangrammer i’s it more clear now?
2019:04:01 12:38:05 alexmiller Sounds like a nightmare for such a critical role
2019:04:01 12:48:49 tangrammer could you expand a bit why you think it could be a nightmare?
2019:04:01 13:04:14 alexmiller why would you base something as critical as auth on something as fragile as with-redefs?
2019:04:01 13:07:38 tangrammer sorry i didn’t realise about “These temporary changes will be visible in all threads” 😬 … I was trying for something thread isolated … but with-redefs was my quick first attempt
2019:04:01 15:20:26 djtango if you want thread-local rebinding - use binding
2019:04:01 15:21:54 djtango though so far the only things I've found with-redef useful for is orthogonal instrumentation of functions (e.g. timing functions) or stubbing in testing
If it's a nested spec, you could try using :req-un to have (s/keys :req-un [:very-specific/a]) and (s/keys :req-un [:general/a]) alternatively, you could try experimenting (s/or ...)
If you're using namespaced keys on your data, it's hard to understand why the spec for a namespaced entity would change
2019:04:01 21:33:48 tangrammer more than change, the spec should be more specific. so instead of having a predicate string? would be #{"value1" "value2" "value3"}
2019:04:01 21:34:59 tangrammer being these values the returned of a sql execution
2019:04:01 12:42:09 tangrammer the thing (at this level) is getting spec validation messages
2019:04:01 13:05:41 djtango given that specs are typically named with just data can't you dispatch at real-time the spec you might want to validate against
2019:04:01 13:05:55 djtango (with no other knowledge of what you're trying to achieve)
2019:04:01 16:04:03 mathpunk I've got a question about spec design. The situation: I have an entity in mind, which I call a run (for test run). It has two variants: it might be a test run from CI, or it might be a test run from my local machine.
2019:04:01 16:06:36 mathpunk Some data is present in runs of any variant, like results and a revision (the SHA of the codebase during the run). Only CI runs have a pipeline. Both variants have logs but only the local variant has api logs under that key.
2019:04:01 16:08:04 mathpunk My design problem is, I'm trying to use mostly namespaced keywords, so that I can nest specs. For instance,
2019:04:01 16:08:40 mathpunk But then I realized, ooooh when I start modeling runs of the local variant, project.ci.runs/revision is overly specific
2019:04:01 16:09:29 mathpunk Do you have any thoughts on what I should keep in mind while spec'ing these variants out? Thanks!
2019:04:01 16:11:04 mathpunk I guess I should add, I'm trying to be careful about not over-spec'ing things --- this could be an example of, Don't spec this part
2019:04:01 16:18:14 mathpunk (The value-add that I see of spec'ing this data is, I've been doing these runs for months, and iterating on how they're shaped, so I'm interested in validating that I understand what 'type' of run I'm looking at)
2019:04:01 16:19:37 alexmiller a good guiding principle is to just try to spec the truth - what are all the values an attribute can actually take on?
2019:04:01 16:20:01 alexmiller if it's not constrained to the regex, then that's not the truth, it's just one variant
2019:04:01 16:20:20 alexmiller it's always possible to spec generally, then s/and an additional constraint for a particular context
2019:04:01 16:23:01 mathpunk 🤔 So maybe,
:project.run/revision
:
2019:04:01 16:23:54 mathpunk oh wait, you said 's/and', not 'and'... I'll have to think about that a minute
2019:04:01 17:14:21 kenny How do you guys use clojure.test.check.generators/let with Spec? It doesn't appear to be exposed in clojure.spec.gen.alpha.
2019:04:02 01:14:53 taylor I just add the dependency explicitly to the test check namespace
2019:04:02 01:15:21 kenny We don't allow test.check as a runtime dependency.
2019:04:02 01:15:29 taylor Instead of using the aliased stuff from spec
2019:04:02 01:15:43 taylor You can add it as a dev only dependency
2019:04:02 01:16:02 kenny Right but then you can't define your generators next to your specs.
2019:04:02 01:16:18 taylor Also, you’re depending on it already if you’re actually using any of its functionality
2019:04:02 01:17:04 kenny We don't use the functionality in production but want to define the generators next to our specs.
2019:04:02 01:17:19 taylor The stuff in spec are merely aliases to the test check definitions
2019:04:02 01:17:54 kenny True but it does it lazily -- it only requires test.check if you actually invoke a function in the gen namespace.
2019:04:02 01:18:02 taylor I supposed you could use the same strategy
2019:04:02 01:19:08 kenny Yes but that begs the question of why it wasn't included to begin with. It may be because it is a macro.
2019:04:02 10:24:29 gfredericks yes, a macro has to run at compile time
2019:04:02 10:25:03 gfredericks so the normal trick that allows you to not have a runtime dependency on test.check doesn't work
2019:04:02 17:14:42 kenny So the solution is to copy and paste the let macro elsewhere?
2019:04:02 18:58:25 gfredericks You can do that, you could relax your requirement to not have test.check in prod, you could wrap in a simple proxying macro that expands to an exception throwing generator when tc isn't on the classpath
2019:04:02 19:00:52 kenny The problem with including test.check on the cp in prod is certain code paths check input structures via s/valid?. If, by accident, a function is passed in a map of things, it would get generatively checked. That would have a significant impact on performance.
2019:04:02 19:03:12 gfredericks They skip that based on the namespaces being requirable or not?
2019:04:02 19:03:40 kenny Not sure what you mean.
2019:04:02 19:04:12 kenny If test.check isn't available on the cp and a fn is passed to the s/valid? call, an exception will get thrown.
2019:04:02 19:05:00 kenny If it is, no exception is thrown and the performance will slow down silently.
2019:04:02 19:06:44 gfredericks Ah, I see, I didn't notice the "by accident" part
In any case, the other two approaches would work though
2019:04:02 19:07:49 kenny The latter seems like a good approach. Seems like a worthy addition to the Spec gen namespace as well.
2019:04:03 01:11:51 gfredericks I can see that
2019:04:02 17:07:34 zalky Hi all, is there an predicate for whether something is a valid spec? Something like specize (not public api), but returns true or false. I guess I could do:
(defn spec?
[spec]
(try
(do (s/conform spec nil) true)
(catch Exception e
false)))
2019:04:02 17:13:31 zalky @mattmorten, thanks for the response. The issue with that is it returns false things that are valid specs, like functions and keywords that resolve to spec objects.
2019:04:02 17:14:23 butterguns Ah, sorry, misunderstood. No I don't think I know of anything like you ask
2019:04:02 18:13:51 robertfw Are there any suggestions for tracking down "Couldn't satisfy such-that predicate" errors? The exception it throws doesn't appear to give any clues as to which predicate is the problem. I'm wondering if there is a faster option that my current approach of narrowing down my recent changes
2019:04:02 18:16:25 alexmiller unfortunately, not really atm. I will tell you that most of the time it's from an s/and
2019:04:02 18:16:48 alexmiller where the initial gen is too wide for the subsequent filter(s)
2019:04:02 18:17:22 robertfw yeah, I narrowed it down pretty quickly and it was indeed an and. just one of those minor annoyances.
2019:04:02 18:22:16 alexmiller there is some new stuff in the latest test.check but we haven't yet looked at how to make it better to expose the info out through spec
2019:04:02 18:22:32 alexmiller certainly, it's annoying, and I have spent a lot of time doing the same
2019:04:02 23:20:32 didibus Question, How do you s/keys a Java Map? Is that even possible? It fails because it says a Java Map is not a map?
2019:04:02 23:26:00 robertfw My first guess would be to write your own predicate function to validate it yourself, I wouldn't expect s/keys to support it
2019:04:03 01:49:25 alexmiller spec is based on Clojure maps and is not intended to support Java colls
2019:04:03 03:19:49 didibus Ya, I think I was confused, I thought all Java collections were Clojure's, but its the other way around only. And since Java Lists are seqable, anyways. I got around it.
2019:04:03 03:19:55 didibus thanks
2019:04:03 10:22:19 djtango @alexmiller I seem to have some vague memory that you guys were maybe looking at alternatives to generatively testing functions that are argument inputs or map entries, but can't seem to find anything to follow on this subject?
2019:04:03 10:22:42 djtango am I remembering correct? This was prompted by some of the discussion @kenny and @gfredericks were talking about
2019:04:03 15:57:16 alexmiller not sure what you're talking a bout
2019:04:03 15:58:20 alexmiller can you give me some more hints? :)
2019:04:03 16:17:37 kenny I believe he may be referencing the ability to disable generative fspec testing in production.
2019:04:03 17:27:43 djtango ^ this
2019:04:03 17:28:05 djtango but am I imagining that there would have been an alternative way to check if the specs were still correct?
2019:04:03 17:38:42 alexmiller oh, yeah, my current thought on this is to instead of generatively testing the function to wrap the function with something that checks arg specs of the inputs
2019:04:04 09:09:47 djtango Ah nice - is any of the ideas/progress on this happening anywhere public?
2019:04:03 21:28:53 hadils Nvm, I answered my own question…
2019:04:04 11:32:57 orestis Is there a way for spec to express “if this key is present, this other key must also be present”?
2019:04:04 11:33:34 orestis Actually - if this key has this value, this other key must also be present
2019:04:04 11:34:07 orestis I suspect it’s something you can do with multi-spec?
2019:04:04 11:58:27 orestis Got something working multi-spec, but it seems that I have to name the intermediates — like, if I need a timestamp to be present if something is of type A, I have to name a spec “something-with-time”, whereas the normal case is “something-plain”. Or perhaps naming isn’t necessary, and my multi-method can just return an (s/keys [])?
2019:04:04 13:31:24 alexmiller (s/keys) is valid
2019:04:04 13:31:37 alexmiller and will still validate all registered keys
2019:04:04 13:32:20 alexmiller sounds like an excellent use case for schema/select in spec2
2019:04:04 13:43:37 orestis Oh that’s right — I guess what the actual requirement is, is that for some specific subtype of foo, you have to provide “bar”. If you provide “bar” for a different subtype, well, it’s still valid but going to be ignored anyway.
2019:04:04 13:46:38 orestis Looking forward to seeing how schema/select looks like.
2019:04:04 13:52:01 alexmiller you can try it now if you like
2019:04:04 13:58:49 djtango This feels like a bit of an abuse of s/keys but is this what you mean @orestis (s/def ::m (s/keys :req [::a (or ::a (and ::foo ::bar))] :opt [::foo ::bar]))
2019:04:04 13:59:31 alexmiller note that or here is not xor
2019:04:04 14:00:17 djtango indeed - got caught out by that once
2019:04:04 14:00:42 djtango but really was just loose reasoning more than anything
2019:04:04 14:02:40 orestis I thought that there were incompatibilities between current spec and spec2 — that’s why I shied away so far.
2019:04:04 14:03:11 orestis (As in, I’d rather migrate my specs and introduce the new stuff to the team when it’s released)
2019:04:04 14:11:02 orestis BTW today was the first time I shared a spec between frontend and backend. So nice! Thanks a lot to everyone involved 😄
2019:04:04 16:35:29 alexmiller correct - spec and spec2 are not compatible, but you could try it if you like (I wouldn't use spec2 for anything yet)
2019:04:04 17:10:02 seancorfield @orestis We have a branch of our 80K+ line codebase at work running on Spec2 but it's still changing a lot and we've tripped over numerous bugs that have led us to huddle with @alexmiller several times in order to figure out workarounds (and sometimes just waited for the bug to get fixed).
2019:04:04 17:10:29 seancorfield I agree with Alex that it isn't production ready yet but it's definitely interesting to try out.
2019:04:04 17:11:11 seancorfield Our biggest issue was that we had sort of mixed predicates and specs in some places -- that Spec1 allows but Spec2 draws a harder line in the sand around.
2019:04:04 17:12:11 orestis Irrelevant to spec2: Someone mentioned that in 1.10 destructuring is done via a spec, but it seems like destructure is same as before. Is there a spec for what could go into a destructuring binding?
2019:04:04 17:12:53 orestis Thanks for chipping in @seancorfield — I’ll put it down on my list to try out, but the backlog is very big 😄
2019:04:04 17:13:24 seancorfield I know that feeling 🙂
2019:04:04 17:13:53 alexmiller there is a spec for destructuring in core.specs.alpha, used by many other specs
2019:04:04 17:14:11 alexmiller it's not used for destructuring
2019:04:04 17:14:31 seancorfield We've been on the bleeding edge for a long time at World Singles Networks. We regularly take alpha/beta builds to production, and we were early adopters of spec (and very heavy users of it in production). But even for us, Spec2 isn't quite ready yet 🙂
2019:04:04 17:15:06 alexmiller btw, Sean, current spec2 should fix that issue you reported (as a side effect of other changes)
2019:04:04 17:15:28 alexmiller schema/select have been changing a bit this week, but probably not in ways that break anything you've done
2019:04:04 17:15:41 seancorfield (oh @alexmiller meant to mention that we've had 1.10.1-beta1 in production for all processes since 3/26 -- no problems!)
2019:04:04 17:16:30 alexmiller will write up some more in weekly journal
2019:04:04 17:16:39 seancorfield Thanks, Alex. I may go back to our branch tomorrow and update it and see where we are now. I was working on an enhancement over the last few days that was just screaming out for schema/`select` BTW.
2019:04:04 17:18:24 seancorfield Business wanted an extra field added to a database table (for which we have a spec) and that cascaded up through several layers -- and several specs -- to two of our REST APIs (which also have specs). Dealing with optionality across those tiers is something that schema/`select` will really help us with, I think.
2019:04:04 17:18:56 orestis On the destructuring front then — I’d like to write a macro that given a destructuring form as in ::binding-form, and some kind of other specification (a GraphQL input object, specifically), ensures that the symbols used in the destructuring are present and and in the correct place in the other specification. This is a partly an aid to catch typos, partly a cool thing to show off. Would that ::binding-form spec be of any use? Perhaps conform could give something back?
2019:04:04 17:19:46 orestis Or should I “just” hand-roll a validator?
2019:04:04 17:27:37 seancorfield @orestis My understanding is that binding form spec already exists and you can use it directly (we do this in one place in our code -- which led to an interesting workaround when we tried Spec2 since the core specs continue to use Spec1 and we needed to bridge between them!).
2019:04:04 17:28:09 orestis Yeah, TIL: (s/valid? ::cs/binding-form '{:keys [a b c :d]}) ;;=> true
2019:04:04 17:28:25 orestis Never expected that a keyword would be a valid destructuring key.
2019:04:04 17:29:15 seancorfield (I'm shocked that the above workaround even works but...)
2019:04:04 17:34:27 alexmiller we added keyword support a few years ago, specifically so that you could use ::a, ::x/a, etc and get autoresolve functionality
2019:04:04 17:34:54 alexmiller that's somewhat been superseded now by :x/keys support but still there for backwards compatibility
2019:04:04 17:37:20 orestis Gotcha.
2019:04:04 17:39:17 orestis Seems like spec isn’t the tool I’m looking for the job I want to do. I would like something that given something like {foo :bar} and {:keys [a :b]} would give me the keys that will be destructured on the map. Perhaps I could do some hackery by actually calling destructure and looking at the generated bindings — but that seems a recipe for trouble.
2019:04:04 17:42:44 orestis Sorry for the stream of thought about this. I am wondering if this is actually viable or possible, and if I could get some value out of the core specs (apart from seeing that there are a ton of possible use cases that I never knew existed 😄 )
2019:04:04 17:56:35 alexmiller oh, yeah, I don't know of anything that will do this job
2019:04:04 17:57:24 alexmiller unless you're will to eval, in which case you could probably abuse the destructure function
2019:04:04 18:00:07 alexmiller
user=> (->> '[{:keys [a :b]} {foo :bar}] destructure (apply hash-map) keys)
(a map__2953 b)
2019:04:04 18:01:12 alexmiller you will see some gensym cruft (although the cruft is real! those are intermediates that are getting bound)
2019:04:04 18:01:39 orestis This will probably lose the structure for nested restructuring though. Since it will linearize all the bindings.
2019:04:04 18:02:00 alexmiller yes, it resolves to a single let binding
2019:04:04 18:02:49 alexmiller but that is the reality of what's happening. going in reverse seems challenging.
2019:04:04 18:03:40 orestis I thing walking the tree and having a cond based on the core specs to see which kind of restructuring is there and recurring accordingly would make more sense. I will have to maintain the “current” position against an external tree structure anyway, to answer questions such as “is this key valid for this sub tree”
2019:04:04 18:05:19 alexmiller there's a lot of destructuring options so lots to do there
2019:04:04 18:06:45 orestis Hopefully there’s a 80/20 reward (as is usually the case with this kind of linting).
2019:04:04 18:08:13 orestis Or there are dozens of academic paper arguing about whether this is possible 😄
=> this complains that let doesn’t conform to spec. Bummer. I hoped that my macro would run first.
2019:04:04 18:26:27 alexmiller nope :)
2019:04:04 18:26:55 alexmiller due to delayed eval, outer is in control first
2019:04:04 18:28:06 orestis Perhaps I could do a tagged literal?
2019:04:04 18:36:07 alexmiller tagged literals are read as objects at read time, can't see that's going to help you in expansion
2019:04:04 18:36:43 alexmiller you could make your own let variant
2019:04:04 18:38:07 orestis Was hoping that I could replace the tagged literal with a valid let binding form at read time, then the let macro wouldn’t know the difference (if I understand the compilation phases correctly)
2019:04:04 18:43:24 orestis Too much for a Thursday evening — gotta run! Thanks for your time.
2019:04:04 18:57:51 alexmiller might actually work to some limited degree (I imagine some editors and context would run into issues with this)
2019:04:04 19:15:48 orestis Hm, I was doing something wrong. Oh well, thanks for clarifying. Doing it as a tagged literal seems weird but I don’t want to write a separate let macro (perhaps in practice it won’t be an issue). Thanks again :)
2019:04:04 19:18:15 bronsa what, that should not work
2019:04:04 19:18:47 bronsa @alexmiller was that from the same repl you set! earlier? I think it was only working because of that
2019:04:04 20:37:45 alexmiller ah, it was
2019:04:06 02:08:56 gfredericks set! in that context, used as a past tense verb, is pronounced "set-bung"
2019:04:06 12:54:52 ikitommi Is there a way to construct the selection in s/select programmatically? e.g. have the selection somewhere as data (in db, edn, etc) and make a select out of it at runtime.
2019:04:06 12:55:17 ikitommi same for s/schema…
2019:04:06 12:59:20 ikitommi also, could the selection syntax allow something special that effects the whole keyset, not present an individual key. Could be just a function or a special Protocol.
2019:04:06 13:07:59 ikitommi … if that special keyset-validator would be implemented with a protocol, it would be great to have a function which takes both the spec and the value as arguments, so one could easily write something like ::closed-keyset: the function could extract the list of defined keys from the spec argument and use that to verify that the value doesn’t have any extra keys. And if the validation could return the ::s/problems instead of just ::s/invalid, it would be easy to write those, e.g. no need to write separate explain too.
2019:04:06 13:40:51 metametadata +1, would be nice to have something like that. because currently relying on forms, macros or low-level *-impl helpers is quite cumbersome (e.g. see my attempt at implementing merge-able closed keys spec with user-friendly explain here: https://groups.google.com/d/msg/clojure/duY3ojPwPYo/Wgvk9PsaCAAJ)
2019:04:06 14:36:12 ikitommi I too have almost mergeable closed specs with Spec1, but next hurdle is to get deep-mergeable specs...
2019:04:06 17:40:22 alexmiller You have two routes to programmable specs in spec 2
2019:04:06 17:40:40 alexmiller You can make forms, and call spec*
2019:04:06 17:41:03 alexmiller Or you can make your own syntax that wraps something with defop
2019:04:06 17:41:30 alexmiller Or actually third option is making your own spec instance from scratch but that’s a much heavier lift
2019:04:06 17:42:39 alexmiller We are evaluating support for preds in specs - tbd
2019:04:06 17:43:11 alexmiller But we are working on another kind of checking for closed specs
2019:04:06 17:43:46 alexmiller Sorry, gotta run
2019:04:07 03:57:14 didibus > But we are working on another kind of checking for closed specs
That's really good news. I've had to implement closed keys spec in spec 1 to deal with the following three common case of defects:
1. Keys are mistyped, wrong qualified ns or name, like thinking it is id instead of identifier. Or just plain typo.
2. Keys are misplaced. Like putting the address on the user when it actually goes in the nested profile map. Address on profile being optional, open specs don't catch this bug.
3. Combinations are mistaken. Like say a map which has a query and insert key, this is supposed to be interpreted by some engine, but it is now ambiguous if the operation is supposed to be to query or to insert.
2019:04:07 04:01:51 didibus I don't favour the use of closed specs, because they can cause unecessary breakage. And I'm not saying closed specs is the best solution to these above issues. But my team has faced these often, leading to real production bugs. And I haven't found a better solution to them yet.
2019:04:07 04:04:04 didibus The only better idea we had was to seperate the spec of the producer from the one of the consumer. Specifically, code that looks to an input map to read from doesn't need to validate the input as being closed. As all the above bugs are on the producing code. That way, consumers are not broken by producer accretion that are backward compatible.
2019:04:07 04:04:30 didibus But producers would validate their output maps using Closed spec. To ensure they didn't make any of the above mistakes.
2019:04:07 04:05:29 didibus We never implemented it, because with spec1, it would be too tedious to keep seperate producer/consumer specs.
2019:04:07 05:37:34 orestis Yeah, catching typos is a great use of closed specs. I was attempting going at it from a usage point of view, warning if you are trying to a lookup a key that isn’t guaranteed to be there.
2019:04:07 05:37:51 orestis A spec-aware linter would be great.
2019:04:07 06:56:58 ikitommi @orestis maybe you could work with @borkdude on clj-kondo for that? Would be awesome.
2019:04:07 06:57:49 orestis Yeah, I was actually looking into this before. A tricky thing is that you probably need runtime info to get to the spec registry
2019:04:07 06:58:34 orestis You could perhaps just collect s/keys calls statically for a good first approximation though.
2019:04:07 07:00:11 ikitommi @didibus there is https://github.com/bhauman/spell-spec for spec1 just for that. Sadly, it also macros so can't program/compose with those.
2019:04:07 07:17:36 ikitommi @alexmiller thanks for the info. My use case is this:
reitit components can declare partial specs for the route data they are mounted to, e.g. a parameter-coercion-interceptor says that the route data can have :parameters key, with a map with keys :query, :path etc. for a given route. All the partial specs of the components mounted to a route will be deep-merged for the final effective spec. e.g. interceptor requiring data of form {:parameters {:path ...}} and another with {:parameters {:multipart ...}} should be merged to require effectively {:parameters {:path …, :multipart …}}. And there should be an way to validate that merged spec as “closed” to fail-fast on typos.
I have just that working using data-specs + spell-spec + spec-tools coercion and will demo that soon, but I would want the users of the library not to require anything else than the core.spec to define the component specs.
With spec2, I would like to support the selection / schema syntax as data so that one could say that “my component requires [{:parameters [{:query ::parameter-map}]}] and the other [{:parameters [{:multipart ::parameter-map}]}] and I could just merge those + spec to close them (with a small helper) into [{:parameters [{:query ::parameter-map} {:multipart ::parameter-map} ::closed-spec]} ::closed-spec] kinda thing, that could be used to check the strict configuration.
2019:04:07 07:20:24 ikitommi also, noticed that the nested non-qualified keys don’t work yet, is that a bug or a feature? [{:a int?}] seems to work, but [{:a [{:b int?}]}] doesn’t.
2019:04:07 11:11:07 alexmiller It works if you wrap the inner in s/schema (or register the inner with a name)
2019:04:07 07:27:42 borkdude @orestis > You could perhaps just collect s/keys calls statically for a good first approximation though.
That’s a nice idea. I’ll consider that for clj-kondo.
2019:04:07 07:29:27 ikitommi with top-level merging & spec1
2019:04:07 07:58:29 didibus That would be neat yes
2019:04:07 07:59:18 didibus One thing I was thinking also is to make a custom validate function which can be called with an option to be closed or open.
2019:04:07 08:00:19 didibus That way I could use the same spec, but have the producer code validate it assuming no added keys can be present
2019:04:07 11:12:07 alexmiller That’s exactly the direction we’re going
2019:04:07 11:12:38 alexmiller It’s in how you check, not in the spec
2019:04:07 11:38:47 orestis That makes a ton of sense!
2019:04:07 15:26:50 gfredericks literally friday at work somebody was debugging some code that walks & transforms a plumatic/schema to add openness everywhere, with exactly the same motivation
2019:04:07 19:00:10 quoll I’m pushing up against leaving for my daughter’s swimming training, so I figure I should ask in here…
I’m using an ns form with metadata, in the same way that clojure.core does. i.e.
(ns ^{:doc "some docs" :author "me"} my-ns)
2019:04:07 19:00:38 quoll However, this fails on Clojure 1.10, due to “Extra input”
2019:04:07 19:01:07 gfredericks works fine for me if I paste that in a repl
2019:04:07 19:01:13 quoll sure, I can just go back to a standard docstring and drop the :author metadata, but given that clojure.core uses this I thought I’d question it
2019:04:07 19:01:18 quoll yup, same here
2019:04:07 19:01:28 quoll but leiningen hates to build it 🙂
2019:04:07 19:02:30 quoll let me put together something minimal
2019:04:07 19:02:43 gfredericks yeah, I just tried it in leiningen and it worked fine
2019:04:07 19:03:26 gfredericks half of the time when I'm debugging something, trying to get a minimal reproduction also exposes the real problem
2019:04:07 19:09:50 quoll OK, it’s looking like I had a typo elsewhere that got through the compiler before spec was introduced 😳
2019:04:11 14:48:21 vemv Is there something like (describe-spec :my/spec) that will print what :my/spec is about? Maybe recursively.
Loosely analog to macroexpand/`macroexpand-all`
2019:04:11 15:02:41 vemv (for now I use Emacs niceties but I'm interested in editor-agnostic tooling)
2019:04:11 15:19:52 alexmiller doc works on registered specs
2019:04:11 15:20:18 alexmiller it is, however, not deep/recursive
2019:04:11 15:20:41 alexmiller (doc :my/spec)
2019:04:11 16:38:01 fedreg Hi all, can someone help me with spec’ing a higher-order fn? One of the arguments to a fdef is a higher order fn that I’ve defined as:
(s/def ::foo-fn (s/fspec :args any? :ret ::foo))
This works except running something like test.check on the fn that takes foo-fn as an arg is super slow. …If I just pass ::foo in instead of ::foo-fn everything generates as quickly as usual so I’m guessing I’m doing something wrong… 😞 . Any thoughts? thx!!
…oh… and ::foo has no args but not sure how to express that.. :args nil doesn’t work
2019:04:11 17:06:24 alexmiller :args (s/cat) is prob best
2019:04:11 17:07:15 alexmiller fspecs are instrumented by generating args and invoking the passed function a bunch of times
2019:04:11 17:07:30 alexmiller this is sometimes surprising and/or bad for people
2019:04:11 17:07:52 alexmiller an alternate option to using an s/fspec here is to just spec it as ifn?
2019:04:11 17:10:48 fedreg @alexmiller Will try those out! Thanks!!
2019:04:11 17:18:21 fedreg …Using :args (s/cat) instead of any? definitely sped things up.. Can’t use ifn? for my specific use case but still generating / running through my system about 5 results per second which is pretty fast. …just spoiled by how fast test.check usually runs data!
2019:04:11 17:34:13 alexmiller any? can generate very large nested colls
2019:04:11 17:35:21 alexmiller (s/cat) can only generate one thing, ()
2019:04:11 17:43:42 fedreg yes.. I knew that any was slowing things down but couldn’t figure out how to do no args… Didn’t think to try (s/cat) so thanks for that!
2019:04:12 17:06:06 fedreg Can anyone point me to an example of passing custom generators into the options of spec.test.alpha/check?
:gen map from spec names to generator overrides
…I need to limit the range of some generated ints and can’t seem to get it. thx!
2019:04:15 06:23:27 holyjak Given a spec, how can I generate random value(s)? Does spec.gen/generate take a spec or is there another fn? Thank you!
2019:04:15 07:46:40 jaihindhreddy clojure.spec.alpha/gen takes a spec and returns a generator for it.
clojure.spec.gen.alpha/sample takes a generator and gives you a few examples. Takes an optional number of examples to generate
clojure.spec.alpha/exercise takes a spec and gives you a seq of pairs, each containing generated example value, and the same value conformed.
Check out their docstrings.
Also, it's highly recommended to read the Spec guide in its entirety (https://clojure.org/guides/spec)
It clears a lot of these things up (including potential gotchas like you need to include test.check for some of the functionality to work)
2019:04:15 15:43:53 jaihindhreddy @holyjak ^
2019:04:16 14:22:31 borkdude can macro spec checking (for core macros) be disabled from the REPL instead of with a Java property?
2019:04:16 14:32:40 alexmiller not currently. such a thing could be done by just setting the clojure.lang.RT/CHECK_SPECS flag, but that's not public currently
2019:04:16 16:40:50 jumar Is there any reason why the clojure.spec.skip-macros isn't re-loaded dynamically in Compiler every time it's needed? Performance?
2019:04:16 14:33:02 alexmiller you could remove those specs from the registry
2019:04:16 14:33:21 alexmiller s/def with nil will remove
2019:04:16 14:34:19 borkdude I tried (clojure.spec.alpha/def clojure.core/for nil) already, but that didn’t do it
2019:04:16 14:34:49 borkdude oh it does work, sorry
2019:04:16 14:35:59 borkdude I was testing if for supported beginning with a let but it doesn’t 🙂
(for [:let [x 1] y (range x)] y)
2019:04:16 14:36:27 borkdude and I wanted to make sure I wasn’t getting an error from the spec. thanks
2019:04:16 21:18:11 hiredman I dunno if nit picking on spec-alpha2 is useful feedback yet, but https://github.com/clojure/spec-alpha2/blob/master/src/main/clojure/clojure/spec_alpha2.clj#L51-L54 is checking if an object implements the interface behind the Schema protocol instead of checking for satisfying the Schema protocol (the namespace imports the interface behind the Spec and the Schema protocols, but is careful to use the Spec protocol everywhere and not the interface)
2019:04:16 21:47:34 alexmiller Yeah, that’s actually intentional for perf, same in spec 1
2019:04:16 21:47:51 alexmiller Well same for spec?
2019:04:16 21:48:01 alexmiller May change
2019:04:16 21:52:17 hiredman ah, I see, I missed the use of the interface there. and it looks like spec? will return true if you implement the interface, or if you have metadata with the conform* function(satisfy the protocol via metadata), but will return false if you extend the protocol to some type, which seems idiosyncratic
2019:04:16 21:55:31 hiredman I guess I'll stay tuned
2019:04:16 22:05:36 alexmiller Yeah
2019:04:16 22:05:56 alexmiller In any case, I’m aware :)
2019:04:19 16:58:30 lilactown I have a multi-spec for different query types to a service. I also want to dispatch on the query type to actually query the service
2019:04:19 16:58:55 lilactown what's the best way to consolidate these two things?
and then I need to create a wrapper function so I can fdef the implementation, right?
2019:04:19 18:12:27 misha afaik spec does not work for multimethods, but you can declare implementation function(s), fdef it/them, and make your multimethod call it/them
2019:04:19 21:39:21 lilactown it looks like multi-specs s/keys check all available keys, even when not specified?
2019:04:19 21:39:29 lilactown actually not sure if this is multi-specs or s/keys doing this
2019:04:19 21:40:51 alexmiller s/keys does this
2019:04:19 21:41:00 lilactown hrm
2019:04:19 21:41:02 seancorfield If you have a qualified key that matches a spec, it will be checked, whether it's specified in s/keys or not.
2019:04:19 21:41:18 alexmiller this is documented in the s/keys docstring
2019:04:19 21:41:39 lilactown gotcha - it's just not what I want right now 😂
2019:04:19 21:56:52 dominicm @alexmiller just read your inside clojure. The api seems very global for closed specs. Have you given much thought to how it might be scoped?
2019:04:19 21:58:08 alexmiller Scope in what dimension?
2019:04:19 21:58:31 alexmiller It is very global, like instrumentation
2019:04:19 21:59:26 dominicm Maybe with a dynamic binding, so thread local.
2019:04:19 22:00:23 dominicm Thinking of a web server with many concurrent requests where you want the closed spec on the boundary, but internally within processing the open spec may be used (as other sources are merged).
2019:04:19 22:01:45 borkdude Interesting choice to turn off and on “closedness” for specs. Would it make sense to combine this with select somehow?
Like (s/def ::closed-spec (s/select ::open-spec ...))?
2019:04:19 22:02:31 alexmiller It should work with select too, I just haven’t done that yet. There’s just a lot of copied code there at the moment
2019:04:19 22:03:14 borkdude no, I mean, turn an open spec into a closed one using select, not by a global flag
2019:04:19 22:03:38 borkdude (then select would have to support that)
2019:04:19 22:03:43 alexmiller then, no :)
2019:04:19 22:04:16 alexmiller Closeness is intentionally not part of the spec language and won’t be
2019:04:19 22:04:23 alexmiller It’s part of the api
2019:04:19 22:04:52 alexmiller @dominicm still thinking about this aspect
2019:04:19 22:05:12 borkdude but when you would like to use a spec in different ways, you would have to toggle this flag all the time?
2019:04:19 22:05:35 alexmiller That’s the aspect we’re think about still
2019:04:19 22:06:18 borkdude I guess you can just make a copy of the spec, like (s/def ::foo ::bar), but if you close bar, will foo also be closed?
2019:04:19 22:07:31 alexmiller Well that’s not a copy, it’s an alias
2019:04:19 22:07:38 borkdude yeah
2019:04:19 22:07:39 dominicm @borkdude feels like working around the intentional limitation to make closed&open copies.
2019:04:19 22:07:47 alexmiller The answer is tricky, and may change
2019:04:19 22:09:03 dominicm @alexmiller btw, why the global on/off switch? My reaction was that I'm going to see someone turning it on globally unconditionally in some project and removing the upside of open specs.
2019:04:19 22:10:34 alexmiller Do you just mean the no-arity version of close/open?
2019:04:19 22:23:53 dominicm I do, yeah
2019:04:19 22:11:25 borkdude is s/valid? part of the spec language? why not make it an option to those functions?
2019:04:19 22:11:56 alexmiller s/valid? is api, not spec language
2019:04:19 22:12:18 borkdude yeah ok. I think having it as an option is more flexible than setting it globally
2019:04:19 22:12:36 alexmiller Still might do that. Breaks all existing APIs and specs though
2019:04:19 22:12:50 borkdude extra arity doesn’t break?
2019:04:19 22:13:01 alexmiller It will break the protocol
2019:04:19 22:14:37 alexmiller We’ve also been thinking about a fast s/valid? path. Right now that’s tunneled through the conform* protocol, which does a lot of useless work if just validating. So that could also be a flag, or a separate protocol method
2019:04:19 22:14:59 borkdude (s/valid2? …)😉
2019:04:19 22:15:21 alexmiller Those could all be flags internally on conform*
2019:04:19 22:15:40 alexmiller We are already in a new namespace, no need to number
2019:04:19 22:16:08 alexmiller But ideally could minimize breakage for some of those spec libs
2019:04:19 22:17:01 alexmiller Anyhow, what I’m interested in hearing now is how you would use it
2019:04:19 22:17:16 alexmiller We can work out the best answers from that
2019:04:19 22:17:24 borkdude a fast s/valid? path would certainly help things like core specs which quickly become noticable in performance
2019:04:19 22:18:07 alexmiller It’s not going to be like 10x or anything
2019:04:19 22:18:47 alexmiller It really matters when you are constructing conformed outputs
2019:04:19 22:18:56 borkdude I think the closedness aspect is something where people choose different tools or use workarounds for spec right now at API level things, like maybe Reitit and yada (which now uses Schema), but I’ll let those maintainers chime in
2019:04:19 22:22:26 alexmiller Like how often are you likely to use the same spec in both open and closed check modes on the same spec? And where different, is it per usage or per thread or per what?
2019:04:19 22:27:43 dominicm @alexmiller I can imagine closed for the incoming boundary, open internally when merging additional internal data, and potentially closed again when sending it on to another consumer (e.g. A service which accepts too much or a database)
2019:04:19 22:29:01 borkdude that’s exactly how I would use it too and probably in combination with select
2019:04:19 22:30:18 dominicm I'm not the maintainer of yada, but I'm influential there. For incoming web requests, I'm more interested in:
- not checking extra keys
- not seeing extra keys
2019:04:19 22:31:07 dominicm I don't want to throw if there's extra keys, probably, just silently drop them.
2019:04:19 22:33:02 dominicm Sorta, but recursive :)
2019:04:19 22:57:29 seancorfield My first reaction is: ugh, no! Mutable global state on spec behavior just feels horribly wrong. I could have a spec defined in one place and just looking at that and the uses of it no longer tells me whether extra keys are permitted or not, because any random code, anywhere in the system can toggle that setting :face_vomiting::exploding_head:😠
2019:04:19 22:58:32 seancorfield It seems like the absolute worst of all worlds, to be honest.
2019:04:20 00:27:31 drone so, we use records and have a macro that generates a spec which includes checking fields as well as asserting the object is an instance of the backing record class (the latter part can also be used in isolation as a fast check using instance?). because of this, I didn’t realize keys generates open specs
2019:04:20 00:30:02 drone ah, partial nm. forgot about :req and :req-un, which we use behind the scenes
2019:04:20 00:30:26 drone what’s the use case for a spec that describes keys but permits maps that have none of those keys?
2019:04:20 00:32:31 drone and agreed with above that mutating closedness seems like a bad idea
2019:04:20 00:55:21 seancorfield @mrevelle One observation about records is that they are open too -- you can assoc in any additional keys you want and it remains an instance that record type.
2019:04:20 00:56:18 drone yeah, and that makes sense. but my expectation is that a record (or keys spec) provides a base set of keys that should be present
2019:04:20 00:56:33 seancorfield My view of :opt in s/keys is that it mostly exists for generative testing (and partly for documentation).
2019:04:20 01:11:15 seancorfield The schema/`select` stuff is going to be very helpful for us -- it will simplify our API specs quite a bit. Instead of needing to have different specs for otherwise very similar API calls, we can have a schema for the "family" of parameters that can be passed in and then use select to specify what a particular API requires.
2019:04:20 01:12:10 seancorfield I'm still thinking about how it can help with the specs around our database, but I think it can help with the required-for-insert vs required-for-update scenarios.
2019:04:20 01:17:35 drone yeah, sub-spec selection seems like a good idea. I think it’s just that the undefined default behavior is to not include any of the keys/fields
2019:04:20 03:58:56 seancorfield @mrevelle This is not valid because ::f's value is not string? per your spec for ::f. That makes perfect sense to me.
2019:04:20 03:59:32 seancorfield s/keys works that way today.
2019:04:20 04:01:02 drone Oh, I don’t disagree. Was pointing out that a unselected spec field becomes active when it matches a key even though it’s not selected.
2019:04:20 04:01:56 seancorfield Yes, which makes sense in the context of the way Spec has always worked.
2019:04:20 04:02:38 seancorfield If you added a spec for ::x as string? then (s/valid? ::s {::x 10}) would fail -- because 10 is not string?.
2019:04:20 04:03:29 seancorfield That's how s/keys works today. That's how s/schema and s/select work in Spec2: if you provide additional keys, they're still checked if there's a spec for them.
2019:04:20 04:06:24 drone again, I understand and am not disagreeing with you. But you pointed out that the docs for spec2 say use of a schema without select is undefined and I’m providing examples of how saying something is undefined doesn’t mean there isn’t a default
2019:04:20 04:07:14 drone and for my use case, it’s a poor default
2019:04:20 04:07:45 seancorfield I didn't say it was undefined (as in "undefined behavior"). I quoted the part that says schema says nothing about required-ness -- that's what select is for -- and that decoupling is exactly what Rich was talking about in Maybe Not.
2019:04:20 04:09:22 seancorfield What I do think is odd, right now, is that you can select keys that are not in the .
2019:04:20 04:10:17 drone from the docs:
> Schemas do not define “required” or “optional”
2019:04:20 04:11:11 drone and that is weird
2019:04:20 04:11:24 seancorfield I disagree. It's exactly what Rich talked about.
2019:04:20 04:11:40 seancorfield This, on the other hand, seems wrong:
2019:04:20 04:12:13 seancorfield (there's no spec for ::x but the s/select call makes it required, even tho' it isn't part of ::s)
2019:04:20 04:12:52 drone I guess my point is, based on Rich’s talk and the current spec2 docs, (s/valid? ::s x) shouldn’t be permitted since the schema is being used without a select
2019:04:20 04:13:20 seancorfield In the first cut of schema/`select` that landed in the repo, schemas were not specs, so that was not allowed.
2019:04:20 04:13:53 seancorfield I'm not sure why the change was made to turn schemas into specs.
2019:04:20 04:17:29 drone I’m mostly just bummed that spec seems to be going away from being a more-flexible structural typing substitute and into something that I’m not sure fits at all with how I use Clojure
2019:04:20 04:18:00 drone thanks for the link
2019:04:20 04:22:33 seancorfield I'm not sure I follow you... select is much more powerful than s/keys -- and now you can define the overall shape of your data once, and then validate against the relevant parts of it as needed, without having to write multiple s/keys specs that all overlap.
2019:04:20 04:24:59 seancorfield Reading over the wiki page again in more detail, I wonder if the drive for that change (the commit above) came from wanting to treat schemas like specs because of data generation?
2019:04:20 04:26:55 seancorfield There's an example of generating against a schema, which produces random selections of keys from the set in the schema. Which would all be "valid" in any context that accepted that schema without placing any required-ness constraints on the contents. Seems like a slim driver to me tho'...
2019:04:20 04:32:25 drone that’s true, it’s not that something can’t be done. but it’s unclear how compatible it is with other use cases. don’t want to end up working against the language
2019:04:20 04:33:17 drone and I think you’re probably right about generation as motivation
2019:04:20 17:40:53 seancorfield Alex responded to my question on that commit: "The big thing that tipped it is the unqualified key support which does enforce the specs of unqualified keys (in addition to checking the values of any qualified keys against the registry)." -- because the unqualified key schemas include the spec and don't exist outside the schema (so the schema is a useful spec mostly for unqualified keys which would not otherwise be checkable).
2019:04:20 01:18:25 drone but I didn’t select ::f from ::s…
2019:04:20 01:24:06 ikitommi @alexmiller About the closed specs. I agree on @dominicm on the open/closed on the borders. We want to drop out extra keys at the borders (recursively) and also coerce the values. Inside they can be open. Spec-tools (and by so, reitit) does both stripping extra keys & coercion already, but using the s/form parsing, which feels wrong. There is CLJ-2251 I wrote about the different ways to walk the Specs from inside, not outside. Also the fast path of just validating.
2019:04:20 01:25:54 drone i.e., based on what Rich presented and my lurking on here, I’d expect select to be used to choose a subset of keys when the full record isn’t expected or needed. having it default to none seems silly when it’s much easier to write (s/select ::s []) when none are required than enumerate all the fields in a spec when you want all to be required. I also think it’s less surprising, but that’s subjective
2019:04:20 07:25:47 seancorfield > when it’s much easier to write (s/select ::s []) when none are required than enumerate all the fields in a spec when you want all to be required
FYI (s/select ::s [*]) is the "all fields required" case
2019:04:20 15:47:32 drone cool, that’s a bit better
2019:04:20 01:34:15 ikitommi I would put the closed-checking into select: Schemas would be open by default, but one can define select default to open or closed maps, which can be explicitly defined to different (sub)selects too.
(s/select ::user [::id ::name ::address {::address [::street]}] [s/closed]) ;;3rd arg will be merged into all submaps
2019:04:20 01:46:42 wilkerlucio maybe moving to the select get back on the closed problem again, it will always be closed, I agree with @borkdude in this one, maybe the point to make this decision is s/valid?, this way you keep the possibility to stay open, but can do closed checks when it makes sense
2019:04:20 01:53:47 ikitommi with s/valid? is would be all or nothing. In s/select, one could close it just partially. Not sure if there is a real life case for that thou.
2019:04:24 05:48:24 didibus It seems you register which spec are to be validated as closed and open. So the s/valid could be partially closed as well, if you broke it up into named specs.
2019:04:20 03:24:39 eggsyntax Independently wrote a response to the open/closed approach which pretty much echoes the same things other folks are saying above — global-stateful-ness bothers me, what about threads, etc. The need for closedness seems to me like it’s usually context-specific and scope-limited, so I was expecting a limited-scope approach. My first inclination would be to be able to do something like
(s/valid? (s/closed-variant ::bar) my-bar)
but I understand you’re trying to make sure it doesn’t become part of the spec language.
If it’s about limiting it to the API, what about a separate s/closedly-valid? (or strictly-valid? or strictly-correct? or whatever)? And maybe an equivalent for s/explain. Or I like what I think @wilkerlucio is suggesting, making it available as an extra-arity arg to s/valid? (off by default so it doesn’t break existing code).
2019:04:20 03:56:21 seancorfield @eggsyntax My first thought was s/valid-closed? but then you need a variant of each of the explain* functions and conform and probably several others... which gets ugly fast...
2019:04:20 20:42:52 eggsyntax What about maybe a separate s/closed? function that just verifies whether the data structure contains only the keys in the spec? It doesn’t seem like a big burden to do
(and (s/closed? ::spec x) (s/valid? ::spec x))
2019:04:20 07:29:36 metametadata If closed/open status becomes an option to some s/valid* fn then I don't see how the closed spec can be specified to validate args in defn-spec or something like that. So "closedness" should be attached to the spec instance.
And I guess I'd also intuitively expect (s/close-specs ::s) to return a new instance of the spec. Even though I don't yet have cases where I need to open or close the already created spec, I'd expect it to return new values instead of "mutating" existing ones. Otherwise, there's a risk to have all the old problems with mutable data like threading, reasoning about code, etc.
2019:04:20 17:41:49 seancorfield @somedude314 We use Spec for that. We use it for API parameter validation, server-side form validation, and several other places in our production code.
2019:04:20 17:42:58 adam @seancorfield do you use any helper libraries built on top the spec or just vanilla spec?
2019:04:20 17:44:00 seancorfield Just vanilla spec. We wrote some code on top of explain-data to turn Spec failures into domain-specific error messages but that's it.
2019:04:20 17:45:31 seancorfield And we mostly do it like this:
2019:04:20 17:47:10 adam Cool thanks. I will give it a try. This is the first time I am trying to validate stuff in Clojure so still unsure what is practical and what is not.
2019:04:20 22:47:01 ikitommi Did a demo in Clojure/North about closed specs, with spec.alpha, spec-tools, spell-spec and expound, using the new reitit error formatter: https://vimeo.com/331602826. The code calls spec-tools.spell/closed-spec function on a spec and gets a closed (spell-)spec back.
2019:04:23 20:45:02 robertfw I have a question about how people handle writing specs for an API endpoint. I'm speccing out ring request maps, and have several endpoints in the same namespace. a ring spec typically involves providing specs for :status, :params, sometimes :body. However my spec names end up being quite wordy; for example, the expected body for a successful response of a given endpoint ends up being :my.ns.endpoint-name.response.success/body. Over the course of defining several endpoints, with potentially multiple possible responses, I end up with quite a lot of typing. I've been considering either moving each endpoint spec into its own namespace, so that I can use :: shorthand, but I'm not wild about the proliferation of files that will create. My other thought is to create some macro or function to help generate these specs. Any thoughts?
2019:04:23 21:35:39 seancorfield @robertfrederickwarner You can create a namespace alias without needing an actual namespace file.
2019:04:23 21:53:32 robertfw this looks to work nicely 🙂 thanks
2019:04:24 05:23:22 didibus Wow didn't know that
2019:04:23 21:36:27 seancorfield
(alias 'm (create-ns 'ws.domain.member))
and then use ::m/whatever
2019:04:23 21:37:34 robertfw ah, I did not know about alias. I had considered trying using (ns ...) but rejected it as it seemed to be pretty unidiomatic
2019:04:23 21:39:11 seancorfield @alexmiller has mentioned the possibility of improving alias usage but I don't think any concrete proposal has been made yet.
2019:04:23 21:39:21 robertfw It would be nice if it were possible to define a map spec assigning specs to given keys; I know that seems to be going against the grain of spec but it would cut down some lines of code needed to spec out json endpoints
2019:04:23 21:40:14 robertfw e.g... we get data coming in as :some_value whereas in our system it's :some-value, so there's another spec to handle the conversion
2019:04:23 21:40:35 robertfw I've been digging around to see if anyone has written helpers for that but no luck
2019:04:23 21:43:26 alexmiller spec2 has support for this for unqualified keys in s/schema
2019:04:23 21:45:39 robertfw Is there a rough idea of when spec2 might be "ready"?
2019:04:23 21:46:26 seancorfield Define "ready" 🙂 We have a branch of our codebase at work that runs on Spec2.
2019:04:23 21:48:53 robertfw good question - I guess, active development finished and moving into a phase of broader community use, similar to where spec.alpha is now
2019:04:23 21:52:24 alexmiller I would say it's likely on order of months given the set of work we have talked about
2019:04:23 21:53:21 seancorfield Could you give us an overview of what additional work on Spec2 is still in the pipeline?
2019:04:23 21:53:38 alexmiller hard to say whether that's 2 months or 6 but certainly I would love have it before the conj
2019:04:23 21:54:16 robertfw that's what i suspected. thanks 🙂
2019:04:23 21:58:29 alexmiller we have a list of 10 areas of work for spec2
2019:04:23 22:00:19 alexmiller some of those we'd like to work through before "release" and re-integration, some could be add-ons later
2019:04:23 22:01:41 seancorfield I'd like to cut our project at work over to Spec2 but I don't know how much rework to expect (I mean, I'm used to some level of rework given how often we rely on Clojure alphas 🙂 )
2019:04:23 22:03:36 alexmiller you mean rework between now and "the end" ?
2019:04:23 22:05:16 seancorfield Between now and non-alpha status.
2019:04:23 22:05:38 alexmiller well, I wouldn't recommend that yet but I'm not expecting that the existing apis are going to change much
2019:04:23 22:05:47 seancorfield Although we're heavily reliant on Spec1 Alpha in production so I don't know why I'm all that worried...
2019:04:23 22:11:12 eggsyntax @alexmiller don't know if you already saw it, but there are a few more ideas above on s/closed.
2019:04:25 18:33:21 pyr Hi
2019:04:25 18:33:45 pyr now that usage of spec is pervasive and given the fact that defrecord produces handy functions
2019:04:25 18:35:00 pyr would it make sense to add another one which would create a predicate
2019:04:25 18:35:21 pyr to test for protocol satisfaction?
2019:04:25 18:38:32 borkdude you mean like (instance? Foo (->Foo))?
2019:04:25 18:38:35 drone where MyRecord? is a predicate returning true if its argument is an instance of the backing MyRecord class and/or it has the required record fields with values that match some field specs?
2019:04:25 18:39:41 pyr @mrevelle: yes
2019:04:25 18:40:22 pyr the former
2019:04:25 18:40:49 pyr meh, now that I say it out loud, it makes more sense for protocols
2019:04:25 18:40:51 pyr nevermind
2019:04:25 18:43:00 drone I think there’s value in spec’ing fields of records (just as there is in spec’ing plain-old maps). as for protocols there are some technical reasons they aren’t supported. the workaround that was thrown around in the past is having protocol method implementations call a spec’d function
2019:04:25 18:45:40 pyr ok
2019:04:25 18:48:10 drone re: records. it sounds like most people don’t use them (?) and stick with maps and occasionally add some :tag or :type entry to achieve the ability to determine the “kind” of map
2019:04:25 18:51:15 drone not sure if records were too JVM-centric or if they feel too restrictive to the free-spirited majority opposed to types
2019:04:25 18:52:27 borkdude weren’t they a little bit better for performance?
2019:04:25 18:53:09 borkdude also in Clojure Applied it is recommended for “domain” objects, although I rarely see them used
2019:04:25 18:53:15 pyr I like the fact that you can directly attach protocols
2019:04:25 18:54:24 drone yeah, you get fast single dispatch (like a Java class, since they’re implemented as Java classes)
2019:04:25 18:54:39 alexmiller fwiw, I would probably de-emphasize records in a 2nd ed of Clojure Applied
2019:04:25 18:56:49 pyr extend-via-metadata will help with that
2019:04:25 18:57:37 drone I use them all over in machine learning, data analysis, and program analysis work
2019:04:25 18:57:39 pyr @alexmiller so fewer records but protocols still emphasized, correct?
2019:04:25 18:58:21 pyr spec and qualified keywords make maps more attractive indeed
2019:04:25 18:58:41 pyr breaking out of the protocol + record-implementation will be hard
2019:04:25 18:58:51 pyr especially in our component heavy codebase
2019:04:25 18:58:59 drone seems like there’s plenty of boiler plate that packaging things as records could manage for you
2019:04:25 19:00:17 drone defrecord-spec where each field has an associated spec. spec’d record creation functions, reference the field specs elsewhere when performing a subset select with select
2019:04:25 19:01:17 drone the only place I see them being awkward is when you’re building up the fields for a record. which I believe is an example Rich referenced in Maybe Not or another recent-ish talk
2019:04:25 19:02:29 drone but, it’s not that bad to just use maps with a sub-select spec until you’ve gathered all the required fields and then throw it into a record
2019:04:25 19:04:57 drone feels like serialization is motivating much of this. e.g., you may have some fields that are handy but can be computed from other fields in the map. so when you serialize maps for storage in a database, you drop those derived fields. but then results from the database are “incomplete” and processing is required to generate the derived fields before you could construct the record
2019:04:25 19:08:15 drone datomic killed records, is what I’m saying (with tongue in cheek)
2019:04:27 10:55:45 kommen playing around with spec-alpha2, I was wondering if there is a way to make s/conform of an s/select only return keys of a map which are actually specified in the select.
2019:04:27 16:16:30 alexmiller It’s not working this way at the moment but select over a closed schema should throw invalid on that case
2019:04:27 16:16:57 alexmiller The close stuff is going to get completely overhauled though so current api will change
2019:04:29 14:02:55 dominicm I was curious about this, could you share what the current thinking is? Even with the disclaimer of it being heavy WIP?
2019:04:29 14:05:06 alexmiller rather than the stateful api (which was intended to be similar to instrument), will be more op-specific, and actually more generic than just "closing", will open up some api space for other kinds of checking variants
2019:04:29 14:05:38 alexmiller so, new options on valid?, conform, assert, etc - anything with checking
2019:04:29 14:11:09 dominicm Sounds really interesting. Looking forward to seeing how much leverage this provides.
2019:04:30 08:20:40 ikitommi checking variants like runtime transformations / coercion? or just validating? anyway, generic sounds great. :+1:
2019:04:30 11:41:26 alexmiller Not transformations
2019:05:01 15:03:01 metametadata > so, new options on valid?, conform, assert, etc - anything with checking
it looks like it will create friction for adopting closed maps in defn-spec and other libs which don't know about new options in the methods and rely on the fact that every spec itself describes everything needed for validation
2019:05:01 15:05:09 alexmiller that's something defn-spec will have to reckon with. function specs themselves are going to go through significant changes and making an integrated defn+function spec syntax is in scope.
2019:05:01 15:14:42 metametadata thanks.
fwiw, this is the typical way I add specs to protocol methods using defn-spec at the moment (`sp` here contains custom spec helpers):
2019:05:01 15:16:12 dominicm I guess there's a open question of whether the function level is the right place for deciding closedness. Maybe the other flags will clarify this.
2019:05:01 15:19:53 metametadata yes, it's interesting to see how function specs will play with closed maps.
and speaking of that, this is how I use closed maps in functions currently:
2019:05:01 15:25:56 dominicm I'm suggesting that maybe it's up to the caller sometimes whether to apply certain constraints.
2019:05:01 15:32:31 alexmiller these flags will probably be available on calls like instrument or check so you could supply those constraints that way
2019:05:01 15:37:33 metametadata would be cool if it allowed to spec a function in such a way that some args are closed and some are not, e.g. (foo <closed-map> <integer> <open-map>)
2019:05:01 15:41:38 dominicm That sounds like an important filter.
2019:05:01 16:58:47 alexmiller the planned api will allow that
2019:05:01 16:59:42 alexmiller well, on a per-spec basis that is. if you want to use the same spec in both modes in a single context, not sure about that (or whether that's even a real use case)
2019:05:01 17:28:40 dominicm Hmm, merging an old-user with new-user where the new user is coming over the wire and needs to be locked down, but old-user is from the db?
2019:04:30 08:25:03 ikitommi Bending spec1 to support things that we need for validating configuration, and that hopefully are available with spec2 itself: closing (data-)specs recursively. No macros, just data.
2019:04:30 08:26:20 ikitommi uses spell-spec and expound for most of the work.
2019:04:30 10:26:39 vemv
(spec/def a int?)
(spec/valid? `a 2) ;; -> true
Sometimes I make this typo (`a` vs. ::a) but it happens to work
Is it intentional?
2019:04:30 11:54:38 alexmiller It’s a side effect of the impl
2019:04:30 11:55:15 alexmiller Symbols are used as valid names for function specs (s/fdef)
Is it possible to define a keys spec such that the :foo/bar key is required, but the :baz/quux spec is validated for that specific key?
i.e. I want to override a spec without renaming the key
(rationale: :foo/bar is a Datomic attribute. I cannot or don't want to rename it. And :foo/bar has a existing spec, useful/correct 99% of the times)
2019:05:01 16:40:03 valerauko not that i know of, no
2019:05:01 16:42:35 vemv Working around that with a fn for now 🙂
2019:05:01 20:29:02 eskemojoe007 I have a hash-map where the keys and values are used almost as a linked list {"a" "b", "b" "c", "c" "a"} where a,b,c could be other strings. This is used in a map where I have a spec, but I don't know how to write a spec that says, let the keys and values be any ol' string.
2019:05:01 20:38:02 eskemojoe007 Well that was easy...
2019:05:01 20:39:33 eskemojoe007 One more question for now. I have a unique ID for many items. They are used throughout the speced map. When using generators, it doesn't understand the relationships between the IDs. Any way to enforce that?
2019:05:01 20:41:03 seancorfield @david.folkner You can either (s/and (s/map-of ...) uniqueness-predicate) or write your own generator. The former is "easy" if it's able to actually satisfy the criteria. The latter is harder but more likely to work.
2019:05:01 20:42:31 eskemojoe007 Perfect. I'll take a look at making a generator, that will likely fit my application a bit better. I already have the functions to build up the data for my actual application, so a custom generator can leverage those.
2019:05:01 21:13:09 alexmiller for the latter, see gen/fmap and/or gen/bind
2019:05:03 01:36:32 Nolan just trying out alpha2 for the first time and finding (s/or ...) to be StackOverflowErroring on everything i try. seems like im missing something pretty big but havent been able to piece it together—do the regex macros need to be used with s/spec*?
2019:05:03 01:43:53 Nolan ah, seems like its nrepl related…
2019:05:03 01:47:58 Nolan seems to have resolved itself, what a baffling experience indeed.
2019:05:03 02:20:52 Nolan is there a concise way of expressing something similar to s/or where data that conforms to multiple predicates returns all of the matches instead of just the first? something similar to:
2019:05:03 02:31:04 Nolan good to know. thank you alex!
2019:05:03 12:54:58 jumar I'm not sure how to approach this problem...
I have multiple implementations of an "auth provider" (e.g. db, ldap, etc.) and they are represented as a namespaced map like this:
The problem here is the :config key. It's widely different across different implementations and I'd like to have different specs for that.
My problem is that AFAIK you can have only one spec for such a key - in my case that's fairly generic:
Now I want to "override" spec for the :auth-provider/config key for each implemantation - currently I use this hacky approach:
(s/def ::ldap-config (s/keys :req-un [::host ::port ::connection-timeout ::response-timeout]
:opt-un [::bind-dn-format ::search-base]))
(s/def ::ldap-provider-spec (s/and
::auth-specs/auth-provider-spec
;; this is less descriptive than using `s/keys` directly
;; but we cannot use `s/keys` because `:auth-provider/config` default
;; spec is already registered in auth-specs namespace
;; and you cannot have two different specs for the same namespaced key (i.e. `:auth-provider/config`)
#(s/valid? ::ldap-config (:auth-provider/config %))))
But that's not only awkward it's also problematic from the "error reporting" point of view - in case something inside the :config key is invalid I get pretty useless error message that the spec failed but I don't know why:
2019:05:03 12:56:28 jumar I tried merge and multi-spec (might not know how it should be used properly) but failed basically for the same reason - cannot define different specs for the key :auth-provider/config
2019:05:03 13:03:03 manutter51 Wonder if you could do something with the namespaces on the key? Like, instead of :auth-provider/config for all the variants, have :auth-provider-ldap/config, :auth-provider-db/config, etc.
2019:05:03 13:06:29 jumar That's not an option I'm afraid.
2019:05:03 13:04:36 alexmiller You’re in the ballpark of s/multi-spec - have you tried that?
2019:05:03 13:07:17 jumar As commented in the reply, I got stuck basically for the same reason (not able to define different specs for the same namespaced-key) - I might be doing it wrong, though.
2019:05:03 13:10:43 jumar roughly something like this:
2019:05:03 13:11:33 jumar Now I don't know how to define different versions of :auth-provider/config...
2019:05:03 13:20:25 alexmiller one way would be to do this one level up - the map containing :auth-provider/config
2019:05:03 13:21:16 alexmiller or I find it's always helpful to come back to the truth of the matter - what values can a key take on? here :auth-provider/config has multiple sets of things
2019:05:03 13:21:32 alexmiller so spec it as s/or of different s/keys specs
2019:05:03 13:21:56 alexmiller (some of which might reuse the same attributes)
2019:05:03 13:26:20 jumar idea with s/or is interestig - I'll try that. Thanks!
2019:05:03 18:33:29 denik it looks like unform is not aware of nonconforming or am I doing sth wrong?
(s/def ::bar
(s/or :baz integer?))
(s/def ::foo
(s/nonconforming ; <========
(s/keys :req-un [::bar])))
(s/def ::tez
(s/tuple any? ::foo))
(s/conform ::tez [123 {:bar 4}])
; [123 {:bar 4}]
(s/unform ::tez [123 {:bar 4}])
; Error: nth not supported on this type function Number() { [native code] }
; at Function.
2019:05:03 22:21:48 alexmiller those both roundtrip fine for me w/o error
2019:05:03 22:22:02 alexmiller are you on cljs? or clj?
2019:05:06 17:29:53 denik cljs
2019:05:06 17:42:50 denik just tested and can't reproduce in CLJ.
2019:05:06 19:53:41 yuhan Is it good practice to place all specs together in a dedicated my-project.specs namespace?
2019:05:06 20:05:58 holyjak I do not know but on my project I created a separate .domain ns with specs so that I could share them with multiple namespaces and it is nice not to have it all through the core ns. So it works for me 🙂
2019:05:06 20:33:16 yuhan Thanks! I was reading a few posts and it seems there isn't really a one "correct" way of doing things
2019:05:06 20:34:48 yuhan sharing specs between namespaces is the reason I'm thinking of doing it as well, and it probably makes sense for s/fdef specs to remain next to their respective functions
2019:05:07 18:26:24 butterguns I personally put specs above the code, in the same namespace. It makes sense that logically, if I want to see the "Event" spec, it would be in the event.clj namespace for example
2019:05:07 18:28:35 butterguns I also have a common.clj namespace, which contains shared functions. It also contains common specs, like ::timestamp, which is used in many places
2019:05:10 15:52:24 rickmoynihan I’d say it can make sense for libraries at a certain scale… though even there you may just want to put them where you need them.
For an application you’ll inevitably end up wanting to spread things out.
2019:05:06 19:58:34 yuhan I also avoid using namespace alias ::keyword syntax and most of the specs have short 3-8 character prefixes that don't actually correspond to any of my project namespaces, wondering if that's good practice too
2019:05:06 20:06:25 holyjak They don't need to correspond to actual namespaces, the main thing is I guess readability.
2019:05:08 19:55:42 favila what's the best way to spec that some relationship holds between the keys and values of a map?
2019:05:08 19:56:52 favila in my case I have a map whose keys describe some range, and the vals are vecs of items which should be in the range described by the key
2019:05:08 20:02:53 favila generator for that seems easier to make
2019:05:09 11:07:33 ben this is probably a really simple question, but what are the practical differences between using :pre and :post vs spec/fdef for function validation?
As far as I can tell, fdef gives you a bit more control, but otherwise :pre/`:post` are just there because they’re more “lightweight”/easier to read?
2019:05:09 13:31:46 eskemojoe007 I'm a noob with this stuff, but I think :pre and :post run at runtime. fdef doesn't run at runtime, only during testing.
2019:05:09 13:53:30 ben this rings true with the docs now I read them back, actually
2019:05:09 16:47:14 jumar And yes, it's not usually recommended for production use, only for testing/development
2019:05:10 07:08:27 thumbnail :pre, and :post can be toggled by rebinding *assert* as well though
2019:05:10 17:41:34 eskemojoe007 I'm trying to make a custom generator for the deck of cards example in the spec guide. I have a spec that looks like (s/def ::player (s/keys :req [::name ::score ::hand ::scored])). Each key already has the proper tested generator, but for ::player It shouldn't allow duplicate cards in the hand and scored category for example. I need to make a custom generator for ::player that checks that forces them to be unique.
2019:05:10 17:42:16 eskemojoe007 I guess I didn't ask a question. Question1: Is it possible to make a custom generator for just part of the ::player?
2019:05:10 17:45:09 seancorfield I'm not sure I'm reading your problem statement correctly: are you just trying to ensure ::hand generates unique cards, and ::scored generates unique cards, or are you also trying to ensure there's no duplication between ::hand and ::scored?
2019:05:10 17:45:40 seancorfield If it's the former, you can just write custom generators for those specs. If it's the latter, you pretty much have to write a generator for the whole ::player spec.
2019:05:10 17:47:52 eskemojoe007 Its the later. Given a typical deck of 52 cards. Scored is cards that have scored, then are out of play (Making go-fish).
2019:05:10 17:48:35 seancorfield Then, yeah, you have no choice really -- you need a generator for ::player
2019:05:10 17:48:53 eskemojoe007 So I have to make a custom generator for the whole ::player and I can't rely on the generator even for the other keys such as ::score?
2019:05:10 17:49:29 seancorfield Your ::player generator will run the generators for the individual elements if it wants default behavior.
2019:05:10 17:50:16 eskemojoe007 What does that syntax look like?
2019:05:10 17:51:53 seancorfield (s/gen ::my-spec) gets you a generator for a spec
2019:05:10 17:52:09 eskemojoe007 Ahh...perfect!
2019:05:10 17:52:25 eskemojoe007 That makes sense, just didn't realize thats what it was doing.
2019:05:10 17:52:32 eskemojoe007 Thank you.
2019:05:10 17:54:08 seancorfield And (gen/hash-map ...) to generate the ::player hash map from those generators.
2019:05:10 17:54:23 seancorfield (assuming gen is clojure.spec.gen.alpha)
2019:05:10 17:58:54 eskemojoe007 So a very redundant generator would look something like this:
2019:05:10 17:59:18 seancorfield Yup, that would get you started.
2019:05:10 17:59:42 eskemojoe007 Perfect. I'll post my solution just for reference, but I appreciate the help.
2019:05:10 18:00:37 seancorfield I'd probably have a spec for a deck of cards with a custom generator to ensure they're all distinct (it could just shuffle an ordered generation of all the cards), then ::scored would be the first random N of those and ::hand would be the next random M of them.
2019:05:10 18:04:15 eskemojoe007 The ::scored spec generator is weird, it takes exactly 4 of the same rank card to score. So I was going to run that first, then ommit any results from the ::hand based on that.
2019:05:10 18:05:12 eskemojoe007 But taht seems like a terrible idea.
2019:05:10 18:05:19 eskemojoe007 I'm gonna make a full deck spec.
2019:05:13 17:55:44 eskemojoe007 Finally got back to this. I came up with something that looks like
2019:05:13 17:56:29 eskemojoe007 So it generates the scored cards first, uses gen/bind to get those, and bases the the other generation based on that (I didn't include all the functions).
2019:05:11 16:48:56 denik I'm using s/conform to give a richer shape (e.g. places -> names / sequential -> associative) to data flowing through my program. However, after I conformed a value, it trickles through my program and is updated. The crux is that I don't have a spec for the conformed value, which would be useful. Since conform is doing the reshaping work, it must be possible to generate that spec. Currently, I'm hacking this behavior with s/unform and then s/conform
2019:05:11 19:11:44 alexmiller that's why you shouldn't use conform to transform your data
2019:05:11 19:12:15 alexmiller conform is designed to tell you how it parsed, not to serve as a general purpose transformation engine
2019:05:12 02:35:39 denik Is anyone aware of a library that is designed to transform data based on a spec or similar? cc @alexmiller
2019:05:12 15:46:30 denik @alexmiller @seancorfield @ikitommi to be clear, I'm not looking to coerce data, rather I want to to give it a richer shape (e.g. places -> names / sequential -> associative). I have a concise place-based syntax like [:user/add [:post 5 :authors] {:id 10 :user/name "Eve"}] that I want to turn into
This would be easy in vanilla clojure but there are some gotchas, for example path is optional. With other edge cases writing that in vanilla clojure would get ugly fast. Spec's s/? + conform (or similar) seem like a perfect fit. So if it's not conform, I'm wondering if there is a straightforward approach to parse and unparse while leveraging specs?
2019:05:12 15:49:24 stathissideris @denik s/conform would work for this, but I’ve seen people advising against using it
2019:05:12 17:38:19 alexmiller cond-> is great for optional stuff
2019:05:12 17:38:31 alexmiller I’d just use core stuff
2019:05:12 18:53:37 denik @alexmiller not for parsing
2019:05:12 19:31:02 alexmiller there are libs for data structure parsing like https://github.com/cgrand/seqexp that do the "parsing" of data
2019:05:13 15:33:28 denik thanks! unfortunately clj only
2019:05:12 20:18:06 ikitommi Puzzled. I would have said that s/conform would have been a perfect tool (without any add-onns) for parsing just that example. What are the downsides of using spec for parsing?
2019:05:12 21:59:25 alexmiller conform is fine for parsing
2019:05:12 22:00:01 alexmiller Request above is for additional transformation and enhancement of the result
2019:05:13 15:31:39 denik conforming/parsing + generation of a spec for the parsed result (comformed shape) so that it can be checked in other function invocations as well as fspecs.
I would find this very useful:
1. sparse data comes in from a request, e.g. [:user/add [:post 5 :authors] {:id 10 :user/name "Eve"}]
2. conform data to use names instead of indexes/places=>
3. use a derived spec of the conformed value from (2) to check validity of the value as it is updated throughout the program
2019:05:13 12:02:08 ben I have a bunch of specs that are quite repetitive to write out manually. Is there an easy way to define spec (e.g. w/in the current ns) with a function?
2019:05:13 12:10:05 ben Essentially I have a map that looks something like:
{:event :some-kw
:metadata {:a 1 :b 2}}
where the spec of :metadata depends on the value of :event, which I want to check with spec
2019:05:13 12:13:13 ben So I think I could do something like:
(s/def :event1/event ...)
(s/def :event2/event ...)
;; and so on
(s/def :event1/metadata ...)
(s/def :event2/metadata ...)
;; and so on
(s/def :event1/message (s/keys :req-un [:event1/event :event1/metadata]))
(s/def :event2/message (s/keys :req-un [:event2/event :event2/metadata]))
;; etc
(s/def ::message (s/or :event1/message :event2/message ...))
But this seems extremely inelegant. Feels like I’m missing something obvious but I’m not sure what
2019:05:13 12:26:07 codonnell Sounds like a good use case for a multispec.
2019:05:13 12:36:42 alexmiller Yes, also repetitive code can be made less repetitive with a macro
2019:05:13 12:59:13 ben > One common occurrence in Clojure is to use maps as tagged entities and a special field that indicates the “type” of the map where type indicates a potentially open set of types, often with shared attributes across the types.
yes it does 🙂 thank you, @codonnell
2019:05:13 23:06:28 yogthos I guess dropping down to clj 1.9.0 works
2019:05:13 23:08:22 alexmiller The linked issue in the comments has some patches people have been using.
2019:05:13 23:08:47 alexmiller Patches to Clojure that is
2019:05:13 23:09:07 alexmiller I’m not sure which of the many approaches on there is really the best
2019:05:13 23:09:30 alexmiller But we will definitely take a look at it in 1.11
2019:05:13 23:17:05 yogthos thanks, and 1.9 seems to work so I'll just stick with that for the time being
2019:05:14 12:57:55 rickmoynihan Any ideas on how to spec this?
Essentially I have a heterogenous map; if a key in that map is a vector of length N then I’d like to spec that the value of that key in the map is a sequence of tuples also of length N.
If a key in the map is not a vector, than its key can be any?
2019:05:14 13:02:50 alexmiller spec it as a collection of kv tuples
2019:05:14 20:03:58 WhittlesJr When defining your fspec's :fn, what's the intended way to interact with args that are s/or specs? For example:
{:fn #(-> % :args :arg-name) for such an arg yields a vector that starts with the keyword for the s/or variant. Here's how I could approach it:
Here, :fn ensures that the first arg is a key in the second arg if the second arg is a map. Is this how I should be writing these :fns, or is there a better way?
2019:05:14 21:21:22 alexmiller if it's just an inter-arg dependency, you can s/& that predicate onto the args spec
2019:05:14 21:21:53 alexmiller rather than using :fn, which has access to both args and ret
2019:05:14 21:22:32 alexmiller alternately, you could encode the args with an s/alt for your two alternatives
2019:05:14 21:26:25 alexmiller something like that for the args if I understood all that right
2019:05:14 21:27:16 alexmiller and as an aside, when something is hard to spec like this with options, it's often a good sign that your function is doing two things and having 2 functions might be better
2019:05:15 14:02:51 WhittlesJr Ahh that looks better! And thank you for the advice, I'll think about whether I can split this up or not.
2019:05:15 21:13:03 lellis Hi all! There is some lib that i can create an ER model based in my spec definition?
2019:05:16 07:47:01 jaihindhreddy Take a look at Hodur. Tries to DRY the essence of the domain model. You define one central model from which you generate the models for each thing you use (search engine, database, message queues etc.)
https://github.com/luchiniatwork/hodur-engine
2019:05:16 07:47:54 jaihindhreddy I'm sure you can extend hodur to make it generate an ER model too. Hodur's possibly too heavy conceptually for your specific use case.
2019:05:16 13:38:10 lellis Tks!
2019:05:16 20:47:25 robertfw I'm trying to create an fdef :args spec for a multi-arity function in the general form of
(defn my-func ([x] (my-func x nil)) ([x y] (do-thing x y)))
I have a handful more args than just x & y, and don't want to repeat myself, so my first thought was to do something like
(def base-args [:x ::my-x-spec]) (s/fdef my-func :args (s/alt :without-y (apply s/cat base-args) :with-y (apply s/cat (conj base-args :y ::my-y-spec))) - but alas, s/cat is a macro so I can't do that. I've had another peruse of the spec docs but nothing jumped out at me. any suggestions?
2019:05:16 20:49:39 robertfw Can I make an (s/cat base-args) and then append onto it somehow?
2019:05:16 20:53:45 robertfw I understand that I can just provide a full list of [:x ::my-x :y ::my-y], but I'd like to be able to generate examples of each arity
2019:05:16 20:58:02 alexmiller Just build a single s/cat with nested optional additional args with s/?
2019:05:16 20:59:07 alexmiller Or you could build each arity as an s/cat that combined the prior with an additional arg
2019:05:16 21:27:57 robertfw Thanks, I'll look at those options
2019:05:17 10:07:02 vemv Is there a performance difference between running essentially the same spec checking via instrumentation, vs. via :pre?
2019:05:17 19:28:02 andy.fingerhut For a general spec, I have no idea. For a very specific very simple spec of checking whether some clojure.set functions are given arguments return true for the predicate set? or not, there is a pretty big performance difference I measured, with measurements given starting in about the second screenful of the README for this project: https://github.com/jafingerhut/funjible
2019:05:18 18:17:05 vemv nice one!
But my question was focused on instrumentation vs :pre.
Guess I can try it myself though
2019:05:17 11:10:25 kszabo Am I right to assume that until spec2 gets finished the way to go to emulate s/schema behavior is to only define (s/keys :opt [:schema/values]) kind of specs and use fdefs with use-case specific :req specs?
2019:05:17 11:11:09 kszabo that way those can be later transformed in s/select’s
2019:05:17 12:23:19 alexmiller Yeah
2019:05:18 05:03:22 dottedmag I'm trying to write a dissector of a complex protocol for Wireshark. I have started by writing it manually in C, but it seems to be a very tedious task. I'm going to consume this protocol from Clojure, so I was thinking about specifying it using spec, and then generating C code from it. Has anyone seen anything similar?
2019:05:18 05:04:03 dottedmag (C can be replaced with Python or Lua — does not matter much in this context)
2019:05:18 05:04:31 seancorfield I bet if you ask that during Pacific daylight hours, my colleague @hiredman would have a fair bit to say. He's a big fan of generating stuff from spec and similar data-based systems.
2019:05:18 05:49:28 dottedmag Is there a way to have "switch-by-value" without describing all variants in one gigantic (s/alt)? I'm dealing with a protocol that has a command ID byte that describes format of what is following, and then another subcommand ID in some commands etc. I don't want to have (s/alt :0x00 ::0x00-command :0x01 ::0x01-command ...) — maybe I can register them separately?
2019:05:18 05:49:34 dottedmag Something like a multi-spec for sequences.
2019:05:18 18:15:08 y.khmelevskii Hi! Does anybody know when spec2 for clojurescript will be available? Also, just curious, was spec2 release date announced?
2019:05:18 18:45:24 alexmiller Still months to go probably
2019:05:18 18:45:27 alexmiller No date
2019:05:18 18:54:54 y.khmelevskii got it, thank you
2019:05:18 19:26:34 alexmiller Although at some point maybe we’ll decide to lock it and just make additive changes after that so who knows
2019:05:18 20:02:53 y.khmelevskii It would be great to deploy cljs.spec-alpha2 in the same state as clojure.spec-alpha2 now. During development not stable clojure.spec-alpha2 is ok but I need cljs.spec-alpha2 as well for sharing new specs between frontend and backend
2019:05:18 20:19:31 alexmiller afaik no one has actually started working on a port of the changes in spec-alpha2 to cljs yet. The implementation changes are fairly large (and still under way) so I'm not sure it makes sense to start that work until we feel the internals have started to stabilize, and I wouldn't say that yet.
2019:05:18 20:21:46 y.khmelevskii understand, thanks for info! I will look forward to it
2019:05:19 20:31:39 holyjak how do you do generative testing with spec? I now use
(defspec xyz
100
(prop/for-all [v (s/gen ::my-spec), res (myfn v)]
(do (s/assert ::my-result res)
(my-check-some-property-on v res))))
but shouldn't s/exercise-fn be usable for this? Though, contrary to s/gen it doesn't allow me to specify overrides and thus test more special / infrequent cases?
clojure
(ns ic.test-util
(:require [clojure.spec.test.alpha :as stest]
[clojure.test :refer :all]
[expound.alpha :as expound]))
(defn check
"Passes sym to stest/check with a :max-size option of 3 (i.e. generated sequences will have no more than 3 elements,
returning true if the test passes or the explained error if not"
[sym]
(let [check-result (stest/check sym {:clojure.spec.test.check/opts {:max-size 3}})
result (-> check-result
first ;; stest/check accepts a variable number of syms, this does not
:clojure.spec.test.check/ret
:result)]
(when-not (true? result)
(expound/explain-results check-result))
result))
2019:05:20 10:37:31 conan then my tests look like this:
2019:05:20 10:38:30 conan this will either pass, or give an expound-formatted error showing where the :args, :ret and :fn specs went wrong for the generative tests run for my function (in this case, inst->local-date-time)
2019:05:20 11:54:31 borkdude that function also checks if the sequence of results is not empty
2019:05:20 16:23:59 boyanb Does anybody have experience they would be willing to share in regard of human readable messages at API boundaries via spec? We've looked at expound(which doesn't fit) and phrase(which could do the job, but I am personally not convinced by the predicate focused approach). It feels that a simpler solution focused around explain-data and a message registry(similar to the one found in expound) would produce a better result. Has anybody implemented/is currently using spec for this purpose?
2019:05:20 16:27:08 jeroenvandijk @boyanb Can you share how you feel expound is not a fit?
2019:05:20 16:30:14 boyanb It's not really designed with the idea of message formatting for "users". Expound could very easily be enhanced or parts/ideas of it lifted into a library that could fit it. AFAIK, while looking at the code, there were several places where we needed paramtrization/additional control that is currently not available by the public facade of the library,
2019:05:20 16:30:24 boyanb (had to do with custom printers and expected outputs)
2019:05:20 16:32:00 boyanb I could probably dig in a little further and find the concrete examples. Are you using it with user facing messages in any way?
2019:05:20 16:32:06 jeroenvandijk I've used expound like this
(defn validate-data [spec data]
(if (s/valid? spec data)
:ok
(if-let [explain-data (s/explain-data spec data)]
(let [expound-state
(try
(expound/expound spec data)
:ok
(catch Exception e
(println "Expound had difficulties using " explain-data)
:error))]
(throw (ex-info (if (= expound-state :ok)
"Spec error (see stdout)"
"Spec error (see explain data)") {:explain explain-data})))
(throw (ex-info "Specs are in a weird state, as we can't explain why data is invalid" {})))))
2019:05:20 16:32:17 jeroenvandijk But I agree it is not perfect
2019:05:20 16:34:13 jeroenvandijk I've actually used this in a clojurescript environment (using with-out-str). Worked pretty will when you do a validation on every key change
2019:05:20 16:34:22 boyanb Yes
2019:05:20 16:34:25 kenny @boyanb We had a similar problem. We wanted to spec our API using Clojure Spec but we didn't want to expose Spec's error messages to our users (most people are not used to seeing error messages in that format). This immediately meant Expound was out of the question (too similar to Spec). I looked at Phrase but it seemed like you'd need to duplicate some code to get error messages. I didn't really like that. The overall approach Phrase took made sense -- take the output of explain-data, match it, and convert it to error messages. The matching part seemed like a perfect fit for core.match. So I took that direction and wrote https://github.com/Provisdom/user-explain. I didn't have time to write docs for it 😞 IMO the library is much more general than Phrase due to all the features of core.match. It's also pretty simple -- only 75 LOC 🙂
2019:05:20 21:06:20 ericstewart thank you for sharing this! Going to take a look as I have been on the same path as you and you are further ahead it seems.
2019:05:20 21:07:52 kenny Of course! LMK if it works out for you.
2019:05:20 16:34:28 boyanb our implementation was with-out-str exactly.
2019:05:20 16:35:30 boyanb (when with expound). In the end, we still didn't have the formatting we wanted. Main point is, underlying users are really not familiar with anything clojure and shouldn't care about implemenmtation and in the end we needed fine grained control to explain-data to be able to format path within spec + spec message as we needed.
2019:05:20 16:36:11 boyanb @kenny - what you describe matches exactly our needs. I'll take a look.
2019:05:20 16:39:43 kenny The tests may be helpful for documentation. I'd recommend just reading the 75 line implementation though. There's a lot of areas I want to improve. As you start diving into the data produced by explain-data, you realize that there's a ton of "core" predicates that need to be handled. A common case is #(contains? % kw) which is used with s/keys to validate keywords exist on the map. I'd like to provide a way get automatic nice error messages for all the "core" predicates.
2019:05:20 16:40:04 boyanb Oooh, that's great. @kenny. Looking at the tests it's almost exactly what we are looking for. I'll play with it and let you know if we decide to ship.
2019:05:20 16:40:17 boyanb Yes, the tests are where I started ;o)
2019:05:20 16:49:11 kenny There's a couple weird things about the implementation that I really need to write down before I forget:
1. Since defexplainer does not have a name associated with it, the only way to uniquely identify "explainers" is via the matching map. If you change the matching map (i.e. add or remove keys), the old matcher will still be def'ed. You'll need to run clear! to reset everything. This is really annoying and I don't have a great solution atm. The most obvious solution would be to name every defexplainer.
2. Order of defexplainers does matter. If you def the most general explainer first, it will always get matched first. Ensure more specific matches are def'ed before general ones. Technically this could be fixed by sorting based on some sort of heuristic but I didn't have the time to work out what that should be.
2019:05:20 16:50:38 boyanb Thank you
2019:05:20 16:51:40 kenny Sure. LMK if it ends up working out or if you guys take another approach.
2019:05:20 16:51:50 boyanb Will do
2019:05:21 14:25:37 holyjak does (spec.test/instrument) (w/o args) instrument 1. all functions (ie vars) in all namespaces, mine, libs, and clojure? or 2. just the current one? thanks!
2019:05:21 14:25:37 holyjak does (spec.test/instrument) (w/o args) instrument 1. all functions (ie vars) in all namespaces, mine, libs, and clojure? or 2. just the current one? thanks!
2019:05:21 16:18:26 holyjak OK I guess I can run it to get the answer ,thx
2019:05:21 16:28:48 seancorfield It will instrument all the s/fdefs that have been loaded.
2019:05:21 16:29:48 seancorfield As an example, clojure.java.jdbc has all its s/fdefs in a separate namespace so the library's functions would only be instrumented by (s/instrument) if you have required the clojure.java.jdbc.specs namespace.
2019:05:22 11:38:35 holyjak thanks a lot, Sean!
2019:05:21 18:19:08 ejelome This might be answered, but I watched Matthias Felleisen's talk about types: "Types are like the Weather, Type Systems are like Weathermen".
And I remembered clojure.spec and noticed that it somehow reflects his ideas (putting contracts instead of making a type system for an untyped language, e.g. typed.clojure [which afaik is now discouraged/discontinued] and that type inference is near to impossible to implement in a untyped language like Clojure).
So the question is ... is clojure.spec the answer to Felleisen's talk?
2019:05:21 18:28:10 alexmiller no, although spec was influenced by Racket's contracts, along with other stuff
2019:05:21 18:29:18 alexmiller spec had been in development for about 4 months at the time of that talk
2019:05:21 18:33:11 seancorfield "typed.clojure [which afaik is now discouraged/discontinued]" -- I thought Ambrose was still actively working on this...?
2019:05:21 18:35:18 alexmiller he just defended his dissertation on it
2019:05:21 18:35:37 alexmiller so it has been, but not sure where it's going from here. would be a good question for him
2019:05:21 18:38:26 alexmiller iirc, while I was away at Clojure/west when that talk was given, Rich wrote all the regex stuff in spec
2019:05:21 18:54:33 ejelome thanks @alexmiller, although it didn't came directly from racket's contracts, I'm still glad that they're going almost on the same direction (no type inference, or implementing a type system [just to say we also can do type checking]).
2019:05:21 18:59:35 alexmiller well, it's not type checking, and the difference is important
2019:05:21 18:59:50 alexmiller type checking is about proving things early
2019:05:21 19:00:06 alexmiller spec is about verifying predicative constraints dynamically
2019:05:22 12:14:32 holyjak Should I make a jira issue and send a patch?
2019:05:22 12:20:48 alexmiller Go for it
2019:05:22 12:23:07 alexmiller I assume you are aware we have migrated jira to new system...
2019:05:22 17:10:38 holyjak @alexmiller I wanted to try my patch by including local clone of spec.alpha in my project but starting clj in my project then fails with
2019:05:22 12:14:19 holyjak Question: I often do something like the following, to restrict the domain of the generated values to ensure interesting conflicts:
I.e. I have a let where I use sample to generate a random, small set of data then used in generator overrides in the test itself. Do you do that too? Is there a better way?
update Sometimes I need to use the customized random data from multiple generators, which prevents I. believe the usage of simple generator derivation such as gen/let and gen/rmap
2019:05:22 14:41:03 holyjak thank you! but what if eg 2 different generators need the value? any tips?
2019:05:22 14:56:11 tangrammer yep, that’s a very interesting need …
2019:05:22 15:09:29 holyjak example: having a function taking a map with known values and a list of "things", the fn throws if any " thing" has an unknown value. If I want to test its other functionality I must ensure that things only use the values in the map get I prefer not to hardcode the map.
2019:05:22 15:46:48 misha you can test.check/let the generator, which generates [map things] tuples, and then apply function you want to test to the generated tuples
2019:05:22 16:27:35 holyjak good idea! But I guess I would need to invoke it manually instead of using spec.test/check
2019:05:23 10:05:10 misha defining wrapper which applies your function to the tuple, and specing/checking wrapper instead might be an option for you
2019:05:23 10:08:45 tangrammer could anyone write/share a gist detailed example 🙏 ? 🙂
2019:05:23 11:11:35 holyjak BTW I tried to replace my (let [uids ..] (check .. {:gen {::uid (const. (sg/elements uids)))}})) with
but it does not seem to really work. If I replace it with (constantly (sg/return (s/gen ::uid))) then I get many (desired) "collisions" in the tests but with ☝️ I get none. So my hypothesis is that the set of values is re-created every time that a new value fpr ::uid is generated, instead of creating it once and reusing it every time ::uid is requested.
2019:05:24 06:51:46 holyjak @U051KJGTX a gist of what exactly? As mentioned above, bind did not work for me, and showing a let wrapping check with some :gen overrides is perhaps not all that interesting?
2019:05:24 12:52:44 holyjak @U051KJGTX here is an example of what misha proposed, a wrapper fn taking a tuple of related inputs:
(defn apply-profile-compute-totals-wrapper
"Smart wrapper around apply-profile-compute-totals that 'unpacks' the subscr-summaries+userid->profile tuple
we generate so that both have the same subscribers before invoking the wrapped fn
"
[subscr-summaries+userid->profile]
(let [[subscr-summaries userid->profile] subscr-summaries+userid->profile]
(apply-profile-compute-totals
{:db/userid->profile userid->profile}
subscr-summaries)))
(s/def ::subscr-summaries+userid->profile
(s/with-gen
(s/cat :subscr-summaries ::kd/subscr-summaries, :userid->profile :db/userid->profile)
;; GENERATOR: Ensure that the generated userid->profile have a profile for every subscriber in
;; subscr-summaries (b/c those without profile would have been filtered out before)
(constantly
(gen/let
[subscr-summaries (s/gen ::kd/subscr-summaries)
profiles (sg/vector (s/gen :db/profile) (count subscr-summaries))]
[subscr-summaries
(zipmap
(keys subscr-summaries)
profiles)]))))
(s/fdef apply-profile-compute-totals-wrapper
:args (s/cat :1 ::subscr-summaries+userid->profile)
;:args (s/cat
; :org (s/keys :req [:db/userid->profile])
; :subscr-summaries ::kd/subscr-summaries)
:ret ::kd/subscrs+profile-usages)
(st/check `apply-profile-compute-totals-wrapper)
2019:05:24 13:12:51 tangrammer thanks a lot for sharing … I’ll give a try next week!
2019:05:23 06:54:15 holyjak (st/instrument (st/enumerate-namespace 'my.core))
Enum. enumerates all vars but instrument seems to be happy with that and presumabely skips those that are not spec-instrumentable.
2019:05:23 07:03:51 yuhan that's just what I was looking for, thanks!
2019:05:23 07:07:05 yuhan (stest/instrument (map symbol (vals (ns-interns *ns*))))
2019:05:23 12:36:17 Nolan am i missing something regarding s/union and s/select? i’m trying to do something like this:
2019:05:23 12:55:30 alexmiller probably broken atm. top-level union is probably going to go away anyways
2019:05:23 12:56:44 alexmiller I'll take a look when I next get a chance
2019:05:23 13:05:06 Nolan ok, perfect. appreciate the info!
2019:05:23 19:36:20 ikitommi could the s/def fail-fast on totally bogus spec-forms? what would be a spec for the spec-form arg in s/def?
2019:05:23 19:36:34 ikitommi
(s/def ::a 1)
;; :user/a
(s/valid? ::a 1)
;; Syntax error (ClassCastException)
;; java.lang.Long cannot be cast to clojure.lang.IFn
2019:05:23 19:39:40 alexmiller it could check some things better, for sure
2019:05:23 19:55:54 alexmiller what's valid is changing between spec 1 and 2
2019:05:23 19:58:48 alexmiller in spec 2 the valid symbolic forms are keywords, sets, symbols, and lists/sequences which are spec forms where a known spec op is in op position (ops are an extension point via multimethod)
2019:05:23 19:59:17 alexmiller we may add another datafied/map form, still tbd
2019:05:23 20:00:28 alexmiller notably, function objects are not allowed (different than spec 1)
2019:05:23 20:00:54 Diego hey all, does anyone know why can’t I do something like this with spec?
2019:05:23 20:02:08 alexmiller you mean, inline a spec in s/keys?
2019:05:23 20:02:54 alexmiller by design, spec is trying to enable a shared global registry of attribute specs
2019:05:23 20:03:24 alexmiller the attribute spec is considered to be more important than containers of those attributes
2019:05:23 20:03:56 alexmiller as such, we require you to strongly associate a spec with an attribute name
2019:05:23 20:04:47 alexmiller spec 2 will have more support for inlining attribute spec defs for unqualified keys (common with json interop)
2019:05:23 20:09:09 Diego I see. Being new to spec I find that not being able to inline specs makes it harder for me to reason about them but I suppose that might change once I get more familiar with it, and one I learn about the best practices on creating specs (any suggested links?).
2019:05:23 20:13:11 Diego btw, thanks for the response @alexmiller
2019:05:23 21:06:05 Nolan curious if there is a significant expected runtime difference between computing s/selects dynamically and defing them prior. e.g. if the following s/valid calls were going to be called in a loop:
2019:05:23 21:16:45 alexmiller you should expect that we have spent no time yet on perf aspects of select :)
2019:05:23 21:19:29 alexmiller in general though, the first two are reusing the same spec object, which means work can be done once at spec object creation time (optimization work is basically shifting as much work as possible into construction time here) whereas the last one would pay that cost every time
2019:05:23 21:19:54 Nolan exactly—was about to say that i suppose i’d always def it, if it weren’t truly dynamic. was just tangentially curious about the perceived cost of the dynamism there
2019:05:23 21:20:03 alexmiller so in general, I would tend toward either using an object saved in var or from the registry (those are probably approx the same)
2019:05:23 21:20:45 alexmiller but also note that the tradeoff, as usual, is in not picking up changes to specs in the ... there
2019:05:23 21:21:17 alexmiller so you might make the opposite choice at the repl, if you're in development
2019:05:23 21:22:29 alexmiller generally, I am just working in a file, that has all the specs I'm working on in it, and I just reload the file, which registers and recompiles all of the specs, and I don't care
2019:05:23 21:22:42 alexmiller it is useful to have that mental model though
2019:05:23 21:25:24 Nolan right, right. got it. thats a premium tip. spec2 is awesome, have been having a blast getting some experience. it alleviates essentially all of my old s/keys woes 🥳
2019:05:23 21:34:59 alexmiller great!
2019:05:24 06:54:04 holyjak Does anybody have any idea why might I be getting the error
> Syntax error macroexpanding clojure.core/defn at (alpha.clj:78:1).
> #object[clojure.spec.alpha$and_spec_impl$reify__2183 0x1698fc68 "clojure.spec.al/cdn-cgi/l/email-protection"] is not a fn, expected predicate fn
when trying to start a REPL in the spec.alpha project and loading the code? I did this:
2019:05:24 07:06:02 alexmiller spec is aot compiled, might be a conflict since you are forcing a load of the clj, which recompiles the protocol (yet all the old instances are from the prior protocol). should probably use load instead?
2019:05:24 07:46:08 holyjak Ah, OK, thank you, that helped!
I am used to load-file and do not really know how it works, I guess I should learn 🙂
2019:05:24 17:47:27 seancorfield @U0522TWDA I'm curious how you got into the habit of using load-file in the first place? It seems like require would be the "obvious" thing to learn first...
2019:05:24 17:49:51 alexmiller yeah, that seems weird
2019:05:24 17:51:00 holyjak Perhaps a bad habit from Node.js where require is idempotent and does not actually reload the code from the disk.
Also, I see that Cursive's "load file in repl" used to reload the code actually calls load-file, at least when running REPL via main (instead of via lein)
2019:05:24 17:52:50 holyjak Ah, no, Clojure require does not reload the code either
> Loads libs, skipping any that are already loaded.
So what are you saying? That the 1st time I should use require but after I change the code, to get the changes in, then I should use what?
2019:05:24 17:53:09 holyjak (I guess I will get the answer after I finish Eric's REPL course)
2019:05:24 17:53:09 seancorfield (require ... :reload) or (require ... :reload-all)
2019:05:24 17:54:28 holyjak Also, require requires that the code is on the classpath while load-file does not care about that. But I guess that is not a problem if I only use lein repl in lein project and clj in deps.edn projects as those have the correct paths autom.
2019:05:24 17:54:59 seancorfield I think editors are likely to use load-file since they have a filesystem path, rather than a namespace -- but I view load-file as a tooling/system-level hook. Always interesting to hear how other folks developed their Clojure habits...
2019:05:24 07:06:12 alexmiller or require?
2019:05:26 14:41:30 y.khmelevskii Hi everyone! Can you please explain me why this spec doesn’t work correctly:
I'm not sure what restrictions are in place for unqualified key usage. It is all alpha, after all.
2019:05:26 23:54:51 benzap Has clojure.spec 2.0 landed already, or is it still in the development stage?
2019:05:26 23:59:02 seancorfield Very much alpha at the moment @benzap
2019:05:26 23:59:32 seancorfield There hasn't even been an "alpha" release yet -- it's just on GitHub.
2019:05:27 00:05:22 benzap Okay, thanks for the update!
2019:05:27 00:09:10 seancorfield At work we have a branch of our code running on spec-alpha2 just to keep an eye on changes. We haven't started using any of the schema/`select` stuff yet. Alex has repeatedly told me it's not ready to run in production yet 🙂 Even for us, who run alpha stuff in production all the time 🙂
2019:05:27 00:12:07 benzap haha, well it sounds like it's on it's way then, that's good to hear 🙂
2019:05:27 00:13:31 seancorfield Yeah, still some weird quirks to be ironed out but it wasn't too painful to get our code running on the new version straight from GitHub (since we're an all-`deps.edn` setup at work)
2019:05:28 08:25:27 zclj any advice in how to create a spec for a "string-date" where the string should be a valid RFC3339 format?
2019:05:28 10:24:54 Mikko Harju Hi! Is there a mechanism to get exhaustive errors from a spec or how would one approach validating maps with required keys and keys that depend on the content of another one? For instance, given a spec like
does (sg/sample (s/gen ::account)) fail with
> Unable to construct gen at: [] for: :myapp/account
? Doesn't the with-gen add the generator?
2019:05:29 01:49:36 Chris Reyes I’m interested in using spec (for the first time) for a side project I’m working on. Where should I put the spec definitions? (Or is that a controversial question?)
2019:05:29 01:53:47 seancorfield @chrisreyes Most people tend to put data specs in their own namespace, possibly with a few utilities for processing that data, but put function specs next to (above) the functions to which they apply.
2019:05:29 01:54:26 seancorfield If you want specs to be "optional" for some functions, it makes sense to put them in a separate namespace, so that users can decide whether to load them or not.
2019:05:29 01:55:53 Chris Reyes Okay, thanks!
2019:05:29 01:56:26 seancorfield There is no "right way" or "wrong way" -- whatever is most convenient/makes the most sense.
2019:05:29 01:58:05 seancorfield That ns actually contains fdefs for two other namespaces within the next.jdbc project, just because that was the most natural/convenient way to set things up.
2019:05:29 01:58:31 seancorfield But having fdefs separate isn't as common as having s/defs for data separate.
2019:05:29 01:59:42 seancorfield Part of the issue is that specs can serve a lot of different purposes. They can be used for testing in several ways. They can be used to support development (`st/instrument` for example). They can be used in production for data validation (and other things).
2019:05:29 09:23:30 hlolli What's & rest in spec. I want to spec [:a :B :C :D] and make sure that first is :a and the rest can by of any amount of any type? (s/cat :need-a ::need-a (&rest?))
2019:05:29 09:32:42 djtango the nested s/* (and other regex ops) is flattened out by default so that doesn't mean [:a [:b ...]]
2019:05:29 09:34:07 hlolli ahh I see 🙂 thanks dj! Still after 2 years of spec, these basics are still troubling me
2019:05:29 09:34:47 djtango that one definitely requires "grokking"
2019:05:29 09:35:55 hlolli yes I would have expected a nested sequence for that
2019:05:29 09:36:09 djtango I guess with a lisp hat on you would expect it to nest, but if you were trying to imagine expressing regex via s-expressions it would look like that too so :shrug:
2019:05:29 09:37:59 djtango s/tuple behaves how you might expect iirc
2019:05:29 09:47:22 djtango Do people do generative testing over side-effectful code and/or integration tests?
I feel it is nice to be able to put an fdef spec over an API endpoint on a webserver (e.g. required params and possible responses) but am unconvinced setting up the check as it starts becoming a bit like a super verbose example test. I suppose instrument with :stub option could work but I'm trying to find a nice way of tying fdefs into integration tests...
2019:05:29 09:57:00 holyjak Speaking about integration / system tests, these talks https://lispcast.com/testing-stateful-and-concurrent-systems-using-test-check/, https://youtu.be/zjbcayvTcKQ and Datomic Simulat could be of interest.
Regarding side-effects, I guess it depends on their kind. Some you certainly want to avoid when testing. (x with datmoic, you can use an "in-memory" local copy of the DB and never "commit" / transact into the original DB yet have everything there for verification.
2019:05:29 10:01:53 djtango thanks for sharing
2019:05:29 10:01:56 djtango will take a look!
2019:05:29 10:03:06 djtango alas we're not using Datomic at work though
but there seems to be no instrumentation if I forget to provide required args, or type them incorrectly? Any obvious error here?
2019:05:30 16:59:48 drone there’s no key for the second key-pred pair in s/cat
2019:05:30 17:13:09 drone and [env]seems wrong should be removed, unless there’s some form of fdef I’m unfamiliar with
2019:06:04 16:25:00 hlolli yup that's right, works now, thanks!
2019:05:31 19:40:04 mpdairy so i have an idea for instrumentation with spec. It would be nice if the arguments to a function with a failing fdef were saved somewhere so they could be inspected in the repl, since a lot of times the spec error message is not very useful, and sometimes is so large it nearly crashes emacs.
2019:05:31 19:41:35 mpdairy speaking of that, is there a way to truncate the length of spec error messages?
2019:05:31 19:42:30 ghadi spec error messages should be really short as of the last changes
2019:05:31 19:42:44 ghadi what version clojure + spec are you using?
2019:05:31 19:46:09 mpdairy org.clojure/spec.alpha "0.2.176" and clojure 1.10.0
2019:05:31 19:48:16 ghadi are you sure you're not mistaking the ex-data for the exception message?
2019:05:31 19:48:33 ghadi (that's the latest release)
2019:05:31 19:49:37 mpdairy yeah, it's the explain data. it's when it's showing the "actual" datatype, which can be huge
2019:05:31 19:50:10 ghadi that's a tooling thing... CIDER should have a toggle for it
2019:05:31 19:50:18 mpdairy oh ok
2019:05:31 19:50:37 ghadi (It used to be that the ex-data was also serialized into the message string, which is admittedly awful, but that is no longer the case)
2019:05:31 19:51:57 mpdairy oh yeah i remember those times
2019:06:04 10:38:09 holyjak Hello, how do I spec protocol methods? Just (s/fdef protocol-method :args (s/cat :this any? ...)) ?
2019:06:04 10:56:03 metametadata You'll have to wrap the protocol method into a spec-able function. E.g. using defn-spec:
So it's obviously a string, but it's a macro, so I understand it sees a list at this point, any good tip to make this string check?
2019:06:04 21:38:15 hlolli (s/def ::orc-string #(string? (eval %))) works actually
2019:06:04 23:05:53 alexmiller That’s not a good pattern
2019:06:04 23:06:50 alexmiller By eval‘ing here you’re basically ruining the lazy evaluation of macros and could even cause issues
2019:06:04 23:07:36 alexmiller In general macro specs are often tricky to write unless you’re trying to enforce positional constraints
2019:06:04 23:07:56 alexmiller For something like this I would probably not spec it at all
2019:06:04 23:08:18 alexmiller But would spec that arg as any? if I was
2019:06:05 00:00:48 seancorfield @hlolli Remember that macros take code as input and produce code as output -- they don't see the runtime type of that code.
2019:06:05 00:05:17 hlolli hmm, my macros are the few things in my app that I actually want to spec, because of their interface nature to my app. Yeh, I can ofc spec a presence/absence of an argument. Or spec a function that the macro calls... also a solution.
2019:06:05 01:00:57 alexmiller the latter is a good idea
2019:06:05 01:01:35 alexmiller keep in mind that macro specs are checked at compile time. so they can only check things you know at compile time.
2019:06:05 01:02:00 alexmiller they can't check things about the values, because the values don't exist yet
2019:06:05 01:02:51 alexmiller so macro specs are great for checking syntax (in core we use them for ns, defn, destructuring, etc) - things that are macros with their own syntax defined by the macro
2019:06:05 01:03:33 alexmiller function specs are checked (when instrumented) at runtime when you have values in hand
2019:06:05 01:03:35 hlolli yup, so this makes total sense to me. I should keep it a habit to do as little work as possible in a macro, I only need the def on a symbol, and I can forward the rest to a function*
2019:06:05 01:04:09 alexmiller in general, having pairs like that is one common thing people do
2019:06:05 01:04:48 alexmiller with the caveat that the macro expands to a function call, so if you want that to be part of your public api, available outside the ns, then the function needs to be public too
2019:06:05 01:05:01 alexmiller sometimes that feels dirty
2019:06:05 01:05:42 alexmiller spec itself has this all over (s/and is a macro that expands to s/and-spec-impl etc)
2019:06:05 01:06:07 alexmiller we have significantly changed that in spec 2... but that's a longer story
2019:06:05 01:07:48 hlolli yeh I see, it's dirty in a forgiveable way. Other plus is that when I'm working in the repl, I only need to change the function once (given that I have a `(def ~symbol function) pattern), instead of re-evaling all def instances
2019:06:05 01:09:22 alexmiller yep
2019:06:05 16:58:58 dangercoder Hi! Never looked at how Spec works when it comes to data that depend on other data.
Is it possible to use clojure.spec to validate and generate relational data? Let's say I have a map like this:
:amount must be within the bounds of min-amount and max-amount.
2019:06:05 16:59:27 alexmiller you can do this with custom generators
2019:06:05 17:00:23 alexmiller gen the min and the max and then use gen/bind to create a generator that produces values in the range and packages them all together
2019:06:05 17:01:08 dangercoder Yeah I am listening to a podcast about Specmonstah right now
2019:06:05 17:01:10 kszabo not for the usecase you mentioned, but for relational data generation
2019:06:05 17:01:38 dangercoder Thanks @alexmiller I will look more into how custom generators work! 🙂
2019:06:05 22:08:15 plins hey everyone, I want to spec a map with 2 values (`:key1` :key2), they are both ints so ill use integer?..
but the second key must be bigger than the first, whats the best way to achieve that?
2019:06:05 23:34:04 seancorfield @plins wrap the s/keys in s/and and add a predicate
2019:06:05 23:38:22 seancorfield @plins if you need more concrete guidance than that, LMK and I'll paste an example
2019:06:06 00:07:01 plins thank you very much but I've managed to do it 🙂
2019:06:06 15:45:38 sveri
2019:06:06 15:50:37 alexmiller you're just getting the spammed ex-info data there, not a crafted message
2019:06:06 15:50:45 alexmiller which clojure version and where are you seeing this?
2019:06:06 15:51:25 alexmiller seeing it in repl (if so, which repl) or from lein command or other command?
2019:06:07 09:31:31 sveri Getting back to this problem. I did some digging and it turns out, if I run this function as part of a web request it gets wrapped into a exception clojure.lang.ExceptionInfo and printed out like seen above vs calling the function from the repl and seeing the more readable output.
What would be the idiomatic way to use spec in this environment?
I could catch the exception, and display some better error? Is it possible to extract a more readable error from the exception?
Two things that bug me here.
1. ExceptionInfo seems like a very generic exception and I would have to look into it to check if its a exception regarding spec.
2. I would have to do this for every function that accepts / returns a web request while I would like to have it printed readable everytime, without having to do manual exception handling, if possible.
2019:06:06 15:57:48 sveri I see this in the cursive REPL when running a function of mine that I specced.
There is also a lengthy stacktrace available and what fails is clojure.spec.test.alpha$spec_checking_fn$conform_BANG___3024.invoke
Clojure version is 1.10.1 and spec is spec.alpha 0.2.176
2019:06:06 16:06:19 alexmiller what kind of repl in Cursive?
2019:06:06 16:07:18 alexmiller I'm most likely going to suggest you ask in #cursive as its really dependent on who is controlling the printing, which here is either cursive or nrepl
2019:06:06 16:08:05 alexmiller that is, if you used lein or clj in this same scenario, I think you would get a much better output
2019:06:06 16:58:39 vemv I post here a variation of something that I posted in #clojurescript this morning, but is essentially a different problem (it affects JVM clojure)
I have a ns l from library L which requires clojure.core.specs.alpha, and uses one of its specs via spec/def.
And I have a ns a from app A that requires l.
When running A, I get java.lang.Exception: Unable to resolve spec: :clojure.core.specs.alpha/params+body. This doesn't happen when running L.
I use the Reloaded workflow with vanilla requires and project organisation. All deps are the latest
2019:06:06 17:19:00 vemv @alexmiller :
The problem appears to boil down to the fact that just that specific spec is absent from the registry
2019:06:06 17:19:07 vemv
2019:06:06 18:10:37 alexmiller there were some core spec renames in the 1.10-era core.specs.alpha - are you seeing mismatches maybe?
2019:06:06 18:11:50 alexmiller yeah params+body was a rename in 1.10-era from 1.9-era
2019:06:06 18:12:44 alexmiller changes happened in core.specs.alpha 0.2.44
2019:06:06 18:14:11 alexmiller so would be good to know the version of that being used by L and A, and also whether aot is happening on L
2019:06:06 18:15:46 vemv hi! I've been doing archeology for the last hour and I still can't make sense of things
even when using 1.10.1 and the latest clojure.core.specs.alpha dep (stable, or snapshot, or none at all, since it's bundled), (require '[clojure.core.specs.alpha] :reload-all) won't get me the params+body spec in this particular project
maybe Leiningen is doing sth evil
2019:06:06 18:19:23 vemv no aot in either side. tried lein clean, no luck either
2019:06:06 18:23:32 alexmiller just using clj here
2019:06:06 18:29:49 vemv yes, in the original library ("L") it just works
in a larger app ("A") using L it doesn't
Did whatever I could to bisect, no luck
I'll try creating another app, since A is quite large tbh, increasing the chances for weird interactions
2019:06:06 18:41:08 alexmiller does the registry contain any of the core.specs.alpha stuff?
2019:06:06 18:48:18 vemv most of them yes. see #clojure , found the thing
2019:06:06 19:17:47 sveri @alexmiller its the same in the clj repl. But I have a lot of libraries loaded, not sure of something messes around with the output.
2019:06:06 22:48:27 plins did some googling but not found the anwser, is it possible to share specs between clj and cljs files? do I need to write .cljx files?
2019:06:06 22:57:39 alexmiller you can write specs in .cljc files and share them
2019:06:06 23:40:09 plins any recommended documentation on how to setup clc files and share them? something has to be done in project.clj?
2019:06:07 00:10:28 seancorfield @U3QUAHZJ6 It's mostly automatic. The compilers look for both .cljc files and their own suffix (`.clj` or .cljs).
2019:06:06 22:58:07 alexmiller .cljx is old and shouldn't be used anymore
2019:06:07 07:08:46 sveri @alexmiller I just checked my error in a minimal example and indeed the output is very different:
2019:06:08 11:13:02 misha need it for a recursive spec, to uniformly label all tree nodes, to dispatch on conform-labels in a case
2019:06:08 12:48:58 alexmiller Can’t say I have any better idea other than to do this walk after the conform instead
2019:06:08 13:33:14 misha (re-)labeling conformed tree is doing (almost) double work. and implies having double knowledge of "how nodes look like" in the code: in spec and inline during the walk.
Or, worse, dispatching with call to s/valid? or s/conform again, which will conform entire subtree from current node down.
the (s/or) wrappers are hacky, but not that bad for irregular trees, all things considered:
2019:06:08 14:48:17 alexmiller my complaint is basically that you're turning conform into a meat grinder to get a specific data structure you want, and that's not what it's designed for - it's designed to tell you a) is this valid? and b) why?
2019:06:08 15:15:36 potetm I thought it was also deigned to be “destructuring on steroids”
2019:06:08 15:15:54 potetm (I forget where I heard that.)
2019:06:08 15:23:45 alexmiller it has some capability in that regard when conforming regex for syntax structures in macros, but even then it's not destructuring, more making structures amenable to destructuring
2019:06:09 06:38:19 misha I completely agree with your complaint. But it works! kappa
2019:06:09 06:39:21 misha (went with loop+case and weaker jit errors instead of spec for such tree this time, though)
2019:06:09 07:08:26 misha further destructuring of a form conformed to regex-spec and to or-spec is different: map vs "twople". In case of a tree, you'll walk 1 node at a time, which makes
2019:06:09 07:09:52 misha but latter brings the same edge cases as (contains? #{...} x) vs. just (#{...} x): nil, false
2019:06:10 02:38:26 andy.fingerhut There is an example spec using s/double-in on the spec guide page: (s/def ::dubs (s/double-in :min -100.0 :max 100.0 :NaN? false :infinite? false)) I have tried changing the false occurrences to true, and leaving out the :NaN? and :infinite? keys, but I always see (s/valid? ::dubs ##Inf) evaluate to false, as well as (s/valid? ::dubs ##-Inf) and (s/valid? ::dubs ##NaN). Is that expected?
2019:06:10 02:44:04 andy.fingerhut I guess it makes sense for ##-Inf to be invalid if a :min value is specified that is a double value other than ##-Inf, and similarly for :max values and ##Inf.
2019:06:10 02:44:30 andy.fingerhut But it seems like specifying either :min or :max changes the validity of ##NaN from true to false.
2019:06:10 02:44:54 andy.fingerhut no matter what the value associated with the :NaN? key happens to be
2019:06:10 02:49:40 alexmiller might be the ordering of the checks
2019:06:10 02:50:14 alexmiller if you (s/form :;dubs) you can see what it expands to
2019:06:10 03:02:52 alexmiller maybe buggy at a glance as it's an s/and and it starts flowing the boolean rather than the double
2019:06:12 13:23:48 carkh multi-spec dispatching on the first value in a sequence and returning the syntax for the rest of that sequence, possible at all ?
2019:06:12 13:24:06 carkh all examples of multi-specs i could find were about maps
2019:06:12 13:26:31 carkh what i'm trying to do : (s/def ::node (s/cat :tag keyword? (multispec dispatching on the tag here)))
2019:06:12 13:30:23 alexmiller it is not only maps, can definitely be made to work with other things (I've got some stuff doing that)
2019:06:12 13:32:00 alexmiller you can make your multimethod dispatch on first
2019:06:12 13:32:09 alexmiller and then you have to be a little careful with the retag function
2019:06:12 13:32:23 alexmiller I actually have an example of this in CLJ-2112 (specs on specs) I think
2019:06:12 13:33:24 alexmiller you'll see spec-form is a defmulti on first
2019:06:12 13:33:59 alexmiller and the ::spec spec uses (s/multi-spec spec-form (fn [val tag] val))
2019:06:12 13:34:26 alexmiller and then the defmethods cover the different cases with different starting symbols
2019:06:12 13:38:43 carkh thanks looking at your example
2019:06:12 13:42:16 carkh ok got it working on the whole "sequence", i'm guessing this can't be done the way i want, dispatching on the first element in order to spec the rest of the sequence
2019:06:12 13:42:48 carkh not a huge deal, just a little usability issue i guess
2019:06:12 13:44:21 alexmiller isn't the example above what you're describing?
2019:06:12 13:44:50 alexmiller I'm using it to spec a form sequence based on the operator type at the beginning, so seems like the same thing?
2019:06:12 13:52:27 carkh (s/cat :tag keyword? :rest (s/multi-spec node-type :rest)) it looks like it's looking for a sequence in the rest
2019:06:12 13:53:03 carkh the mmultispec is returning something like (s/cat ....) a flat thing
2019:06:12 13:53:32 carkh is multispec like s/spec in this regard ?
2019:06:12 13:54:00 carkh (i have a hard time following your example)
2019:06:12 13:55:07 alexmiller you don't want an s/cat here at all - the multi-spec is the sequence spec
2019:06:12 13:55:39 carkh but i need something to put around my stuff returned from the multimethod =)
2019:06:12 13:55:58 carkh but yes i understand
2019:06:12 13:56:03 alexmiller I don't understand what that meant
2019:06:12 13:56:13 alexmiller the multimethod returns the spec for the sequence
2019:06:12 13:56:18 carkh right
2019:06:12 13:56:49 carkh in the typical example for a typed map, you spec for the type keyword outside the multimethod
2019:06:12 13:57:12 carkh then only spec for what's relevant to the multispec user in the multimethod
2019:06:12 13:57:56 carkh but i think what i'm asking is impossible
2019:06:12 13:58:17 alexmiller afaict what you're asking for is the example I gave. I don't get why it's not what you want.
2019:06:12 13:59:19 alexmiller are you looking for something different in the conformed value shape?
2019:06:12 14:00:03 carkh in your example, i want the :f #{'clojure.core/fn} out of the multimethod
2019:06:12 14:00:44 carkh just like for the typical typed map examples
2019:06:12 14:01:10 carkh actually i'm wrong on that
2019:06:12 14:01:54 carkh event/type is indeed present in the multi-method in the guide
2019:06:12 14:02:39 carkh ok then sorry for the noise ><
2019:06:12 14:03:11 alexmiller you can post-process the conformed value to do whatever you want of course
2019:06:12 14:04:10 carkh i was just weary that all the users of the multimethod will have to spec for the "tag"
2019:06:12 14:04:20 carkh not a huge deal since all those users are me =)
2019:06:12 14:04:31 carkh thanks for your time !
2019:06:12 14:16:40 alexmiller yeah, you can just spec it as any? if you like
2019:06:12 14:16:58 alexmiller it's already been chosen by the time you get into the multimethod and the value will flow through
2019:06:12 14:17:25 alexmiller I spec'ed it in that code just for clarity
2019:06:12 17:04:17 colinkahn When multi-specing where the specs are s/keys and the retag is a keyword, is it best practice to include the retag key as :req in the defmethods keys spec?
2019:06:12 17:23:11 alexmiller not sure what you mean by "include the retag key as :req"
2019:06:12 17:23:45 alexmiller oh, you mean as a :req in the s/keys
2019:06:12 17:24:20 alexmiller well it is probably required to get to that point in the first place
2019:06:12 17:25:03 alexmiller and either way it's going to end up in the conformed value
2019:06:12 17:25:50 alexmiller I'm not sure that it matters either way, depends on whether you see the retag key as part of the data or as frame around the data, but that's probably dependent on your code
2019:06:12 17:51:26 colinkahn Ok cool, yeah I was doing it quite consistently in :req and ran into something where I was adding a :gen to the keys spec in the defmethod and found it a bit unergonomic to have to include the retag in the generator as well, which got me feeling like maybe it was better not to include it at all. I have been also specing the retag key mostly as something rather generic like keyword? too.
2019:06:13 11:12:33 vemv Hey there,
I wrote down this design https://github.com/reducecombine/fridge/issues/9 50% for fun, 50% b/c it might actually be necessary at work.
It doesn't seem too extravagant (in fact I view it as very aligned with RH-isms: immutable/never-broken specs, growth by accretion, no renames etc), but still, there is a gap between theory and practice 🙂 i.e. my design might actually have flaws in a real system.
It would be super appreciated if someone with relevant experience could comment in the issue.
2019:06:13 11:12:33 vemv Hey there,
I wrote down this design https://github.com/reducecombine/fridge/issues/9 50% for fun, 50% b/c it might actually be necessary at work.
It doesn't seem too extravagant (in fact I view it as very aligned with RH-isms: immutable/never-broken specs, growth by accretion, no renames etc), but still, there is a gap between theory and practice 🙂 i.e. my design might actually have flaws in a real system.
It would be super appreciated if someone with relevant experience could comment in the issue.
2019:06:13 11:45:39 alexmiller Seems fine. I guess my only real question is whether apps and models really need that level of absolute separation that comes from putting the app name in the attribute qualifier
2019:06:13 11:47:34 alexmiller Seems like you will end up basically copying the same attribute definition everywhere (and require unnecessary data transformation at each step). The whole idea of the global registry is to get reuse
2019:06:13 11:48:15 alexmiller There is more coming on datafied specs in spec 2 btw
2019:06:13 12:15:10 vemv > Seems like you will end up basically copying the same attribute definition everywhere
Not sure I can follow.
Let's say I have N repos, 1 per microservice, plus 1 special repo, the 'immutable spec registry'
Over there, I (spec/def :messaging.blue-app.v1/age integer?)
And microservices can consume that spec (and quite concisely, via require ... :as ...). There's no copying in that
> and require unnecessary data transformation at each step
What did you mean?
2019:06:13 13:22:57 alexmiller I assume these services pass data to each other. It seems improbable that you won't have multiple services handling the same semantic attribute (like "age"). in your scheme, that attribute will show up in the messages for all the services - :messaging.blue-app.v1/age, :messaging.red-app.v1/age etc
2019:06:13 13:23:39 alexmiller if a service ever takes a map with one variant and returns a map with the other variant, you have rebuild the map and change the attribute name
2019:06:13 13:24:15 alexmiller whereas if you had a :messaging.common.v1/age, it could flow between services and be shared
2019:06:13 13:24:24 alexmiller which was the whole point of the global registry
2019:06:13 17:47:18 seancorfield This is why we tend to use more generic domain-level prefixes for our specs, rather than specific/namespace-qualified names.
2019:06:13 18:24:14 vemv Understood!
I'll give it a think, however I find some value in having this design:
Each microservice can retain a specific meaning and type for age. That way, services can evolve independently, without coordination.
The opposite might resemble a distributed monolith, where making a change/decision needs asking everyone else (other codebases / other team members) first
(btw, messaging is via Kafka so the map-in-map-out philosophy doesn't apply that much)
2019:06:13 21:23:19 rapskalian @U04V70XH6 could you give an example of what you mean by “domain-level”? Would you tend to favor :user/email over :myapp.auth.user/email?
2019:06:13 21:49:44 seancorfield For stuff that stays "inside the company", the qualifier doesn't need to be any more specific than the broadest level of uniqueness needed. So, yeah, if you want to standardize on an email representation for all your users across the whole company, :user/email is fine. It shouldn't be part of any public APIs/libraries you create -- but using commonly known domain names inside your company helps reuse across multiple systems.
2019:06:13 21:52:53 seancorfield A specific example is :wsbilling for us -- for a lot of things that are related to our World Singles Billing services rather than being tied to a single app, these specs are reused by all apps that support and interact with those services. We've used :wsbilling there rather than just :billing because there are other generic billing services we interact with, so this just hints that it's about our billing services rather than anything else we interact with.
2019:06:13 23:01:01 rapskalian Very helpful, thanks
2019:06:14 06:36:14 jaihindhreddy @U45T93RA6 "making a change/decision needs asking everyone else (other codebases / other team members) first", as RH says in Spec-ulation, the way we use the word change tends to complect growth and breakage, which are completely different. As long as you're only doing growth, these is no need to ask (or even tell) anyone.
2019:06:14 06:45:18 seancorfield That's why it's so important to me that next.jdbc will never make breaking changes -- only accretive/fixative changes, since it moved into beta. I learned a lot of lessons from maintaining clojure.java.jdbc for the last eight years! 🙂
2019:06:14 07:36:46 vemv @ jaihindh, how do you tell if a change is breaking or not? as hinted in the document, One man's non-breaking is another's breaking. My approach is pessimistic in that it assumes all changes are potentially breaking.
At the same time, it encourages services to keep sending/accepting messages at old versions, so you don't actually break 3rd parties.
This philosophy is partly justified by the fact that atm we don't have a tool that automatedly analyses spec compatibility. Might be unfeasible given that specs can be backed by arbitrary code
2019:06:14 11:22:31 jaihindhreddy I don't have the answer (don't even have an answer) for how to partition "change" into "growth" and "breakage". And no matter how hard we try, at some point people will rely on things that are broken, making one man's fixation another man's breakage, etc. But if we keep changing our names by incorporating versions into them and ask people to assume they are potentially breaking, we still lack a stable set of names that can act as a stable basis to reason about our data across time. That, is the real systems problem right?
Sorry for the blabbering & noise 😅
2019:06:14 16:06:23 seancorfield You don't change the names. Just introduce new names. The old names still exist and have the old behavior. Folks can continue to rely on those old names "forever".
2019:06:14 13:30:26 dmarjenburgh Is there a convenient way to separate specs definitions and their generators? I want to define my specs with my main code, but only use generators in test code. I include test.check in my classpath only with the test alias. I know spec loads generators lazily for this purpose, but not sure how to use s/with-gen when the generator argument uses stuff depending on test.check
2019:06:14 13:42:32 dmarjenburgh Okay, I think I got it. I was calling gen/fmap when my namespace of custom generators was loaded.
2019:06:15 19:20:01 andy.fingerhut one of the places in Clojure where vectors vs. non-vectors makes a difference.
2019:06:15 19:34:24 y.khmelevskii hey, can you please help me to understand why lein throw this error
If there are a lot of uncached dependencies this might take a while ...
clojure.lang.ExceptionInfo: Could not find artifact org.clojure:spec-alpha2:jar:0.2.177-SNAPSHOT in central ()
{:lib org.clojure/spec-alpha2, :coord {:mvn/version "0.2.177-SNAPSHOT", :deps/manifest :mvn}}
2019:06:15 19:37:04 y.khmelevskii I use leiningen together with deps.edn
2019:06:15 19:40:28 bronsa snapshot releases are not in maven central, they're in sonatype
2019:06:15 19:40:42 bronsa try adding in your repos
2019:06:15 19:41:38 y.khmelevskii which way? via :mvn/repos?
2019:06:15 19:58:09 y.khmelevskii and my dependency
org.clojure/spec-alpha2 {:mvn/version "0.2.177-SNAPSHOT"}
2019:06:15 20:00:07 bronsa not sure if lein picks up repos from deps.edn
2019:06:15 20:00:28 bronsa if you're sure that snapshot exists, maybe trying adding that repo in .lein/profiles.clj and retry
2019:06:15 20:00:34 bronsa don't have any other suggestions, sorry
2019:06:16 09:52:59 hlolli how is it possible to prevent a generator from doing a retry without re-calling a generator function? Here's an ugly phone number generator
2019:06:16 11:23:48 gfredericks okay well there's some stuff buried in the jvm method
but for example, is the last portion supposed to be a particular number of digits?
2019:06:16 11:25:04 hlolli yes, it depends I think which country code is used, so I just multiply some random number to sizeof(int), should give me a more likely way of number that purely random number generator, I think?
2019:06:16 11:25:32 hlolli Or I could choose 1 counry code and substring to a specific length
2019:06:16 11:25:59 gfredericks (* Integer/MAX_VALUE (Math/random)) is going to be pretty skewed towards certain lengths
2019:06:16 11:26:18 gfredericks if it's easy to get a length associated with each country code, then you could do it much better
2019:06:16 11:27:01 hlolli yes, these rules are difficuly, despite all lengths being right, sometimes the first digit must fit some standard
2019:06:16 11:27:07 hlolli I can read more about this ofc
2019:06:16 11:27:24 hlolli but I thought the generator could try again and again until it's valid
is the sort of thing I would aim for if I Had that list
2019:06:16 11:28:07 hlolli cool, instersting 🙂
2019:06:16 11:28:12 gfredericks relying on the generator trying again is not a good plan unless you're sure it's highly likely to succeed
2019:06:16 11:28:27 gfredericks because you can easily be in a situation where it only succeeds e.g. 0.001% of the time
2019:06:16 11:28:36 gfredericks and then you'll be burning a lot of CPU
2019:06:16 11:29:22 hlolli yes true, I thought the idea would be to give the generator a bit of help. The odds of alpha-numberic random gen hitting a valid phone number are probably astronomical, I'd assume
2019:06:16 11:29:46 gfredericks yeah, you can definitely do worse than what you have 🙂
2019:06:16 12:11:59 gfredericks @hlolli incidentally, here's a generator that's evenly distributed between digit lengths
2019:06:16 12:14:47 gfredericks it's just syntax sugar for gen/fmap and gen/bind mostly
2019:06:16 12:14:59 gfredericks you can do the above just using gen/bind
2019:06:16 12:17:57 hlolli ah I see, I was searching for it in clojure.spec.gen.alpha
2019:06:16 12:18:50 hlolli thanks 🙂
2019:06:17 01:21:50 Alexander Stavonin Is it possible to check that a spec argument is a function with expected signature?
2019:06:17 01:24:46 Alexander Stavonin I can easily implement such check with protocols, but in one particular case function is preferable for me.
2019:06:17 02:40:17 taylor spec alpha1 “checks” function values by invoking them; so yes, if that meets your needs you could spec function args
2019:06:17 02:40:35 taylor maybe you could do something with meta and :arglists?
2019:06:17 03:11:01 Alexander Stavonin I have a function which accept other function as argument. I’d like to be sure, the argument is: a) function, b) function with expected signature (lets say with 2 arguments).
I suppose, fn? could be helpful here, but what should I do with function arguments test?
2019:06:17 04:33:21 seancorfield @UKHQCLTQV clojure.spec isn't a type system.
2019:06:17 04:36:17 Alexander Stavonin I know, but sometimes type-system like guarantees are very convenient. This is especially important in cross components calls. Can you suggest any other way to which will provide such guarantees?
2019:06:17 04:39:30 seancorfield Maybe you want to look at Typed Clojure instead, in order to analyze code and ensure that you're passing the right sort of function as an argument?
2019:06:17 04:44:55 Alexander Stavonin @U0JLGECPK, are you talking about https://github.com/clojure/core.typed?
I just looked thru readme and got feeling Typed Clojure means – whole project should be statically typed. This is not a goal for me, as I’m more or less happy with dynamic typing, and the only needs is – cross components interface data and types validity guarantees.
2019:06:17 04:48:13 Alexander Stavonin Ok, I should look deeper here:
This work adds static type checking (and some of its benefits) to Clojure, a dynamically typed language, while still preserving idioms that characterise the language. It allows static and dynamically typed code to be mixed so the programmer can use whichever is more appropriate.
2019:06:17 04:57:58 seancorfield Yeah, there are trade offs. And that's an old article. core.typed has had a lot of work since then.
2019:06:17 04:58:17 seancorfield We also tried it back then and gave up for all the same reasons CircleCI gave up.
2019:06:17 04:58:39 seancorfield But the question you're asking is best answered by core.typed rather than spec.alpha
2019:06:17 10:46:42 djtango that said - Racket was able figure out a solution to higher-order functions with their run-time contracts system, that also doesn't involve doing gen-testing on the input function
2019:06:17 10:47:54 djtango but function equivalence is undecidable so tradeoffs are always going to have to be made in this space...
2019:06:17 12:52:59 drone Typed Clojure seems to be pretty inactive.
While spec isn’t a type system, I could see function arity being part of a contract. E.g., for a function passed to reduce. But I also think the spec designers would argue that function arity errors already exist and solve most of the problem.
2019:06:17 16:02:28 seancorfield Ambrose just successfully defended his thesis work on Typed Clojure. It's still an active project.
2019:06:17 16:10:25 drone There was a flurry of activity last winter, and then other than a few commits to the core.typed.analyzer.jvm project in April (likely in preparation for defense) there has been no activity for seven months
2019:06:17 16:17:19 seancorfield If you've watched any of his talks you know that he has a lot of design work ongoing about dealing with the typed / untyped code boundary. Just because there have been no recent commits doesn't mean there's no recent work -- this is an extremely hard topic area that requires a lot of "hammock" time.
2019:06:17 16:23:57 drone sure, but it’s also just as likely he is moving on from the project after completing his PhD program
2019:06:17 19:59:15 seancorfield Not based on what he's talked about at conferences.
2019:06:17 22:11:12 drone that’s great if he’s still working on it. but based on observable output, including: repo commits, the project’s twitter account, his patreon, and activity in #core-typed; not much is going on in the project and that’s what I meant by “inactive”. this seems like a weird thing to argue about…
2019:06:17 12:47:46 holyjak Is there a shorter, nicer way to do this (without nesting if in let)?
2019:06:17 13:32:45 ghadi @holyjak don't do that because you'll be putting a ton of data in the exception message (via explain-str)
2019:06:17 13:33:08 holyjak hm, good point, thanks!
2019:06:17 13:37:04 holyjak maybe I should just use (-> explain-data ::s/problems first (dissoc :val :value)) instead?
2019:06:18 23:01:52 misha implementing this as macro has an advantage of being able to include calling form into error data/message, which saves a lot of time for exceptions purposed for developer and repl.
or just have an error-msg arg in a second arity, with default message like "does not conform to spec <spec form>" because you'll expand exception data in repl anyway, because "first problem" is not always "the only problem"
2019:06:19 16:16:25 holyjak Thanks!
2019:06:19 14:42:51 hmaurer o/. Quick question: is the new way of handling optionality in clojure spec (with selections etc) already usable?
2019:06:19 15:14:29 drone like nilable? or do you mean :opt and :opt-un in keys?
2019:06:19 15:18:35 seancorfield @mrevelle He means in spec-alpha2. Right @hmaurer?
2019:06:19 15:18:44 drone ah..
2019:06:19 15:18:58 hmaurer @seancorfield I think yes; what Rich Hickey talked about at this autumn’s conj
2019:06:19 15:19:13 seancorfield We have a branch at work that runs with spec-alpha2, but it’s still very much a moving target right now, and has a number of bugs.
2019:06:19 15:19:33 seancorfield It’s probably fine to “play” with but Alex has advised against using it for any serious work yet.
2019:06:20 07:56:03 Ben Hammond I am calling generate on a complex data structure.
Sometimes I see this error.
I would normally troubleshoot this by commenting out parts of the spec to zone in on the problem; but I'm wondering if there is a smarter way to debug generators
2019:06:20 07:57:18 Ben Hammond is there a direct way to dicover the specific sub-spec that has the problem?
2019:06:20 09:51:12 Ben Hammond okay another question;
I want a generator that has two modes of operation
• when there is data available in the dynamc variable *existing-ids* then I want it to behave like (element *existing-ids)`
• when *existing-ids* is empty then I want it to behave like (spec/gen string?)
this must be a fairly common use case
how has it been solved before?
2019:06:20 09:55:09 Ben Hammond I'm thinking something like
2019:06:20 10:18:20 Ben Hammond I suppose I could store the test.gen/elements in the *existing-ids* var
2019:06:20 10:23:05 Ben Hammond (spec/with-gen takes a gen-fn, and I'd hope that I could use this to choose between generators, but of course it doesn't get called very often
2019:06:20 10:31:32 Ben Hammond this might be better actually
2019:06:20 15:02:45 misha @ben.hammond afair, Gary touched on it debugging "Couldn't satisfy such-that predicate after 100 tries" here: https://www.youtube.com/watch?v=F4VZPxLZUdA
That yields Don't know how to create Iseq from: clojure.lang.Symbol.
2019:06:20 18:27:51 WhittlesJr I'm guessing that macros are the only way to dynamically define specs?
2019:06:20 18:39:11 favila pretty much
2019:06:20 18:39:50 favila I think spec2 has some improvements to this, but still isn't going to let you plop a let in there
2019:06:20 18:40:21 favila I think spec2 will expose a public ast-like interface for programmatic spec creation/manipulation of specs as data
2019:06:20 18:40:30 favila but the surface syntax will still be macros
2019:06:20 18:41:38 WhittlesJr Hmm, alright. I'm interested in the spec2 feature you mentioned so I'll take a look. Thanks!
2019:06:20 19:23:27 seancorfield Spec 2 has a macro layer and a function layer that implements the macros. You can programmatically construct pretty much any spec in Spec 2.
2019:06:20 19:24:26 seancorfield (we have a branch of our code base at work that uses Spec 2 -- so we can track the sort of differences we'll need to make when we transition from Spec 1)
2019:06:21 07:48:49 Ben Hammond following on from yesterday; I think the tidiest way to achieve
> generator uses id from a database,
> when there is a database
is to use the overrides feature of (spec/gen)
(doc spec/gen)
-------------------------
clojure.spec.alpha/gen
([spec] [spec overrides])
Given a spec, returns the generator for it, or throws if none can
be constructed. Optionally an overrides map can be provided which
should map spec names or paths (vectors of keywords) to no-arg
generator-creating fns. These will be used instead of the generators at those
names/paths. Note that parent generator (in the spec or overrides
map) will supersede those of any subtrees. A generator for a regex
op must always return a sequential collection (i.e. a generator for
s/? should return either an empty sequence/vector or a
sequence/vector with one item in it)
=> nil
2019:06:21 11:40:03 djtango thanks for this btw - am enjoying your writeup! Shame it'll be gone whenever the history runs out
2019:06:21 15:03:20 vemv I was thinking that a spec/not could make sense as exemplified here
2019:06:21 15:04:33 vemv (I guess the generative part would suffer)
2019:06:21 15:14:33 alexmiller I've written a ton of specs and have never needed an s/not
2019:06:21 15:15:52 vemv I've also written a fair number of specs for a few years. Just now I think of not, so that kind of proof is limited
2019:06:21 15:16:03 alexmiller you could do something like (spec/def ::stable-version #(spec/invalid? (spec/valid? ::alpha-version %)))
2019:06:21 15:16:30 alexmiller it is probably inherently difficult to auto-gen
2019:06:21 15:17:10 alexmiller in any case, we have no plans to add it
2019:06:21 15:17:54 vemv 👍 this was more curiosity than anything else. One could always write his own not anyway
2019:06:21 15:39:43 Ben Hammond I've got quite an interesting situation;
I have a spec that generates a fairly complex data structure
I sometimes want to overide some of the generators to make it use foreign keys from the database
but it only works intermittently:
I have hooked up a snitching ILookup to tell me what is happening around
and what I see is that
When a recompile that spec directly to REPL, then the overridy will wok
When I recompile the entire namespace to REPL, it does not work
Which is intriguing behavour
2019:06:21 15:40:12 Ben Hammond so this my ILookup
(reify ILookup
(valAt [_ k]
(println (str "valAt*1: " k))
(println (str "==>" (get m k)))
(get m k))
(valAt [_ k nf]
(println (str "valAt*2: " k ":" nf))
(println (str "==>" (get m k nf)))
(get m k nf))
)
2019:06:21 15:41:08 Ben Hammond when it works, I see
2019:06:21 15:42:12 Ben Hammond is this ringing any bells?
2019:06:21 15:45:24 alexmiller specs compile in their dependent specs so if you modify a spec, you need to reload any specs that depend on it. that's a likely reason you'd see different results for the two cases
2019:06:21 15:45:43 alexmiller I'd expect "recompile the entire namespace" to give you the more accurate answer.
2019:06:21 15:46:25 alexmiller hard for me to tell from this what the actual problem is though
2019:06:21 15:47:39 Ben Hammond perhaps I have misunderstood usage.
I looked at
2019:06:21 16:01:32 alexmiller so test.check.generators expects something different than clojure.spec.gen.alpha - namely, the spec version takes generator thunks, whereas I think test.check just takes generators
2019:06:21 16:01:45 alexmiller can you try it with spec's version?
2019:06:21 16:02:33 alexmiller I'm not sure what test.check supports as far as override marking - I'm not sure it even knows about the attributes as that's all spec registry based
2019:06:21 16:03:11 Ben Hammond well it looks like all logic is buried inside clojure.spec.alpha/gensub
2019:06:21 16:04:17 Ben Hammond so perhaps I don't understand what you are asking.
If I run
2019:06:21 16:23:44 Ben Hammond and says
>Ooh a gfn. My work here is done
2019:06:21 16:24:10 Ben Hammond Thankyou for your help
2019:06:21 16:24:27 Ben Hammond I'm not sure how I'm going to fix it
but at least I understand it
2019:06:21 16:27:05 alexmiller Some of this stuff really needs a rethink, it’s pretty tricky
2019:06:21 16:27:48 Ben Hammond well I thought the the next spec counts as a rethink?
2019:06:21 16:28:45 Ben Hammond discovering where the tripwires are is tricky
2019:06:21 16:29:05 Ben Hammond that's true of my entire time with Clojure
2019:06:21 16:29:34 Ben Hammond you have to trip 'em to find 'em
2019:06:21 16:29:41 alexmiller Yeah, we may get to it in spec 2
2019:06:21 16:32:25 Ben Hammond right its the weekend in my timezone
2019:06:21 16:32:32 Ben Hammond have a goodd weekend Alex
2019:06:21 16:50:29 alexmiller later
2019:06:21 18:54:59 misha what is the difference between: (s/coll-of ... :kind vector?) and (s/coll-of ... :into []) ?
2019:06:21 19:32:43 alexmiller kind adds a validation on the input
2019:06:21 19:32:50 alexmiller into is about gen and conform (output)
2019:06:21 19:33:13 alexmiller (but kind also impacts the starting coll for gen/conform)
2019:06:21 19:42:59 misha so :into [] is "list/seq is fine, but generate me a vector"?
2019:06:21 20:03:05 alexmiller Yes
2019:06:21 20:03:21 alexmiller And conform to vector
2019:06:21 19:44:00 misha thanks. had an impression, that :into is a younger replacement for :kind.
2019:06:21 20:02:53 alexmiller No, they are different purposes
2019:06:21 20:35:41 misha what is the best :ret spec for "predicaty" function? boolean? is insufficient because nil is falsey. and because everything but false/nil - is truthy. any?
2019:06:21 20:50:10 alexmiller Depends on what the function returns
2019:06:21 20:50:16 alexmiller Spec the truth
2019:06:21 20:50:48 alexmiller (s/nilable boolean?) is useful sometimes
2019:06:21 20:51:04 alexmiller Don’t use sets for logically false values
2019:06:21 20:55:23 misha function is supplied by user, and is used as predicate, but return value is not compared to true or false.
2019:06:21 20:56:17 misha limiting it to :ret boolean? or :ret (s/nilable boolean?) would be... limiting
2019:06:21 20:58:59 misha what is the best way to spec atom fn arg? #(instance clojure.lang.Atom %)? any??
2019:06:21 21:00:04 misha it actually will be an atom, not just something dereferable. and fn might call swap! or reset! on this arg
2019:06:21 21:49:49 alexmiller on the boolean question, I'd just not spec it then
2019:06:21 21:49:55 alexmiller not spec the ret at all
2019:06:21 21:50:02 alexmiller nothing to check
2019:06:21 21:50:08 alexmiller on atom, I'd do the first one
2019:06:21 21:50:23 alexmiller or actually, use IAtom, the interface, not the concrete class
2019:06:22 21:54:13 carkh i have this spec : (s/cat :tag ...... :children (s/* ::child)
I have a special case where i want a single child, but i still want it to be conformed as a list of one item.... how would i express this ?
2019:06:22 21:56:03 carkh i have the dispatching ok,, been using conformer and it works, but it swallows the explaining of errors down the tree
2019:06:22 21:57:07 carkh so what i would want is a replacement for s/* i guess
2019:06:22 22:57:48 seancorfield s/+ — one or more.
2019:06:22 22:59:09 seancorfield Maybe wrapped in s/and to check the count is 1?
2019:06:22 22:59:32 seancorfield Not sure how s/and combines with the sequence regex stuff tho’… I’d have to experiment.
2019:06:22 23:03:22 carkh oh i can try that thanks
2019:06:22 23:11:07 carkh not working for me but that was a good try =)
2019:06:22 23:12:40 carkh i'm really trying to use spec as a parser for sexp, but it's not actually one, i guess the limitations are due to the gen part
2019:06:22 23:17:56 carkh This works, but the real thing is not doing integers, and the tree may be deep. so finding a syntax error deep down might be hard without proper explaining from spec
2019:06:22 23:28:50 carkh ahhaaa ! this might work
2019:06:22 23:32:09 carkh and i get nice errors deep down the tree, nice
2019:06:23 23:41:47 carkh gonna have to test this before turning out, thanks
2019:06:23 23:43:15 carkh yes that's exactly it, that combinator flew under my radar, thanks @misha
2019:06:25 01:07:08 miguelb Hi everyone, I’m trying to to write a spect that has “relationships” to other specs. (apologies for the language, I’m still new to spec). For example, a group has a total number of people, number of active people and number of inactive people. active-people + inactive-people = total num of people. I can write a spec for the total number of people but how do I use that spec for in the spec for both inactive and active. Also how does that work with generators? Ultimately I would like to generate a group where total, active and inactive are set.
2019:06:25 01:18:56 seancorfield @miguelb Are these specs all used together in a map?
2019:06:25 01:19:25 miguelb yea eventually, right now i’m defining each with their own s/def
2019:06:25 01:20:06 miguelb not sure if this is the right way to go about it, define each part and then compose together
2019:06:25 01:20:45 seancorfield You can only apply relationship predicates to something that contains all the various related keys.
2019:06:25 01:21:40 seancorfield However, I would question the model design: since that's an invariant that should always hold, you don't need all three values (and probably should not try to have all three). Any two gives you all the information you need.
2019:06:25 01:22:03 miguelb good point
2019:06:25 01:22:18 miguelb my end goal here is to make a generator that will generate groups
2019:06:25 01:22:34 miguelb I was aiming to have the person count part of a “valid” group
2019:06:25 01:22:55 ghadi you'll end up needing to manually control the generator
2019:06:25 01:23:22 ghadi the default spec generators will not find something that satisfies the invariant within 100 tries
2019:06:25 01:23:46 seancorfield Yes, that's another reason that I think it would be easier without the constrained total-people number
2019:06:25 01:24:22 seancorfield If you just have active and inactive, then regular generators will work
2019:06:25 01:24:33 seancorfield (since there's no need for an additional constraint)
2019:06:25 01:24:51 miguelb I’ll try that way, makes way more sense that what I was about to try
2019:06:25 01:24:53 miguelb ty very much!
2019:06:26 08:14:25 kwrooijen Hi, when instrumentation is enabled and I call a function with from arguments within a def, instrumentation doesn’t trigger an error. What’s the reason for this?
Here’s an example:
2019:06:26 08:14:33 kwrooijen
2019:06:26 08:15:16 kwrooijen When running inc-and-format, when it’s defined as a function, instrumentation crashes
When running inc-and-format, when it’s defined as a def, instrumentation doesn’t crash
2019:06:27 11:39:15 holyjak Is it possible to "and the rest of the arguments should match this others spec"? Example:
The thing is that I either have a short input or something+the short input. How to spec this without copy&paste? Thank you!
Answer: s/cat do not nest, they become one so the code above actually works as desired:
2019:06:27 12:29:31 alexmiller the more accurate way to look at it is that (s/+ int?) is a regex op (which is not actually a spec)
2019:06:27 12:29:39 alexmiller regex ops combine
2019:06:27 12:29:53 alexmiller when used, regex ops are automatically "spec-ized"
2019:06:27 12:30:06 alexmiller s/spec is an explicit "spec-ize" step
2019:06:27 12:30:55 alexmiller in all cases, regex ops are matching the elements in a sequential collection though
2019:06:27 14:26:17 misha That’s what I meant kappa
2019:06:27 14:30:56 alexmiller I realize that, it's just the wrong mental model
2019:06:27 16:43:25 jeroenvandijk Is it a good practise to use keyword inheritance for multi-spec dispatch? e.g.
(defmulti example-config :entity/type)
(s/def :entity/type #(contains? (set (methods example-config)) %))
(defmethod example-config :default [_]
(s/keys :req [:entity/type]))
(s/def :entity/config (s/multi-spec example-config :entity/type))
(defmethod example-config :entity.type/generic [x]
(getx x :entity/type))
;; Use keyword inheritance to get to the right spec
(derive :entity.type/specific :entity.type/generic)
(s/def :entity.type/specific integer?)
(s/explain-data :entity/config {:entity/type :entity.type/spfecific})
2019:06:27 17:56:53 alexmiller sounds fine
2019:06:27 17:57:32 alexmiller not sure if there are any gotchas in gen, but might be
2019:06:27 18:33:27 jeroenvandijk Thanks, I'll give it a go. Removes some boiler plate code
2019:06:28 10:27:52 Daniel Hines (reposted from #clojure)
I've got a problem I think spec would be perfect for, but I'm a total spec noob and have no idea how to spec it.
I'm using pseudo set notation to describe the problem (read "m E S" as "m is a member of set S").
---
Given:
There's a set of types, T,
There's a set of attributes, A,
There's a set of specs S
There's a map such that every attribute has a corresponding spec from S.
What is the spec for 2-tuples where:
- The first element R1 is a vector of where every element e E T.
- The second element R2 is a set of tuples in the form [e a v], where e E {1..length of R1}, a E A, and the value of v conforms to the spec corresponding to a.
Any help is appreciated!
2019:06:28 13:02:47 djtango so the first spec seems easy enough:
(s/def ::T #{:t1 :t2 ,,,})
You could map A and S manually by doing:
(s/def ::attribute-that-is-a-number number?
,,,
If you need to make the set of A to be concrete you'd probably want to look up the registry
Then the final spec for your 2-tuple probably involves quite a hairy predicate (remember that all specs are 1-arg preds)
This could then check that every first-element of the R2 is is a member of the set of ints from 1 up to (count R1)
Then a spec that looks up value of a in R2 and does (s/valid? a v)
2019:06:28 13:02:56 djtango hopefully that's enough to get the brain flowing
2019:06:28 13:08:38 Daniel Hines Lol, why did I have to pick this as my first venture into specs 😛. Let’s try making it more concrete.
2019:06:28 14:35:58 djtango R1 is a vector right? not an int
2019:06:28 14:37:16 djtango if you don't think about ::Goal as a spec but more a function that takes the whole 2-tuple what would that function look like
2019:06:28 15:16:31 Daniel Hines Yeah, you’re right, :R1 is more like (s/col-of ?int).
2019:06:28 15:17:49 drone and ::R2 is (s/coll-of ::R2-elem :kind set?)
2019:06:28 15:24:18 Daniel Hines Yeah, technically, t
2019:06:28 15:24:39 Daniel Hines There’s more to it though, right @UCF779XFS? I need some predicate function that @U0HJD63RN was alluding to.
2019:06:28 15:26:38 drone you’d need to a predicate that will take the R2/a and verify the R2/v passes s/valid?
2019:06:28 15:30:06 drone so you’d have something like:
(defn valid-val?
[[idx a v]]
(s/valid? a v))
(s/and (s/cat _ _ _ _ _ _)
valid-val?)
2019:06:28 15:30:33 drone something like that/ where the empty cat is whatever you’re using for the three values in the R2 element
2019:06:28 15:30:53 Daniel Hines Ok.. this is starting to come together!
2019:06:28 15:31:06 drone take with a grain of salt, just woke up and may be stupid
2019:06:28 16:44:34 djtango well from the description of the specification - R2's permissible values for e depends on R1
2019:06:28 16:44:37 djtango so you need the whole 2-tuple
2019:06:28 16:45:02 djtango so you would want a predicate that can compare R1 and R2
2019:06:28 16:45:21 Daniel Hines Yeah, that's right.
2019:06:28 17:01:40 Daniel Hines I’ll give that a shot later, thanks!
2019:06:28 17:23:09 djtango so I guess you could split the spec for your tuple into three checks:
That R1 is valid
all e are valid wrt R1
all v conform to their corresponding a
2019:06:28 17:25:41 Daniel Hines Yes! That sounds very doable
2019:06:28 17:26:36 Daniel Hines I was hoping to get generators for free, but even besides this, there are other constraints that between entities of different types that are much too complicated without some constraint solver.
2019:06:28 12:08:39 salokristian I'm working on creating a spec-based validator for creating human-readable error messages. I have a working implementation which locates errors based on the in key in the returned spec errors for explain-data. This works fine for errors concerning leaf-level specs, i.e. entries in a collection.
However, I have not yet found a way to create useful error messages for custom predicates that concern a whole collection. Is there a (built-in or custom) way to return some sort of metadata from a custom spec predicate, which could be accessed in the spec error message?
For example, the error message for this spec is not useful for locating the error, i.e. the duplicate values.
I would like to be able to return some custom metadata from (apply distinct? %), which would tell me which elements caused the error, and allow to locate the errors precisely. Of course this is not just an error with this particular custom predicate, but all custom predicates. There doesn't seem to be a way to add metadata to the error messages generated by spec.
Is this doable? If it isn't, what kind of a workaround would you suggest for this? I would like to use spec for checking correctness and generate errors based on spec errors.
2019:06:29 06:07:35 salokristian I'm actually using phrase to generate end-user friendly errors. The problem is passing additional error-related metadata from custom spec predicates to spec errors (and therefore to phrase).
2019:07:01 09:16:27 djtango I'm not convinced you can get helpful error messages from the general form of (s/def ::the-name #(apply distinct? %)) - if you consider that the spec is a one-arg predicate function, it would be hard to see how to determine which entry caused the failure.
For the specific case of duplication, you'd probably need to replay the data or reconstruct it - as spec would only recall the input unless you walk it element by element. Even the inbuilt :distinct field in (s/every ...) doesn't seem to know which specific member breaks uniqueness
2019:06:28 15:42:46 Daniel Hines Can test.check reverse regex specs out of the box?
2019:06:28 15:47:41 nwjsmith No, you’ll need test.chuck for that
2019:06:28 15:47:45 nwjsmith oh.
2019:06:28 15:47:49 nwjsmith regex specs
2019:06:28 15:47:58 nwjsmith I don’t think test.check is spec-aware
2019:06:28 15:49:36 Daniel Hines Is there a built in spec for UUID’s that works in CLJS?
2019:06:28 15:49:45 Daniel Hines Just need the strings.
2019:06:28 15:56:22 nwjsmith Do you need a spec for string-encoded UUIDs, or would uuid? work?
2019:06:28 16:47:01 Daniel Hines The former.
2019:06:28 17:43:53 drone there was a whole thread about that somewhere. a Google search would probably find it
2019:06:28 17:50:38 Joe Lane Does that handle all UUID versions?
2019:06:28 17:53:35 nwjsmith It should, but IIRC test.check.generators/uuid will only generate version 4 UUIDs
2019:06:28 17:54:29 nwjsmith The textual representation of UUIDs is the same across all versions
2019:06:28 17:56:54 Daniel Hines Thanks a bunch!
2019:06:28 19:38:15 Daniel Hines Are there any spec power tools for defining large entity maps? Not really sure what I’m looking for - just something more concise if possible.
2019:06:29 01:45:27 alexmiller A value that is invalid according to its spec should always produce explain data. Would be interested to know the spec/value causing this
2019:06:29 18:20:24 Daniel Hines Can anyone offer any advice on how to transform a json object into a proper Clojure object based on a given spec? The final result has a mix of keywords and strings, and I’m not sure how to say “Here’s a spec of the end result, and here’s the JSON, now give me the end result”
2019:06:30 01:46:27 Daniel Hines s/valid? also returns false.
2019:06:30 01:50:01 Daniel Hines Oh, it’s because the invocation was erring out 😕
2019:06:30 01:51:00 Daniel Hines For s/explain-str to return “Success!\n” when an exception is thrown seems like a bug.
2019:06:30 01:52:40 alexmiller It is typically. What’s the spec and the value?
2019:07:01 17:18:23 Daniel Hines I tried reproducing it and couldn’t figure out exactly what I did (I was working at the REPL). If I run into it again I’ll try to find a solid repro.
2019:07:02 01:27:42 seancorfield Updated our code at work to use 0.10.0-RC1 and to switch away from the deprecated generators -- and we were using pos-int on the assumption that it generated strictly positive numbers (although zero doesn't break our tests).
2019:07:02 10:57:20 gfredericks that makes me glad I deprecated it 🙂
did you switch to (gen/fmap inc gen/nat), or something else?
2019:07:02 12:58:04 seancorfield For now we just switched to gen/nat to preserve the behavior but we will revisit that test and see what might happen if it ever generates zero. At least it is clear now!
2019:07:02 12:59:58 gfredericks I can assure you it generates zero quite often 🙂
2019:07:02 13:08:06 seancorfield That test should probably fail with zero (but it doesn't) 😕
2019:07:02 13:28:07 gfredericks computers are crazypants
2019:07:01 16:46:17 seancorfield @gfredericks What's the thinking behind deprecating those generators?
2019:07:01 19:29:27 gfredericks pos-int and neg-int are highly misleading w.r.t. 0, and all five of them are misleading w.r.t. size; I think generators by default should generate a healthy range of things, and if you want to restrict yourself to small numbers you should have to be explicit
2019:07:01 19:30:06 gfredericks so it's basically entirely a naming issue
2019:07:01 19:31:01 gfredericks spose I could've also deprecate-renamed gen/ratio to gen/small-ratio
2019:07:01 20:00:22 seancorfield Are you actually planning to remove them at some point (a breaking change) or just "strongly discourage" their usage?
2019:07:01 20:04:18 gfredericks the latter
2019:07:01 20:04:40 gfredericks there's already another deprecated generator of that sort
2019:07:01 20:05:04 seancorfield I found 15 occurrences of "deprecated" in that file 🙂
2019:07:01 20:06:25 seancorfield Naming is hard. I struggled with deprecation/breakage in clojure.java.jdbc and ended up breaking the API twice to get it consistent and idiomatic. With hindsight I would have done it differently.
2019:07:01 20:19:51 gfredericks yeah I've tried to figure out how to overhaul things so many times, and largely haven't
2019:07:01 18:11:36 rapskalian Does spec have a simple way to express xor with map keys? Meaning map m can have key :a or key :b, but not both.
2019:07:01 18:15:26 alexmiller Not built in, but you can s/and any predicate you like
2019:07:01 18:16:24 akiel Has someone a solution for using st/instrument with :stub were the function is called more than once and I like to have return values depending on the inputs? With Java mock frameworks such things are possible. With spec I don’t have an idea how to do it. Thanks!
2019:07:01 18:18:24 alexmiller You can’t with stub, but you could make a stateful :replace fn
2019:07:01 18:27:50 akiel @alexmiller Work perfect. Thanks! Not as elegant like mock libs but one could build a helper function which maps inputs to outputs.
2019:07:02 08:54:57 roklenarcic Is there a way to get around the limitation of s/merge that it only conforms one branch of merge?
2019:07:02 08:55:16 roklenarcic This is really really really detrimental.
2019:07:04 07:09:03 holyjak Hello! I get some kind of infinite loop in conform after I have renamed and moved around a couple of spec names, any tips how to troubleshoot it? It seems deep-resolve and reg-resolve are involved in the loop as well as a particular spec (`:my.ns/account-invoices`) and if the debugger is correct then we regularly arrive to a point in reg-resolve where both spec and k are null.
Update: I can replicate - calling (reg-resolve :my.ns/account-invoices) never returns.
(Clojure 1.10.1, spec 0.2.44)
This is really weird - all parts of the spec resolve but not the spec itself. I have this:
and both (reg-resolve! :my.ns/acc+invoice) and (reg-resolve! :my.ns/accid) return a spec successfully yet (reg-resolve! :my.ns/account-invoices) never returns....
If I change the name of the spec to :tmp/account-invoices then (reg-resolve! :tmp/account-invoices) succeeds.
SOLVED: This line little more down was the issue (s/def ::account-invoices ::account-invoices) I wish Spec had a loop detection...
2019:07:06 13:23:29 misha @roklenarcic can you give an example, where/how it limits you?
2019:07:06 15:02:01 roklenarcic Let's say you define (s/def ::date-time (s/conformer to-date-time-string from-date-time-string)). This conforming spec will print date time object into a string (or vice versa on unform call). If you define several s/keys specs each containing some fields of this type and then you defined a merged spec with s/merge you will end up with having a situation where after conforming an object only a third of date time properties will be conformed to strings the rest will remain datetime objects and it will fail.
2019:07:06 18:22:01 misha This sounds reasonable, although alexmiller stated many times that conformers are not for (this kind of) data transformation.
2019:07:06 17:09:22 Diego Melendez how do I write a s/fdef for a function that does not receive args?
2019:07:06 17:22:15 holyjak I guess leave out args or eg (s/coll any? :count 0)?
2019:07:06 17:22:33 holyjak Or just empty?
2019:07:06 18:00:52 Diego Melendez 👍
2019:07:06 19:07:07 alexmiller I use (s/cat)
2019:07:08 08:19:13 Stefan Hi, I’d like to write a spec for an entry in a map that must always be a specific string, and also can be generated. So I have {:my-key "my constant string"} (as part of a bigger structure). A spec like this works:
(s/def ::my-key (partial = "my constant string"))
Now I try to generate data inside my test, and this fails. I’m trying to use s/with-gen, but I think I’m missing something.
2019:07:08 08:38:47 Stefan @U1UQEM078 Thanks! Why is that preferred?
2019:07:08 08:39:15 aisamu Because then you get the generator for free!
2019:07:08 08:39:17 Stefan Oh wait that’s ALL the code I’d need 🙂
2019:07:08 08:39:29 Stefan Much simpler indeed 🙂
2019:07:08 08:56:00 valerauko wow every day i learn something new
2019:07:08 10:15:28 misha yeah, it is idiomatic to spec constants with a single element set. however, if that constant is either false or nil – use false? and nil?, because
2019:07:08 10:26:58 misha nil? false? instead of set
2019:07:08 10:34:57 yuhan that's an edge case to look out for in general when using sets as predicate functions, which is really all that spec is doing here
2019:07:08 10:37:34 yuhan
(if (#{1 2 3} x) ;; returns truthy values (1, 2, or 3) if x is in the set
...)
(if (#{false} x) ;; returns a falsey value no matter what x is!
...)
2019:07:08 10:43:33 misha it is
2019:07:08 10:48:27 misha can be worked around with contains? though
2019:07:09 13:45:08 arohner I have an s/keys with a key of type uuid?. I have a generative test where I generate a few hundred records, and insert them into postgres, where the uuid uniqueness constraint is checked. Occasionally the test fails due to duplicate uuids. Is there a way to tell spec/test.check to only generate unique uuids?
2019:07:09 14:41:14 ghadi there are ways of generating unique data within spec/t.c. @arohner but for this usecase I would dedupe between generation and inserting records
2019:07:09 15:04:30 arohner Thanks
2019:07:09 15:39:03 kenny It seems useful to have s/with-gen be passed the generator for the spec it is overriding. Often I find myself generating values from the spec's gen and then fmap'ing to ensure certain constraints are met.
2019:07:09 15:56:36 kenny It'd also be useful to have a variant of s/and that doesn't pass the conformed value to s/and predicates. We have a lot of this in our specs:
2019:07:09 17:04:58 kenny Yeah I suppose I could do that. It'd be great if there was a tighter integration between spec and test.check so I don't need to do that.
2019:07:10 14:23:02 gfredericks The second two args are new
2019:07:10 14:24:58 gfredericks @arohner the uuid generator is fully random with a uniform distribution, so you shouldn't be any more likely to get collisions than you are in Real Life, i.e. it shouldn't ever happen
2019:07:10 14:25:26 arohner What about during shrinking?
2019:07:10 14:25:41 gfredericks My guess is you're seeing the effects of shrinking, where your test gets run repeatedly with similar data
2019:07:10 14:26:27 gfredericks So the UUIDs won't shrink, but your test is run again, so if you have state in a database that you're not resetting between trials, that'd be a problem
2019:07:10 14:31:19 gfredericks I agree it's annoying, because UUIDs otherwise seem like such an effective way of implicitly partitioning a stateful external system by trial
2019:07:10 14:32:00 gfredericks I'm curious whether, in practice, blasting lots of data into a test DB w/o deleting it is faster than truncating between trials
2019:07:16 12:52:10 eoliphant Hi, I’m trying to figure out how to model this with spec.
Say I have (s/def ::rule-type #{:rule1 :rule2}) I then want to create (s/def ::rule1 (s/keys ...), etc where aside from the keys that are selected ::rule-type is ‘narrowed’ to specific value, such that I can then have a ::rule spec that’s an or of those guys
2019:07:16 12:54:44 gfredericks might be what multi-spec is for?
2019:07:16 12:55:11 alexmiller yes, seems like exactly what s/multi-spec is for
2019:07:16 13:00:27 eoliphant ah ok, actually just got it working with and s/and ‘passthru’ but will try the mult-spec as well
but yeah multi-spec seems a bit more elegant
spec2 looks exciting @alexmiller
2019:07:16 13:20:42 jeroenvandijk It also gives much better feedback (more specific) when using with expound for instance
2019:07:16 13:02:05 alexmiller s/multi-spec is also open so you can add more rules later
2019:07:16 13:04:34 eoliphant sweet. Hey is spec2 going to have fspec support for mutimethods?
2019:07:16 13:05:39 alexmiller not something we're working on right now, but maybe eventually
2019:07:16 13:18:22 eoliphant cool
2019:07:17 14:30:43 talgiat Is there a way to reuse fspec in clojurscript? the only way to spec functions in clojurescript seems to use fdef but that can’t take fspec defined somewhere else as an argument.
My goal is to define a function spec in one place and re-use it for all functions that have the same function spec. This can be done in clojure the following way (I believe):
2019:07:17 14:35:49 alexmiller instead of def, use s/def
2019:07:17 14:36:54 alexmiller oh, you're trying to reuse the whole fspec as a function's spec
2019:07:17 14:38:16 alexmiller not sure I know enough about the cljs impl to say if this is possible
2019:07:17 14:53:31 talgiat yes that’s what I’m trying to do. But this example should work in clojure (I think without s/def in the first expression)
2019:07:17 15:18:41 alexmiller it won't work exactly like this in spec 2 in Clojure
2019:07:17 15:19:08 alexmiller but there is a similar path
2019:07:17 15:20:09 talgiat I tested and it works both with def and s/def in this example (but with s/def it should be ::n->n-fn-spec …)
2019:07:17 15:21:27 talgiat Looking at clojurescript fdef code it adds the fn-sym to specedvars atom that is used with instrument.
2019:07:17 15:22:18 talgiat so you can’t use s/def in clojurescript to spec function symbols (at least I can’t get it to work)
2019:07:17 15:23:49 alexmiller the fact that it even works in Clojure is somewhat accidental
2019:07:17 15:49:28 talgiat you mean with def or even with s/def, or you mean reusing fspec in general?
2019:07:17 15:50:53 alexmiller being able to use s/def with function spec objects
2019:07:17 15:51:23 alexmiller that won't work in spec 2, but there is a new function s/register that will be available to do the equivalent
2019:07:17 15:51:43 talgiat will it work with def (not s/def)?
2019:07:17 15:52:18 alexmiller that's really incidental - def doesn't do anything with the registry so no change there
2019:07:17 15:52:26 alexmiller the changes are around what s/def does
2019:07:17 15:52:45 talgiat yeah def just binds a symbol to fspec
2019:07:17 15:53:03 alexmiller importantly, an fspec object (not a form)
2019:07:17 15:53:17 alexmiller s/def doesn't work with objects anymore in spec 2, only forms
2019:07:17 15:53:19 talgiat right
2019:07:17 15:54:41 alexmiller but s/register is a new function in spec 2 that s/def (macro) uses
2019:07:17 15:55:20 talgiat I guess my question is, is reusing fspec is a valid use case for spec? or what is the correct way to have reusable function spec?
2019:07:17 15:55:44 talgiat that works in clojure and clojurescript
2019:07:17 15:58:25 alexmiller I can't really answer the latter, but in spec 2 it's possible by registering the same object via s/register. cljs spec 2 work has not been started so tbd
2019:07:17 16:02:02 talgiat Thanks Alex, I’ll chase David for clojurescript
2019:07:17 16:02:30 alexmiller I think they're going to wait until we're much further along with spec 2 before they start working on it
2019:07:17 16:03:33 talgiat It will be great if with spec 2 things will work the same way in Clojure and Clojurescript
2019:07:17 16:04:51 alexmiller well that's certainly the intent (other than things that can't work the same way)
2019:07:23 12:43:26 henryw374 I want to have generators associated with predicates, like you have already with the clojure.core preds. Are people re-binding gen.alpha/gen-builtins or is there some other way?
2019:07:23 13:12:35 alexmiller make a spec with a custom gen
2019:07:23 13:31:45 henryw374 but with string?, I can use that both as a normal pred, and as a spec with a generator. If I make a spec with the pred, that's two separate things isn't it?
2019:07:23 13:34:49 alexmiller yes, but you can just use the spec
2019:07:23 13:38:49 alexmiller the recommended way to do this with spec 1 is to make a spec with a custom generator
2019:07:23 13:39:01 alexmiller future answer may be different
2019:07:23 13:39:30 henryw374 ok thanks for the info.
2019:07:23 13:40:02 alexmiller rebinding gen-builtins from a lib is problematic - users have to do that for it to work, and whatever you do has to be compatible with whatever some other lib might do
2019:07:23 13:40:49 henryw374 yeah... I'll avoid that I think. bound to be problems
2019:07:23 13:40:56 henryw374 no pun intended
2019:07:23 13:41:00 alexmiller heh
2019:07:23 13:41:37 alexmiller spec 2 has some capabilities to create your own derived spec types with custom gens which is maybe another option
2019:07:23 13:43:07 alexmiller but that's really trying to handle the case of parameterized things more so than pred generators
2019:07:23 13:45:19 alexmiller the example at that link doesn't show how to define a custom gen in a defop, but it has that capability
2019:07:23 13:54:50 henryw374 ok cool. I'll have a look at that with an eye to the future.
on the topic of spec2, with select, say I have a schema with a key a the spec of which is a coll-of x or map-of string? x and x is a schema with keys foo, bar. could you do a select so say you want a and in the value of a you want to select key foo from x? I guess not from what I've read so far, but I think that'd be useful.
2019:07:23 14:01:15 alexmiller not there now, but we have been thinking about how to add it
2019:07:23 14:01:23 alexmiller definitely a common case
2019:07:23 14:01:56 henryw374 cool thanks, looking forward to spec2 definitely
2019:07:23 22:03:08 robertfw What is the recommended way to spec a map that uses strings for keys?
2019:07:23 22:05:13 alexmiller it's a little tricky right now
2019:07:23 22:05:20 alexmiller you can spec it as an s/map-of s/tuple for map entries
2019:07:23 22:05:39 alexmiller possible to s/multi-spec the tuple on the key to choose the val spec
2019:07:23 22:32:38 robertfw ok, thanks
2019:07:23 22:32:54 robertfw Will that become easier with spec2?
2019:07:24 01:23:06 alexmiller not sure
2019:07:24 06:40:41 y.khmelevskii Hi gents! Is there any way to use spec/keys definition for creating new spec as set of keys? For example I have spec:
2019:07:24 12:53:50 danieroux How would I generate a random walk with clojure.spec.gen? Or, how would I generate a successive value based on the current value?
2019:07:24 14:08:29 respatialized I think the easiest way would be to use spec.gen for starting values and then use iterate to do the actual random walks. you could use specs inside the fn you pass to iterate to constrain the return values.
2019:07:24 14:10:44 respatialized does anyone have any thoughts on making spec predicates usable through a java API? should I just use spec/conform to define the class methods I want to expose to the Java API?
2019:07:24 14:18:44 alexmiller can you explain more what you're trying to do?
2019:07:24 14:24:08 respatialized I'm using core.spec to define a series of data integrity/quality predicates that some data needs to conform to. I would like for those specs to be available as widely as possible, including for other consumers of the data who may be using Java instead of Clojure.
2019:07:25 04:01:40 jjttjj is there an example out there of a spec on arguments that look like
([a])
([a b])
([a b c])
([a b c d])
([a b c d e])
where later arguments are optional and can be omitted and replaced with defaults? I feel like there has to be a better way than just a big s/alt with a bunch of s/cats but I can't think of it right now
2019:07:25 07:24:32 carkh not exactly what you're asking but it helps
2019:07:25 07:25:57 maxp great!
2019:07:25 15:55:12 jjttjj Is there a way to get the resolved value for a given spec? I have a set I generate as a spec and I'm wondering if there's a way to access that set from the spec api:
2019:07:26 16:00:52 kenny Any good tools for debugging slow generators? We have lots of specs with tens of s/and preds & custom gens associated. I'd like to know which predicate is causing a such-that to fail often.
2019:07:26 16:05:14 gfredericks I bet you could alter-var-root such-that to default to a lower max-tries
2019:07:26 16:05:27 gfredericks looks like spec just uses the default (10) as far as I can tell
2019:07:26 16:05:43 gfredericks so if you change that to, e.g., 2, you might get more failures
2019:07:26 16:06:14 gfredericks but maybe that's not useful if the "telling you which s/and failed" feature isn't wired up yet
2019:07:26 16:07:27 kenny > isn't wired up yet
Is a feature like this on the roadmap?
2019:07:26 16:08:48 kenny Good idea on the such-that max-tries thing. That should help a bit. Though, the such-that errors are usually pretty obtuse, hiding the actual name of the predicate behind an anonymously named test.check pred.
2019:07:26 16:09:00 gfredericks spec's roadmap, yes, as far as I know; that feature couldn't exist prior to the 0.10.0 versions of test.check
2019:07:26 16:09:29 gfredericks > the such-that errors are usually pretty obtuse
that's exactly the problem that the aforementioned up-wiring would address
2019:07:26 16:11:58 kenny Ah, very cool. Debugging these issues has been a huge time sink for us. It seems like it should be a fairly easy change to spec. What feature in 0.10.0 allows this better up-wiring?
2019:07:26 16:16:23 gfredericks from the such-that docstring:
You can customize `such-that` by passing an optional third argument, which can
either be an integer representing the maximum number of times test.check
will try to generate a value matching the predicate, or a map:
:max-tries positive integer, the maximum number of tries (default 10)
:ex-fn a function of one arg that will be called if test.check cannot
generate a matching value; it will be passed a map with `:gen`,
`:pred`, and `:max-tries` and should return an exception
2019:07:26 16:23:35 kenny I see. Would it be easy to hack this in to spec as a temporary fix?
2019:07:26 16:24:06 gfredericks I have no idea, @U064X3EF3 probably knows at least the easy vs hard part
2019:07:26 16:27:30 alexmiller probably not easy?
2019:07:26 16:28:24 alexmiller well, not sure
2019:07:26 16:28:58 gfredericks I'd think you'd want have some kind of path on hand at the point where such-that is called
2019:07:26 16:29:12 gfredericks so might take some wiring to make that path available
2019:07:26 16:29:32 gfredericks or maybe it's already there, since spec likes to put paths in its error messages
2019:07:26 16:29:49 kenny This would be a huge value add for us. If adding/hacking this in is a day's worth of work, it's definitely worth it for us to dive in
2019:07:26 16:29:57 alexmiller it should be - gens have the path
2019:07:26 16:30:18 alexmiller but they aren't using such-that right now
2019:07:26 16:30:31 alexmiller so I'm not sure how this stuff connects
2019:07:26 16:30:43 gfredericks they? this is about s/and in particular I think
2019:07:26 16:31:00 gfredericks I can't imagine the gen for s/and doesn't use such-that
2019:07:26 16:31:57 alexmiller I guess it does, just not directly
2019:07:26 16:32:10 alexmiller that's generically in gensub, which handles the recursion check stuff
2019:07:26 16:33:44 kenny It appears there's only one place that such-that is used in spec, gensub as you say. It also is passed other info that seems valuable to report back.
2019:07:26 16:36:01 alexmiller are you using deps.edn ?
2019:07:26 16:36:08 kenny Yes
2019:07:26 16:36:12 alexmiller if so I could commit something on a branch and you could test it as a git dep
2019:07:26 16:36:19 kenny Sure
2019:07:26 16:37:13 kenny I currently don't have a repro of one of these such-that issues though, just the slow gen thing. The such-that errors are very frequent though
2019:07:26 16:49:27 alexmiller easy to repro with any restrictive s/and pred
2019:07:26 16:49:42 alexmiller my first hack did not yield anything useful though
2019:07:26 16:49:53 alexmiller I need to step away for a bit, but may play with it later
2019:07:26 16:50:15 kenny Ok. Let me know if I can help in any way.
2019:07:26 16:13:56 alexmiller @kenny 95% of the time the issue with slow generators is generating large (and particularly nested large) collections
2019:07:26 16:14:38 alexmiller setting :gen-max 3 in s/coll-of, s/map-of etc is often a big difference
2019:07:26 16:14:59 kenny This structure is highly nested & has lots of colls. We have these opts set:
2019:07:26 16:15:49 alexmiller none of those affect the gen'ed count
2019:07:26 16:16:04 alexmiller with a 3 level coll, you could have 10x10x10 nested elements
2019:07:26 16:16:28 kenny Oh, right - that param is on the actual generator. Lemme see if we have those set.
2019:07:26 16:16:55 alexmiller well, recursion limit might affect the depth (although I think there are some scenarios where it is not getting applied right now)
2019:07:26 16:18:04 alexmiller if you have fspecs, that's another possible issue where you might want to consider simpler preds
2019:07:26 16:18:11 alexmiller like ifn?
2019:07:26 16:19:03 kenny We have considered that a number of times simply due to the massive amount of time debugging generators take. Every time, however, we always decide not to because those fspec generators catch valuable bugs haha
2019:07:26 16:19:33 gfredericks the eternal test.check conundrum
2019:07:26 16:19:40 alexmiller another testing tip is that gens will get run 1000 times in check and size/complexity increases
2019:07:26 16:20:07 alexmiller so gen/sample by default (20) will not expose generator issues
2019:07:26 16:20:17 alexmiller but you can gen/sample with 1000 and will see those
2019:07:26 16:20:38 alexmiller if you separate the args spec for an fdef into its own spec, it's easy to test that
2019:07:26 16:22:01 kenny I searched for :gen-max in this service and didn't find any so it may not be used. Though, there are tons of custom gens which use the gen/* functions which don't take in :gen-max, I think.
So that's the usual way to debug the slow gens -- something like this (do (doall (gen/sample (s/gen ::db/db-control-args) 1000)) nil). Then that will hang forever and that's where the pain starts.
2019:07:26 16:23:18 alexmiller I would try just blindly adding :gen-max 3 to all the collection specs
2019:07:26 16:23:25 gfredericks @kenny what version of test.check are you using?
2019:07:26 16:23:42 alexmiller there is a ticket specifically for changing this default (or exposing a global default) btw
2019:07:26 16:26:45 alexmiller which I am completely failing to find
2019:07:26 16:26:52 alexmiller but I wrote the patch for it, so I know it exists
2019:07:26 16:43:17 kenny The global default would still respect coll-of :min-count, right?
2019:07:26 17:12:18 alexmiller there's actually a bug report about that
2019:07:26 17:13:07 alexmiller so be careful on that
2019:07:26 16:24:24 kenny Will try that.
We're using 0.10.0-alpha3.
2019:07:26 16:24:42 kenny Yeah, that would help to figure out if the problem is collection gen or something else.
2019:07:26 16:25:03 gfredericks okay, that should avoid some exacerbatory issues with the 0.9.0 release
2019:07:26 16:25:32 kenny Haha, yeah. I remember a huge perf bump when switching from 0.9.0
2019:07:26 16:25:43 gfredericks oh good
2019:07:26 16:47:09 kenny Another thing that slows down debugging slow gens is that the check tests still run in the background even though I interrupted the current op in the REPL.
2019:07:26 16:47:34 kenny I often will need to restart the REPL to get it to stop.
2019:07:26 16:47:44 gfredericks is that a pmap thing?
2019:07:26 16:47:59 kenny We're not using that. Does test.check?
2019:07:26 16:50:25 gfredericks I think spec does
2019:07:26 16:50:30 gfredericks but could be wrong
2019:07:26 16:56:23 kenny pmap is in the check impl.
2019:07:26 16:57:21 kenny Maybe a call to shutdown-agents would help?
2019:07:26 17:00:14 kenny Getting a frequencies map returned after generating a spec with s/and would also be super helpful. The keys would be the forms of each s/and pred.
2019:07:26 17:10:21 alexmiller well shutdown-agents will shutdown the pool never to restart again, so you'd still need to bounce your repl
2019:07:26 17:11:01 alexmiller pmap is lazily parallel
2019:07:26 17:11:44 alexmiller so when you stop using the result, up to n-1 (where n is number of processors) may still be working to compute the next few results in the lazy seq
2019:07:26 17:16:46 kenny Hmm. It seems to go on forever. I’ll add prints to some functions to debug the slow gens and it never stops printing after interrupting.
2019:07:26 17:24:42 kenny Perhaps because of the doall in this line? (do (doall (gen/sample (s/gen ::db/db-control-args) 100)) nil)
2019:07:26 17:52:10 kenny This is helpful :
(let [such-that gen/such-that]
(with-redefs [gen/such-that (fn
([pred gen]
(such-that pred gen 1))
([pred gen opts]
(such-that pred gen 1)))]
(gen/generate (s/gen ::db-control-args))))
2019:07:26 17:54:59 kenny Actually, @gfredericks the problem with taking the above approach is simple generators sometimes need the such-that. e.g.
2019:07:26 17:55:28 kenny I suppose you could write a custom gen for an interval.
2019:07:26 19:26:07 alexmiller there already is an interval spec with a generator - s/int-in
2019:07:26 19:26:36 kenny That generates an integer. This one is an actual interval. e.g. [0 10]
2019:07:26 19:26:42 alexmiller ah, I see
2019:07:26 17:58:10 gfredericks @kenny yeah the general rule with such-that is that the predicate needs to be highly-likely to succeed; so for something like an interval you'd want to write your own generator, which is pretty easy
2019:07:26 17:58:43 kenny I mean, it is highly likely to succeed in the above case.
2019:07:26 17:59:42 gfredericks isn't it just 50%?
2019:07:26 18:00:07 kenny I think so
2019:07:26 18:00:12 gfredericks more precisely what you want is "astronomically unlikely to fail 10 times in a row"
2019:07:26 18:00:24 gfredericks and something that only passes 50% of the time doesn't have that quality
2019:07:26 18:00:38 kenny Fair enough
2019:07:26 18:01:10 gfredericks as such-that retries, it's also incrementing size, so often it's good enough that the probability of failure goes down with larger size
2019:07:26 18:01:14 gfredericks e.g., gen/not-empty relies on that
2019:07:26 18:03:46 kenny It almost seems like you should be able to enable a warning that tells you when your specs use s/and and don't have a custom gen.
2019:07:26 18:31:45 kenny Writing the generator for an interval, like the above is tricky without such-that because you don't know how to generate a value that will match the spec. I was originally thinking it'd just be a gen/bind but that doesn't work because I don't know how to generate a ::m/int+ such that the second one is greater than the first without using such-that.
2019:07:26 19:28:14 alexmiller the trick is to generate a starting value and the range size, then gen/fmap the pair from that
2019:07:26 18:36:00 bfabry in the case that you’re going to throw away generations that don’t match your predicate, I would put “reasonable frequency” at like 50%
2019:07:26 18:37:30 kenny That seems right. This would go back to the original problem then where, in practice, overriding such-that doesn't really work as a debugging technique.
2019:07:26 18:42:13 bfabry a test.check generator mode where all of the various predicates print out their form line number and % success rate at the end would be cool (and quite hard to do)
2019:07:26 18:42:30 kenny That's exactly what I want too 🙂
2019:07:26 18:52:10 gfredericks @kenny w.r.t. avoiding such-that, how about fmap with sort?
2019:07:26 19:29:12 alexmiller I would generate the starting value and the range size, then fmap to get [start (+ start size)]
2019:07:26 19:30:03 kenny I don't think that'd work because you'd need to know how to generate the range.
2019:07:26 19:30:48 kenny For example, we have a "prob-interval" where the interval's lower and upper bounds must be between 0 and 1.
2019:07:26 19:32:58 kenny One issue with @gfredericks approach, however, is if you have a strict-interval. You have a chance of hitting the such-that then.
2019:07:26 19:33:44 gfredericks Sort+inc?
2019:07:26 19:34:04 kenny Can't be that simple because of the above interval case.
2019:07:26 19:34:18 kenny "prob-interval", that is
2019:07:26 19:36:02 kenny You could add Double/MIN_VALUE in that case. But if the interval is integers only, that won't work.
2019:07:26 19:36:35 kenny Well, even adding Double/MIN_VALUE may not work because that may go outside the range.
2019:07:26 20:01:02 gfredericks Math/nextUp 🙂
2019:07:26 20:25:13 kenny I think that's still problematic for the same reason though.
2019:07:26 20:26:03 kenny Why isn't generator? public in spec's gen ns?
2019:07:26 20:34:39 alexmiller Might be an oversight, might be a reason it won’t work with dynaload
2019:07:26 20:38:54 kenny Same with vector-distinct-by
2019:07:26 20:40:31 kenny Might be able to add them to our own libs by using lazy-combinators
2019:07:26 20:47:37 kenny Could s/coll-of take a new option for :distinct-by?
2019:07:26 21:01:45 souenzzo I need something like (s/tuple (s/tuple ::a) (s/tuple ::b)), but should allow values like [["a" "d"] ["b"]] (ignore "extra" tuple values)
2019:07:26 21:26:01 souenzzo not sure how many ::tupN is needed.
The fact is: just the 3 first values are required/used. (each one has it own spec)
But sometimes a larger collection if passed as arg.
2019:07:26 21:27:07 kenny Tuple is usually for a list of a known length. Sounds like you really want cat, like your example
2019:07:29 19:45:42 respatialized at risk of asking an obvious question: how do I refer to a spec from another namespace?
here's an example: in namespace-0, I have some specs defined using (s/def ::consistent ...). In namespace-1, I am using (:require [namespace-0)], but making calls to the specs in the other namespace by doing something like (s/valid :namespace-0/consistent data) fails with a Unable to resolve spec :namespace-0/consistent. How do I refer to these specs from another namespace using fully qualified keys?
2019:07:29 19:57:58 respatialized actually, it works as expected using the fully qualified keys. my issue was just a typo in the spec name. 😑
2019:07:29 21:54:59 kenny Is this how :gen-max is supposed to work? It appears to be a bug.
2019:08:01 14:06:56 ben.mumford are there any best practices anywhere on where in the code to define specs? at the moment i'm defining function specs alongside the function they go with and don't have a consistent strategy for where to define the specs (spec/def ...) - usually in a few large files
2019:08:02 16:43:27 ghadi if you have a reproducible example case, please paste it @kenny
2019:08:02 16:44:03 kenny Working on that. It's pretty coupled to this code but I should be able to pull something out.
2019:08:02 16:44:09 ghadi cool -- thanks!
2019:08:02 16:44:48 ghadi this has been reported before, but AFAIK there's not a deterministically reproducible case
2019:08:02 16:45:24 kenny Hopefully I can get one! It's definitely reproducible for me right now.
2019:08:02 16:49:17 alexmiller the error occurs when you try to create an ex-info with nil data. the case where this happens here is when spec validation fails, but explain-data succeeds (which ideally should never happen). usually, this is a bug in spec. occasionally, it's a bad predicate in user code.
2019:08:02 16:50:03 kenny This error is quite confusing. Seems like a bug either way 😉
2019:08:02 16:50:13 alexmiller well the error should never happen
2019:08:02 16:50:28 ghadi right, it's an invariant violation
2019:08:02 16:52:26 kenny haha, the best kind of bugs
2019:08:02 16:53:55 kenny I pulled some code out to get a repro case and was running st/check on it. It failed with a regular specification error. I thought that was weird because it hasn't failed yet. Turned on instrumentation and ran again and got the "Additional data must be non-nil." message. May be related to instrument.
2019:08:02 16:57:57 alexmiller this won't help you, but I've made a change in spec-alpha2 to better report this case
2019:08:02 18:23:45 alexmiller so I misdiagnosed above - even more subtle... s/conform and s/valid? are returning different answers when run on the same input
2019:08:02 18:24:03 alexmiller which is because the spec is an fspec, which generates its own source of random inputs
2019:08:02 18:25:11 alexmiller this is a known issue and generally we've been suggesting people don't use fspec because it has all these unexpected consequences
2019:08:02 18:50:58 kenny fspec has proven to be quite valuable though. What is the alternative? Just fn?
2019:08:02 18:56:45 alexmiller ifn?
2019:08:02 18:57:07 alexmiller unfortunately it's not even easy to override in a spec alias or anything
2019:08:02 18:58:31 kenny Won't that just generate any old function during gen tests and pass it to my function?
2019:08:02 19:07:50 alexmiller yes, it is significantly harder to use it in check. I'm not saying any of this is good.
2019:08:02 19:09:02 alexmiller this is a known area I'm hoping to spend some rethink time on in spec 2, but we're going to overhaul the function return value stuff first
2019:08:02 19:16:52 kenny Oh ok. Any workaround for now to avoid this error?
2019:08:02 19:21:38 alexmiller I don't think I have any simple advice for you, other than to remove your :ret spec :(
2019:08:02 19:25:00 kenny :ret on the fspec of the top level fn?
2019:08:02 19:27:05 alexmiller no, the :ret on the fdef
2019:08:02 19:27:12 alexmiller that is an fspec
2019:08:02 20:10:01 kenny I was trying to add some gen improvements to clojure.spec.alpha but I cannot load the ns in a REPL. I get
Is there some magic that needs to happen to have a REPL with Spec itself?
2019:08:02 20:49:46 y.khmelevskii hi gents! How I can define spec of vector where I know only first element and don’t know length of vector. For example [:in 1 2 3 4]
2019:08:02 20:56:56 y.khmelevskii it would be great to use something like
Causes ClassCastException: clojure.lang.MapEntry cannot be cast to java.lang.String when used in an or spec.
If I change the catch to Exception I get a failed validation with
Part of the value
{:post/updated "2017-05-05T11:55:00.000Z", ...}
when conformed as
[:str-date-time "2017-05-05T11:55:00.000Z"]
used as :req in a s/keys form.
Any thoughts? Thanks
2019:08:03 18:21:18 seancorfield @hjrnunes That sounds like you're passing a conformed value into str-date-time?
2019:08:03 18:23:10 hjrnunes Yes it does. But it's utterly puzzling. I'm merely calling s/valid? on a map
2019:08:03 18:23:48 hjrnunes Does s/or conform values?
2019:08:03 18:23:59 seancorfield The result of s/or is a tagged pair, yes.
2019:08:03 18:24:14 hjrnunes so, I should cater for this in the predicate?
2019:08:03 18:24:25 seancorfield No.
2019:08:03 18:24:47 seancorfield I need to see more of your specs to see where you're flowing a conformed value into that spec.
2019:08:03 18:24:55 hjrnunes actually, I have the same result with string? or int?
2019:08:03 18:25:46 seancorfield Show the context.
2019:08:03 18:26:45 hjrnunes yes, I'm trimming stuff down. Thanks for the help
2019:08:03 18:27:34 seancorfield Do you have s/and somewhere? That's often where a conformed value ends up flowing into another predicate.
2019:08:03 18:30:53 hjrnunes Yes I do
2019:08:03 18:30:57 hjrnunes
(s/def :post/published (s/or :date-time ::tspec/date-time
:str-date-time str-date-time?)) ;; should be DateTime or :date-time format
(s/def :post/updated (s/or :date-time ::tspec/date-time
:str-date-time str-date-time?)) ;; should be DateTime or :date-time format
(s/def ::post-found (s/keys :req [:post/published
:post/updated]))
(s/def :post/missing coll?)
(s/def ::post-missing (s/and ::post-found
(s/keys :req [:post/missing])))
2019:08:03 18:31:50 seancorfield You probably want s/merge not s/and there.
2019:08:03 18:32:41 hjrnunes Hmm, perhaps. So the idea is to re-use ::post-found and add another key - which is what actually happens to the data
2019:08:03 18:32:56 hjrnunes so s/merge is the right way then?
2019:08:03 18:33:02 seancorfield That's what s/merge is for -- to blend two s/keys specs.
2019:08:03 18:33:55 hjrnunes I see, I read somewhere, the mailing list I think, that s/and would work. Perhaps it was in a previous version
2019:08:03 18:34:51 hjrnunes Yep. That does it. Thank you very much!
2019:08:03 18:35:44 hjrnunes BTW, I listened to the defn podcast you were on recently - very interesting! Best of luck and thanks for all the work!
2019:08:03 18:39:05 seancorfield s/and is specifically for flowing a conformed value into another spec. For example, you might s/and one s/keys-based spec and then a predicate that checks two keys have distinct values perhaps.
2019:08:03 18:39:38 seancorfield s/merge is for extending an s/keys-based spec with additional key specs -- like merge for maps.
2019:08:05 05:27:41 akond explain-data produces a list of problems, but i've only seen a single item in it. can somebody show a case when the :problems vector contains more than one item?
2019:08:06 04:23:11 dorab Why does the sg/generate work in one case, but not the other?
2019:08:06 04:54:30 seancorfield @dorab :kind should not be a spec, it should be a predicate
2019:08:06 04:54:51 seancorfield If you define ::str-vec with :kind vector? it works just as expected.
2019:08:06 04:55:51 seancorfield If you read the docstring for s/every (which applies to s/coll-of), it says
:kind - a pred that the collection type must satisfy, e.g. vector?
(default nil) Note that if :kind is specified and :into is
not, this pred must generate in order for every to generate.
2019:08:06 04:56:49 seancorfield Predicates can be treated as specs, but specs are not predicates (in general).
2019:08:06 15:29:35 dorab Thanks
2019:08:09 03:42:33 valerauko is there any roadmap for spec 2? we plan to add spec validations to our codebase and if spec 2 is reasonably close we'd go straight with that
2019:08:09 03:49:21 alexmiller roadmap in terms of things to do or timing
2019:08:09 03:49:40 alexmiller I guess you mean timing
2019:08:09 03:50:06 alexmiller I would not expect it to be "done" for at least another month or two
2019:08:09 03:51:13 valerauko we could work on adding it from this fall (october <) so that sounds promising to me
2019:08:09 03:51:30 alexmiller well, you might be fine then
2019:08:09 03:51:52 alexmiller you can use it right now as a git dep, but it is definitely a work in progress
2019:08:09 03:54:30 seancorfield @vale We have a branch of our code base running on Spec 2. There are definitely some interesting differences from Spec 1, and if you do anything where you need to share specs between 1 & 2 (perhaps because you use specs from a library that depends on Spec 1), you'll need to jump through some hoops.
2019:08:09 03:55:14 alexmiller I've landed two big sets of changes in the last week there, will have a writeup on some new stuff tomorrow
2019:08:09 03:56:35 seancorfield (just in time for me to go to England for a week so I won't get to play with it until I get back!)
2019:08:09 03:57:21 valerauko i heard you mention that on the defn podcast, but i'm not adventurous enough to use it in production code
2019:08:09 03:58:09 valerauko we're adding spec from zero, and that 1/2 difference thing is exactly the reason i'd prefer to go straight to 2 if it's feasible
2019:08:09 04:01:53 valerauko on a similar note, any plan on making it not-alpha?
2019:08:09 04:11:03 seancorfield Just reviewing those commits @alexmiller -- love the new closed spec approach!
2019:08:09 04:11:33 seancorfield @vale Oh that branch isn't in production -- just dev -- until Alex green lights production use (even in alpha).
2019:08:09 04:12:41 seancorfield We started using Spec 1 in early alpha in production. We take most Clojure alpha builds to production. But Spec 2 isn't yet at that point.
2019:08:09 04:13:45 seancorfield Based on past experience, I'd expect Spec 2 to stay alpha for quite a while before it moves to a beta. tools.deps is still alpha and it's core to the CLI / deps.edn tooling.
2019:08:09 04:33:13 alexmiller yeah, we're still actively changing the api in spec 2 and I don't think that's done yet. we're not going to have a "release" until that settles down
2019:08:09 04:34:14 alexmiller my hope is that it can be stable enough to be non-alpha and so it will never release as spec-alpha2 at all, but will depend on timing
2019:08:09 04:34:57 alexmiller tools.deps edges ever closer to me being comfortable with taking it off alpha
2019:08:09 04:35:16 alexmiller the api is pretty stable there
2019:08:09 04:43:39 seancorfield @alexmiller Good to hear that about Spec 2. Targeting "release" by Conj?
2019:08:09 04:43:53 seancorfield Re: tools.deps -- any decision on add-lib yet?
2019:08:09 05:41:00 valerauko oh nice! it could be tricky to explain to stakeholders how something labelled alpha is production ready
2019:08:09 05:46:59 seancorfield @vale Stakeholders outside the Clojure world, yes. But inside the Clojure world there's much more stability than almost any other language.
2019:08:09 05:47:47 seancorfield We went to production originally, back in 2011, on Clojure 1.3 Alpha 7 or 8, and we've run almost every single pre-release build of Clojure in production ever since.
2019:08:09 05:49:26 seancorfield Quite a few Contrib libs have been in pre-release mode for ages on various revisions and it's "expected" that folks will use them, reporting any problems, but on the assumption that they're "solid". There is, after all, a mindset -- driven from the top down -- of accretive/fixative change rather than breakage.
2019:08:09 05:50:50 seancorfield (that said, some libs have had breaking changes several times -- including clojure.java.jdbc that I have maintained for about eight years!)
2019:08:09 08:41:15 dharrigan upvote for add-lib in tools.deps please 🙂
2019:08:09 11:37:22 ikitommi @alexmiller Closed spec looks so much better now (also read the commits)! As the settings open up more options to do things, it would be great if you would time to re-check the proposed patch in https://clojure.atlassian.net/browse/CLJ-2116, as it had similar approach, but I believe, would be even more generic way to do this, for even to apply coercion. Or if that’s an bad idea, that should be declined as it’s now partially overlapping with the new design.
2019:08:09 11:39:37 alexmiller Well, all spec patches in jira are probably long broken given that I’ve rewritten pretty big chunks of it multiple times
2019:08:09 11:40:19 alexmiller Once we get through the majority of new feature type stuff, I will go through them all but no point now
2019:08:09 11:40:57 alexmiller As I’ve said in the past, I think it’s highly unlikely that we’re going to take clj-2116 regardless
2019:08:09 11:43:42 ikitommi I don’t think it’s a good way anymore either (2116), wrote a the spec walker issue later that could be the one generic way to walk over specs + values, and could be used for both.
2019:08:09 11:44:09 alexmiller Well, we’re working on the walking stuff too
2019:08:09 11:44:15 ikitommi currently spec-tools & spec-coerce walk the spec forms which is IMO not a right way to do that.
2019:08:09 11:45:13 alexmiller The other half of what I’m working on now is dataficatiom of specs
2019:08:09 11:45:50 ikitommi cool. just for browsing or also transforming?
2019:08:09 11:46:17 alexmiller Either - the input side is there now
2019:08:09 11:46:53 alexmiller Once you have the output side, then transformation is just normal clojure stuff
2019:08:09 11:48:42 ikitommi one common case for closed specs has been the “strip out extra keys” functionality, is that doable with the new closed feature? e.g. before saving to the database / when reading input from untrusted clients / intenet.
2019:08:09 11:49:59 alexmiller Haven’t really considered it. Why not just use select-keys?
2019:08:09 11:50:26 ikitommi select-keys for a deeply nested structure mean double-maintaining the keysets. some helper would be nice
2019:08:09 11:51:13 alexmiller I think we’re giving you a lot of new tools to build your own spec that does something like this (flag passed through settings) but we have not talked about any plans to add this
2019:08:09 11:53:10 alexmiller This again feels like a coercion/transformation use case which we don’t believe is a job for conform
2019:08:09 11:53:23 ikitommi ok. the data is there (keysets) already to feed a select-keys. would love to throw away stuff from spec-tools, when they are doable with spec itself. currently:
2019:08:10 13:29:08 alexmiller Yeah, the first one should be spec
2019:08:10 13:29:47 alexmiller Do you have an example of what’s not working for you?
2019:08:10 13:30:27 norton Sorry, I do but not a small sample that I can share.
2019:08:10 13:31:07 alexmiller Ok, would be happy to take a look if you can describe
2019:08:10 13:31:54 norton Or … can you share a recipe how I can test out locally by making this change?
2019:08:10 13:39:02 norton The essence of the test is as follows:
(doseq [spec (filter #(contains? STRING-NAMESPACE (namespace %)) (keys s/registry)))]
(let [p (prop/for-all [g (s/gen spec)] (s/valid? spec g))]
(is (= {:result true} (abbrev-result (tc/quick-check 100 p))))))
2019:08:10 13:39:24 norton where STRING-NAMESPACE is a string used to filter keys from the s/registry
2019:08:10 13:39:58 norton I might not have all of the parens correct … just pseudo-code 😉
2019:08:10 14:45:01 alexmiller that conform thing above was fixed in the next commit btw
2019:08:10 14:46:51 alexmiller so what fails? which spec and what's the failure?
2019:08:10 15:05:19 norton Here is a subset of the stack trace. I bumped up to the latest version 'ca131c1bec353a6ebe4fe8e0a8b6f8825734ef42'.
2019:08:10 15:17:27 norton Here is a subset of the stack trace. I pinned it at the version '000d7d83a98ca3af58a44c20d7bd9fea0f4b03ab'
2019:08:10 15:18:24 norton Interestingly, there seems to be a different failure(s) between these two commits.
2019:08:10 15:23:26 norton The commit '922b0f5886735641e0cdded58fc85011f79cc292' has the same stack trace as 'ca131c1bec353a6ebe4fe8e0a8b6f8825734ef42'. This commit introduced the second failure.
2019:08:10 15:24:04 norton As I posted originally, the commit '2bd68ffee3a775e6fa0f925260b94a9c421787c0' introduced the first failure.
2019:08:10 15:39:20 norton I will try to narrow down the issue to a small repository.
2019:08:10 15:41:38 alexmiller sorry, hard for me to do much without seeing the spec
2019:08:10 16:14:05 eval2020 I would expect the first (s/valid?...) to be without the {:closed ...}
2019:08:10 17:41:53 alexmiller yes, I whiffed some copy/pasta on the example, fixed
2019:08:11 14:52:47 norton Working example with 'dc960158ccb9cf8b5dc555842eea5e1af12f28c2'
2019:08:11 14:53:58 norton Failing example with '2bd68ffee3a775e6fa0f925260b94a9c421787c0'
2019:08:11 14:57:17 norton @alexmiller Please see these two examples. The latest version 'ca131c1bec353a6ebe4fe8e0a8b6f8825734ef42' behaves the same as the failing example.
2019:08:11 20:48:39 seancorfield Definitely related to (s/def ::bar ::baz) since you can do this
2019:08:11 21:36:43 seancorfield Yup. Just confirmed that works now!
2019:08:12 00:32:25 norton @alexmiller Thank you. That issue is resolved. I have a new one. Here is a failing example taken from the spec Guide. This is with the latest version 'e58788107ee2ab765a7d46854b2cffd85429e339'.
2019:08:12 00:55:07 seancorfield @norton I'll defer to Alex for the final word but I think that's a case where Spec 2 and Spec 1 differ. I think I ran across something like that when I tried converting our code over to Spec 2...
2019:08:12 00:56:36 norton @seancorfield thank you. I will take another look.
2019:08:12 01:03:02 seancorfield I'm trying a few things locally but it isn't jogging my memory of what I had to do here. So it may be an unintended bug.
2019:08:12 01:11:51 seancorfield Hmm, the more I look at this, the more I think it is a new bug. I pulled up the Spec 2 branch of our code and we have some specs that look very similar to the above and used to work. Right now I can't test the Spec 2 branch on my laptop so I can't dig in any further until tomorrow @norton
2019:08:12 01:34:36 norton @seancorfield Yes, I think so. This type of spec was working with the 'dc960158ccb9cf8b5dc555842eea5e1af12f28c2' commit.
2019:08:12 02:54:51 alexmiller fixed, just missed the expand-spec for with-gen
2019:08:12 05:20:06 seancorfield @alexmiller That fixes the problem reported above, but you can't do (s/exercise ::kws) -- that produces
user=> (s/exercise ::kws)
Execution error (IllegalArgumentException) at clojure.spec-alpha2.protocols/eval173$fn$G (protocols.clj:11).
No implementation of method: :gen* of protocol: #'clojure.spec-alpha2.protocols/Spec found for class: clojure.lang.PersistentHashSet
user=>
2019:08:12 09:24:17 alexmiller That is a spec 2 difference - the gen needs to wrap s/spec around the set
2019:08:12 15:55:05 seancorfield Ah yes. That was the difference I was thinking of when I first thought this was a Spec 1/2 difference but still couldn't get that example working! /cc @norton
2019:08:12 09:51:23 norton @alexmiller @seancorfield Thank you. The issue above is fixed. There is one more.
2019:08:12 09:52:33 norton This is an abridged version of a larger spec that illustrates the failure.
2019:08:12 14:43:00 alexmiller fixed, added some tests to catch this and prior
2019:08:12 14:49:17 schmee Can you alias a spec to another spec? I have a situation like this where I would like to not repeat the definition of task-state for both old and new:
(s/def ::task-state #{... big set ...})
(s/def ::old ::task-state)
(s/def ::new ::task-state)
(s/def ::event (s/keys :req [::old ::new]))
2019:08:12 14:50:10 schmee the code above doesn’t work, but it hopefully shows the intent
2019:08:12 14:52:57 jjttjj That code should work though?
2019:08:12 14:53:05 alexmiller should work (in spec 1)
2019:08:12 14:54:12 schmee you’re right, this was PEBKAC… facepalm sorry for the noise
2019:08:12 17:13:17 ataggart Anyone know of guidance around organizing specs and the code being spec’d? E.g., putting specs in the same file as the things being spec’d, or in separate parallel nses, or in a separate common spec ns?
2019:08:12 17:31:07 ataggart I swear I read that doc! 🙂
2019:08:12 17:31:12 ataggart Thanks
2019:08:12 18:32:55 seancorfield @alexmiller I'm updating our branch to the latest Spec 2 at work and ran into this exception "No implementation of method: :conform* of protocol: #'clojure.spec-alpha2.protocols/Spec found for class: java.lang.String", -- would that be from this spec: (s/def ::empty-id (s/or :empty #{""} :id ::pos-int)) ?
2019:08:12 18:33:57 alexmiller Dunno, would have to take a closer look
2019:08:12 18:34:46 seancorfield I'll try wrapping #{""} in (s/spec ,,,) and see if that fixes it... but we have a lot of set-as-predicate specs and this has all worked with earlier Spec 2 builds...
2019:08:12 18:37:23 seancorfield Nope. Problem lies deeper. Digging...
2019:08:12 18:38:33 seancorfield Seems to be related to this
2019:08:12 19:02:53 alexmiller @seancorfield you shouldn't have to wrap that so I'd say it's a bug if that fixes it :)
2019:08:12 19:04:29 alexmiller it's probably not that it's dynamically built but rather something about what's being built (my first suspicion would be with conformer)
2019:08:12 19:17:03 seancorfield
user=> (require '[clojure.spec-alpha2 :as s])
nil
user=> (s/def ::foo (s/keys))
:user/foo
user=> (s/valid? ::foo {"bar" 42})
Execution error (IllegalArgumentException) at clojure.spec-alpha2.protocols/eval13332$fn$G (protocols.clj:11).
No implementation of method: :conform* of protocol: #'clojure.spec-alpha2.protocols/Spec found for class: java.lang.String
user=>
Only fails if the map being checked has string keys, rather than keyword keys.
2019:08:12 19:30:49 alexmiller I changed how some of that keys stuff worked, obviously did not test it very well (but not really emphasizing keys as that's going to be replaced by schema/select)
2019:08:12 19:49:45 seancorfield I assume it's trying to check whether unspecified keys conform and not guarding that with a check for whether the key name could actually be a spec?
2019:08:12 19:50:34 seancorfield Switching a large code base from s/keys to s/schema/`s/select` will be a massive piece of work so I hope that s/keys will continue to work while we make those changes 🙂
2019:08:12 19:56:23 alexmiller yeah, I'm trying to keep it working, I'll fix it up
2019:08:12 22:26:28 norton All tests are running fine now with 6af1f3.
2019:08:13 05:09:37 murtaza52 I have a sorted collection, I want to define the relationship between 2 elements of the collection ie the first element is greater than the second element. Is this possible ?
2019:08:13 05:32:56 seancorfield @murtaza52 (sort comparator coll) lets you pass in a comparator that defines the relationship on which you are sorting.
2019:08:13 05:36:44 seancorfield (a silly example, but it sorts all even numbers ahead of all odd numbers 🙂 )
2019:08:13 05:37:50 seancorfield Is that an answer to the question you were asking? (I wasn't entirely sure what you were asking)
2019:08:13 05:39:06 murtaza52 @seancorfield thanks for the example 🙂. However , I am already using a custom comparator for my sorting, however how do I use that in my spec definition ?
2019:08:13 05:39:36 seancorfield I guess I don't understand what you're trying to do...
2019:08:13 05:39:36 murtaza52 I have a function that does the sorting. Now I want to spec the fn, to specify that the output collection is sorted
2019:08:13 12:21:30 alexmiller We are kicking around some ideas for stating this constraint in spec 2
2019:08:13 12:22:33 murtaza52 cool thanks
2019:08:13 05:39:47 seancorfield Ah...
2019:08:13 05:40:16 seancorfield This is one of those cases where the spec would almost need to be the implementation. I think that's over-specification to be honest.
2019:08:13 05:40:41 seancorfield I very rarely specify the result to that level of detail.
2019:08:13 05:41:35 seancorfield After all, it's not like instrumentation checks :ret or :fn so this is really only about generative testing and I think that can be better served by property-based testing, not direct :ret/`:fn` spec testing.
2019:08:13 05:42:50 seancorfield For example, for any collection with count > 1, good properties to test are that the comparator produces the expected result for the first and second items, and for the first and last items.
2019:08:13 05:43:57 seancorfield It's tempting to "spec everything" when you get started but it's really not very productive in my experience.
2019:08:13 05:44:38 murtaza52 ack makes sense from a testing perspective. however what I like about specs is also the documentation it produces. So when I spec a relationship, it also helps other devs to know what the fn is going to return.
2019:08:13 05:44:45 seancorfield Spec should be "just enough" to sanity check what you're writing -- unless you are specifically using it to validate input data (or output data), rather than specific functions.
2019:08:13 05:48:16 seancorfield @murtaza52 I'm not sure that using Spec to that level of detail, just for documentation, is productive. I would say "most functions do not need to be Spec'd" but you could describe the behavior in the docstring without adding the overhead of Spec...
2019:08:13 05:49:04 seancorfield Spec makes the most sense on boundaries and for the "critical" functions. Spec'ing everything makes code brittle.
2019:08:13 05:49:09 seancorfield (and verbose)
2019:08:13 05:49:44 murtaza52 @seancorfield thanks for the insight
2019:08:13 06:09:39 seancorfield In my mind, this is why type systems can make code brittle -- you end up with types everywhere rather than just where they "make sense". It's one of the things I really like about Clojure: you can write a lot of generic code and a lot of abstractions without introducing that brittleness -- but you also have Spec for defining data structures and for certain functions on the edges of modules.
2019:08:13 06:09:39 seancorfield In my mind, this is why type systems can make code brittle -- you end up with types everywhere rather than just where they "make sense". It's one of the things I really like about Clojure: you can write a lot of generic code and a lot of abstractions without introducing that brittleness -- but you also have Spec for defining data structures and for certain functions on the edges of modules.
2019:08:13 06:40:28 murtaza52 yup makes sense
2019:08:13 10:19:34 murtaza52 @seancorfield so trying to think through, in what situations does spec’ing a fn makes sense, so that it could instrumented later ?
Bcoz as u mentioned spec’ing all fns dont make sense. The ones on the boundary I am already validating. So where does the optional spec’ing come in ?
2019:08:13 16:29:12 seancorfield Spec'ing functions can be useful for development / testing (instrumentation / generative checking) but I would generally only spec functions that are part of module APIs or "important" functions. There's no hard and fast rules here -- just spec what is "useful" to spec. For example, in next.jdbc, there are optional instrumentation specs for the public API functions -- for users of the library -- and those are instrumented during test runs too, for me as the developer of the library (but it's also important run the tests from time to time without instrumentation to see what behavior/errors crop up in known bad input data).
2019:08:13 16:29:56 seancorfield I think you just have to find what works for you in terms of the amount of checking you need during development/testing and in terms of what you want to provide for clients of your code.
2019:08:13 16:30:30 seancorfield It's a bit like the question "Which functions should I unit test?" -- the answer is "Not all of them" but there are no hard and fast rules there either.
this is valid - (s/explain ::my-event {::id "a"})
this is invalid - (s/explain ::my-event {:id "a"})
So do all my keys in the input have to be namespaced ? Because I am not able to validate data when the keys are not namespaced.
2019:08:13 06:46:46 seancorfield If you use :req then, yes, the keys must be namespaced. If you use :req-un then you have have simple keywords.
2019:08:13 06:47:42 seancorfield With :req-un, you can have ::foo/bar as the spec for the key -- but the key will be :bar. That allows you to have different specs for the same key name in different contexts.
2019:08:13 06:56:26 murtaza52 thanks, I was confused bcoz when I used gen, it generated data without namespaced keywords. So my assumption was that it will also me to validate ones without ns.
2019:08:13 06:56:52 murtaza52 So moving ahead is it more idiomatic to have namespaced keys in data too ?
2019:08:13 07:42:28 schmee I have run into what I find to be a tricky modeling problem with spec. I have data that looks like this (excluding namespaces on the kws for brevity)
Those three keys are always included, but if the event type is :state-transition and the new state is :exception, then there will be an :exception key as well (there are a couple of more special cases like this).
What I would like is some form of “wildcard”, so that I can dispatch like so:
defmethod foo [:state-transition :exception]
defmethod foo [:state-transition _] <-- wildcard for the default case
Without the wildcard, I need a defmethod for all possible combinations state-transition/new-state which will lead to a lot of duplication.
Is there a better way to accomplish this, in either spec1 or spec2?
2019:08:13 08:51:59 jaihindhreddy Name the special cases and use s/multi-spec with a dispatch fn that returns these names, then impl them by extending the multimethod.
2019:08:13 12:42:41 murtaza52 Is there a pred for specifying only 1 param in a collection. I find this pattern when I want to spec the input args of a fn. Lets say if the fn expects a hash-map, I usually define the spec for args as :args (s/? ::map-spec). This pattern works, however is not correct bcoz it allows no values too.
2019:08:13 12:44:20 alexmiller s/cat is usually the best top level spec for args
2019:08:13 13:26:12 murtaza52 yup have gone through it, and its open on machine as a reference, however sometimes tend to miss a detail.
2019:08:13 13:37:59 murtaza52 what is a good way to generate date time, I have a spec - (s/def ::start inst?), however the dates that it generates are very similar, and most are the same.
(gen/sample (s/gen ::ks/start) 5)
=>
2019:08:13 13:43:58 alexmiller inst-in will generate only dates near the start range in the first few samples but will then range farther
2019:08:13 13:45:10 Joe Lane Is that why sometimes I see a pattern to (drop 1000 ...) with some of the generators?
2019:08:13 13:50:56 ghadi my general pattern for this stuff is to generate a bunch of deltas, and add them to a constant "anchor" value, in this case a date
2019:08:13 13:51:05 ghadi (But since inst-in exists, use it)
2019:08:13 13:51:27 alexmiller which is exactly how inst-in generator works but it "grows" from the start date
2019:08:13 13:51:46 alexmiller so test.check "shrinking" will shrink towards the start date
2019:08:13 13:54:38 Joe Lane Is that a property of the Rose tree or is it designed explicitly for that?
2019:08:13 13:56:38 murtaza52 changed it to inst-in and this worked (drop 990 (gen/sample (s/gen ::ks/start) 1000))
2019:08:13 13:57:55 alexmiller the rose tree is designed to shrink from greater to lesser "size" and inst-in generator builds from start date + int generator (which has its own "size")
2019:08:13 13:58:43 alexmiller whether or not this is good is a separate question :)
2019:08:13 14:00:31 Joe Lane hahah cool, thanks for the insight alex. It's great to read your insideclojure blog, always a joy to see what you're working on.
2019:08:13 15:56:09 murtaza52 is there a way to merge distinct s/every-kv like s/keys can be done ?
2019:08:13 18:14:45 murtaza52 I have speced a few fns with :args and :ret, however it throws an error only when the :args does not conform, no error is thrown when the :ret spec does not conform. Any ideas what I am missing ?
2019:08:13 18:18:52 seancorfield @murtaza52 Per our thread discussion earlier: instrument checks :args but you need generative checking for :ret (and :fn).
2019:08:13 18:19:25 seancorfield Generative checking is about testing whether the function has the expected behavior (given generated conforming arguments).
2019:08:13 18:19:45 seancorfield Instrumentation is about checking that other code calls this function correctly during development/testing.
2019:08:14 04:12:35 murtaza52 just trying to understand why does a empty seq satisfy this spec (s/valid? (s/every integer?) []), shouldn’t it fail ?
2019:08:14 04:32:06 alexmiller because every element in the collection is an integer
2019:08:14 04:32:14 alexmiller for all none elements
2019:08:14 05:19:59 murtaza52 could you please expand on none elements, did not grasp it
2019:08:14 06:14:11 alexmiller (s/every? integer?) specifies a collection where each element is an integer. In this case [] meets that spec, there just happen to be no elements.
2019:08:14 06:17:01 murtaza52 thanks @U064X3EF3 that explains it
2019:08:14 05:22:23 murtaza52 When I run (stest/summarize-results (stest/check)), I see the following error printed in my repl -
No implementation of method: :specize* of protocol: #'clojure.spec.alpha/Specize found for class: nil
all the tests are passing and the :sym names are being printed correctly, however at the end it prints the above too.
2019:08:14 06:15:49 alexmiller don't know, can't tell from just that
2019:08:14 15:17:57 kenny I get this often. Typically it's from forgetting to call s/gen on a spec.
2019:08:14 15:20:47 murtaza52 There was a typo in my s/fdef association with the sym, once that was corrected, the error disappeared
2019:08:14 06:14:53 alexmiller if you require a non-empty collection, you can use (s/every? integer? :min-count 1)
2019:08:14 21:33:19 ataggart conformer doc states:
> takes a predicate function with the semantics of conform i.e. it should return either a (possibly converted) value or :clojure.spec.alpha/invalid
And yet defining such a function gives me an error (minimal reproducible example):
2019:08:14 21:35:41 alexmiller defining that function fails the spec for defn itself
2019:08:14 21:35:59 alexmiller as it has the magic "invalid" value
2019:08:14 21:37:31 alexmiller a workaround is
2019:08:14 21:37:33 alexmiller
user=> (def i :clojure.spec.alpha/invalid)
#'user/i
user=> (defn f [] i)
#'user/f
2019:08:14 22:01:43 ataggart Thanks!
2019:08:14 22:04:14 ataggart I think I have managed to avoid running into this by having predicates before the conformer (surrounded by s/and), thus obviating the need for the forming fn to explicitly yield ::s/invalid.
I expect the random generated data, whenever there is a collection, to have max three elements (including collections nested in collections, speaking of lists/vectors/maps) but that is not the case. What do I wrong? Thank you!!!
2019:08:15 12:18:33 alexmiller What are the specs?
2019:08:15 13:02:19 holyjak The problematic argument to the function is ::account-invoices, the core of its specs is:
2019:08:15 13:27:06 alexmiller off the top of my head, not sure. those :min-count's may be causing something with size to be set in the constructed generator, not sure
2019:08:15 13:29:35 holyjak Yes, that seems likely based on the docs. The question is why the generated data is larger than that....
2019:08:15 14:00:35 alexmiller not something I have time to look at atm, feel free to file a ticket if needed
2019:08:15 18:12:17 zlrth is it possible to make a spec for map that has two keys, and if the value of one key matches, then the value of the other has to match something else? but if the value of the one key doesn’t match, then the spec is valid, regardless of the second value
2019:08:15 19:04:23 alexmiller can you write a function that says yes or no to that?
2019:08:15 19:04:29 alexmiller if so, that predicate is now a spec
2019:08:15 19:04:50 alexmiller specs are limited only to functions ... which can do anything
2019:08:15 19:05:00 zlrth ah of course--thanks very much
2019:08:15 19:05:24 alexmiller perhaps a better questions is whether this is a good idea :)
2019:08:15 19:05:49 alexmiller ime, you're probably doing too much
2019:08:15 19:05:58 alexmiller don't strangle the code, just check it
2019:08:16 09:16:45 metametadata Looking at the latest blog post about spec2: (s/valid? ::s {::f "Bugs" ::l "Bunny" ::x 10} {:closed #{::s}}).
Does this mean that if ::f is also a map then in order to close it the user will have to explicitly put it into set too? I.e.: (s/valid? ::s {::f {:a 1 :b 2} ::l "Bunny" ::x 10} {:closed #{::s ::f}}). If yes, then it seems to be inconvenient for cases with nested maps because when validating a top map I don't want to remember and manually specify the "closed-ness" of all the maps used in validation of every particular key.
2019:08:16 10:51:59 alexmiller Yes, that’s what it means, but it doesn’t preclude some kind of :all setting
2019:08:16 11:54:59 metametadata I see, thanks. I was wondering how easy it will be to migrate my current codebase to this approach. As currently the knowledge of closedness is tied to each spec (I have a custom macro: (sp/speced-keys :closed? true :req ...)). And "parent" specs are not aware of this. :all is fine if all the specs are closed but will not work in case some specs are closed and others are open.
2019:08:16 12:10:00 alexmiller Keep in mind also that s/keys itself is being replaced with s/schema and s/select
2019:08:16 12:10:34 alexmiller The closed checking is currently implemented only in s/schema, not s/keys
2019:08:16 14:26:15 sundbp I’ve got this use-case: I’ve got a couple of macros/fns that I use to dynamically create a few “concepts” and associated helper fns (stuff like serialization/deserialization etc). As part of that I’m also creating specs. I’m looking at spec-alpha2 and getting stuck wanting to do something like this:
(s/register k (s/spec (fn [x] ..something using a captured local value..)
This doesn’t fly, the local value (from e.g. an encapsulating let form) ends up not scope captured as I’d expect and instead breaks everything. Any tips how I can accomplish what I need?
2019:08:16 14:29:56 alexmiller the (fn [x] ...) part here is a symbolic form (not evaluated)
2019:08:16 14:30:17 alexmiller so if you want to use a local value, you need to do something to construct the right symbolic form
2019:08:16 14:30:42 alexmiller you could for example use
`( ... ~foo ... )
2019:08:16 14:33:04 sundbp ah. ok. i’ll give that a go. the symbol stuff still doesn’t really compute very well with me - it feels like even though spec2 allows being driven by code (as opposed to manual s/def’s), I still don’t find that I’m able to do so doing things the way I expect. The macros still “get in the way” for me a lot of the time.
2019:08:16 14:35:38 alexmiller there are no macros involved in your code above. how can they be in the way?
2019:08:16 14:35:47 alexmiller well I guess s/spec
2019:08:16 14:35:53 sundbp so I tried (s/register k (s/spec* (fn [x] .. ~foo ..))` - no cigar.
2019:08:16 14:36:00 sundbp Meant the spec2 macros
2019:08:16 14:36:46 alexmiller can you share more of your code?
2019:08:16 14:37:55 sundbp the 2nd register is what I’m struggling with.
2019:08:16 14:38:54 alexmiller what is entity?
2019:08:16 14:39:30 alexmiller I guess a symbol? kw?
2019:08:16 14:40:56 sundbp entity is a kw
2019:08:16 14:42:20 alexmiller and in how does this not work? error or spec doesn't do what you expect?
2019:08:16 14:43:19 alexmiller is there some reason you're using s/spec and not s/spec* in last line?
2019:08:16 14:43:45 alexmiller I think you'd want the latter there
2019:08:16 14:44:37 sundbp
2019:08:16 14:45:15 alexmiller try spec* in that last line
2019:08:16 14:45:55 sundbp
2019:08:16 14:46:30 sundbp (sorry, for screenshots - I don’t have slack access on a remote machine where the code is and can’t copy across)
2019:08:16 14:50:19 alexmiller that's actually a bug in spec 2 function explication
2019:08:16 14:50:40 alexmiller the [x] in the fn is getting ns'ed
2019:08:16 14:50:54 sundbp thank god for that - I was starting to run out of ideas 🙂
2019:08:16 14:51:14 alexmiller well, maybe
2019:08:16 14:51:20 sundbp can I work around that?
2019:08:16 14:51:49 alexmiller in that fn, instead of x can you do ~'x everywhere?
2019:08:16 14:52:43 alexmiller I'm not sure if the error here is in your expansion or inside spec 2
2019:08:16 14:53:15 sundbp cool. that compiles at least, so hoping that means I’m on my way. wont be able to confirm overall functionality without some more work.
2019:08:16 14:53:39 alexmiller basically you're fixing this issue:
2019:08:16 14:54:12 alexmiller I have seen places where spec 2 has trouble with this, but here I think it's in your expansion
2019:08:16 14:54:38 sundbp more context
2019:08:16 14:55:10 alexmiller the code you now have is not using any macros and there is no "magic" here - it's just passing forms to functions
2019:08:16 14:57:59 sundbp I think that macro+adapt-spec wrapper was a trial and error solution to similar issue I had with passing in a fn (e.g. string?). if I already had the fn and not the symbol pointing to the fn at the time I want to s/register I couldn’t work out any way to proceed.
2019:08:16 15:10:08 sundbp btw, that seems to get me back on track and define the sort of spec I have in mind. thanks a lot for the help - and let me know if you want any more info in terms of it being me vs spec2.
2019:08:16 15:46:22 alexmiller if you already have a function object, then you can't go through the symbolic API (spec*)
2019:08:16 15:46:26 alexmiller as it's not a symbol
2019:08:16 15:46:37 alexmiller but you can directly create a spec object that implements the protocol
2019:08:16 15:47:06 alexmiller w/the caveat that you're not going to be able to s/form or nest that kind of thing in other symbolic forms
2019:08:16 15:47:58 alexmiller which is true to some degree in spec 1 too
2019:08:16 16:29:23 sundbp yeah. compared to a more data driven API both spec1 and spec2 throws up some speed bumps if one is generating specs by code. but spec2 seems to have enough flex to make it possible to get what I need even if it’s not yet intuitive to me to work in the symbolic space (as compared to more data composition).
2019:08:16 16:29:36 sundbp so I’m a bit split on the API so far. it doesn’t feel like “normal” clojure when one comes at it from generating specs in code. however, it does compose fine as long as you’re manually spec’ing things. net-net I’d love a more data-driven approach. I’ve seen https://github.com/metosin/spec-tools/blob/master/docs/02_data_specs.md and perhaps what I’m looking for is doable on top of the symbolic api.
2019:08:16 17:30:02 alexmiller well there is now a map-focused format as well (as of last week)
2019:08:16 17:30:03 alexmiller that is almost certainly going to change formats so I wouldn't touch it yet
2019:08:16 17:33:18 alexmiller
{:clojure.spec/op `s/keys
:req (conj (map first required) ::entity-type)
:opt (map first optional)}
2019:08:16 17:33:34 alexmiller is something you could pass to s/spec* for example and get the identical spec
2019:08:16 17:34:16 alexmiller and there is an s/expand-spec that can do form->map
2019:08:16 17:34:51 alexmiller but this should be much more amenable to doing spec->map->transform->spec type things
2019:08:16 17:37:18 alexmiller there are now 3 or 4 different ways to tap into spec creation/transformation and we're looking at some meta stuff on top of that which might make some of them more accessible, but really there's a lot of options now
2019:08:18 20:33:13 sundbp @alexmiller cool. yeah, I’m not saying there isn’t ways to hook things up. For manual use, which obviously is the major use case, things have felt natural and I haven’t had to read source etc. For manipulating specs programatically I’m convinced all I want is doable, but it hasn’t felt “natural” and I’ve had to read the implementation to try to grok the explicate/spec/spec* machinery. The map syntax you hint at looks likely to be more intuitive to me, so I like that. Thanks for both helping me across my stumbling block and for keeping on improving spec!
2019:08:18 23:28:06 cfleming What are the current workarounds for closed spec-like behaviour? I’m trying to retrofit specs to some existing code, and I’d like to ensure that I haven’t missed any cases during development.
2019:08:18 23:28:23 cfleming In particular, how can I specify a map that should only have the keys I’ve specified?
2019:08:18 23:42:36 cfleming Also, I’d like to spec a map which has some standard keyword keys, but also might have string keys with values specified by a spec. Can I do that? Neither s/keys nor s/map-of seem to fit that.
2019:08:18 23:47:52 alexmiller You can, but not easily. You can do an s/coll-of :kind map? and then spec the possible mapentry types
2019:08:18 23:48:13 alexmiller It’s tedious
2019:08:18 23:50:16 alexmiller Another path (if you care less about gen) is to use a conformer to keywordize the map then s/keys
2019:08:18 23:50:32 cfleming Yeah, I don’t care about gen in this case.
2019:08:18 23:50:42 cfleming Just trying to spec an existing system.
2019:08:19 00:26:29 cfleming @metametadata Thanks! I’ll try yours out since it looks like the error messages might be more helpful.
2019:08:19 00:32:46 metametadata cool! we use an evolved version in company projects (which e.g. is more convoluted because of https://github.com/metosin/compojure-api/issues/417), but didn't have time to publish it yet
2019:08:19 14:32:01 Nick I’m trying to write a spec for a payload wrapper map.
For simplicity, the spec I’m trying to write is for a map with two keys, :query and :payload.
For the purposes of this example, I’m using extremely simple examples of the payload with just maps of one key.
`
(s/def ::query (and keyword? #{:avalue :anothervalue :thirdvalue}))
(s/def ::email string?)
(s/def ::user-id integer?)
(s/def ::avalue (s/keys :req-un [::email]))
(s/def ::anothervalue (s/keys :req-un [::user-id]))
{:query :avalue
:payload {:email “<mailto:/cdn-cgi/l/email-protection|/cdn-cgi/l/email-protection>” }}
{:query :anothervalue
:payload {:user-id 123}}
`
I can use multi-spec and a defmulti to key off the :query object,
but that doesn’t let me use the shared :payload keyword. Is the only way around this to define ::avalue and ::anothervalue in different namespaces ::avalue/payload ::anothervalue/payload? I haven’t created a lot of namespaced keywords in clojure yet.
2019:08:19 14:39:03 alexmiller yes, you would need to use different namespaces in this case
2019:08:19 14:39:32 alexmiller the ::query spec is redundant btw, just (s/def ::query #{:avalue :anothervalue :thirdvalue}) would be sufficient
2019:08:19 14:43:39 Nick Do I need to create a separate valid namespace, or can I define the specs with a a namespace that doesn’t correspond to an existing clojure file? ie, does there need to be a (ns anothervalue) somewhere before I can do a (s/def ::anothervalue/payload string?)
2019:08:19 14:51:59 alexmiller technically here, it's better to use the term "qualifier"
2019:08:19 14:52:08 alexmiller which may correspond to an actual namespace, but doesn't have to
2019:08:19 14:52:46 alexmiller you can use a keyword with any qualifier
2019:08:19 14:54:13 Nick Thanks, that's quite helpful.
2019:08:19 14:55:06 alexmiller a lot of docs in Clojure are not particularly clear on this point
2019:08:20 19:29:48 johanatan have things changed sufficiently that one should follow a different guide?
2019:08:20 19:30:14 johanatan it still seems almost identical to ~1.5-2 years ago when I first used spec and I thought things were changing dramatically for v2
2019:08:20 19:30:51 alexmiller v2 has not been released yet, so that is still up to date for v1
2019:08:20 19:31:10 johanatan hm, are there certain things we could do to "future proof" our specs in prep for v2 ?
2019:08:20 19:31:29 alexmiller given how things are in flux, I would not worry about it much yet
2019:08:21 21:04:17 johanatan [i have tried subbing cljs for clojure as well]
2019:08:22 00:08:58 seancorfield @johanatan Is there a reason you're calling instrumentable-syms there, rather than just (stest/instrument)? The latter seems to work with figwheel, based on my limited testing, but the former seems to behave strangely (based on my very limited testing).
2019:08:22 01:57:46 johanatan @seancorfield it was just copypasta from a previous project. it was the first thing that worked. no particular reason.
2019:08:22 01:57:58 johanatan does (stest/instrument) have the same behavior as I'm attempting to get there?
2019:08:22 02:24:15 seancorfield It instruments all functions... which should be the same as all instrumentable-syms but maybe @alexmiller can shed light on that?
2019:08:22 02:54:02 johanatan [btw, that change did get my code to compile, strangely]
2019:08:23 20:11:22 johanatan is there a way to ask spec: "give me all embedded values in the nested map satisfying spec ::x?" (either in the std alpha or in 3rd party extensions). I'm thinking it wouldn't be too hard to do this with specter + spec but just wondered if something like it already exists...
2019:08:23 20:20:35 alexmiller not in spec
2019:08:23 20:20:40 alexmiller but it's just something like (defn deep-match [s x] (->> x (tree-seq coll? seq) (filter #(s/valid? s %))))
2019:08:23 20:20:51 johanatan 🙂 yea. cool, thx
2019:08:24 20:07:43 tyler Are unqualified keywords not supported for closed specs? Closed specs seem to work as expected with namespaced keyword specs but the unqualified keyword form doesn’t seem to work with closed.
2019:08:24 20:14:40 johanatan are you referring to s/keysreq-un or opt-un specifically? are there other aspects of specs that support "unqualified" ?
2019:08:24 20:26:25 tyler This is using schema, not s/keys
2019:08:24 20:33:11 johanatan Ah, specs2 gotcha . Specs1 didn’t support unqualified except via keys
2019:08:24 20:58:51 alexmiller No, not supported right now
2019:08:26 15:33:04 unbalanced okay so I'm doing this pattern a lot and I'm wondering if there's a more concise way to do it, perhaps to register a s/def with another s/def at creation time?
2019:08:26 15:34:37 unbalanced and, while I'm at it ... I'm wondering if anyone has any best practices in combining spec with datomic? Since we already go to all the trouble of making the schema for datomic, has anyone looked into using that for spec?
2019:08:26 17:26:21 seancorfield @goomba This popped up as the second result on a Bing search for using clojure.spec to create datomic schema for me: https://github.com/Provisdom/spectomic -- seems to be what you're looking for?
2019:08:26 17:27:02 unbalanced oh man, and great name too
2019:08:26 17:28:33 unbalanced that's interesting ... they went the opposite way I would've expected, i.e., s/def -> schema
2019:08:26 17:28:52 unbalanced I would've expected schema-ish -> s/def
2019:08:26 17:29:04 seancorfield That makes sense to me. We treat spec as the "system of record" and generate stuff from it.
2019:08:26 17:29:40 unbalanced makes sense. I'm usually just paranoid about "losing" my schema.
2019:08:26 17:30:24 unbalanced I like to see them all written out explicitly. But that would be easy enough with eval-and-replace
2019:08:26 17:30:57 kenny @goomba clojure.spec can express more info than Datomic schema can so it makes more sense to go from spec -> schema.
2019:08:26 17:30:57 unbalanced wheeee yess! That was another question I had!
2019:08:26 17:31:43 unbalanced @kenny solid point. I was just expecting something more like a hybrid spec/schema, uh... spec
2019:08:26 17:31:56 unbalanced have it all in one place
2019:08:26 17:32:13 unbalanced this is totally fine though
2019:08:26 17:33:11 unbalanced also got a lot of good answers out of it and experienced @seancorfield do a "let me Bing that for you" which is fantastic 😆
2019:08:26 17:35:35 seancorfield I hope you didn't think I was being mean with my comment about Bing? You've said you're sometimes not finding search results for stuff so I figured sharing the search string I used might be helpful. Quite a few of my friends ping me about finding stuff because apparently I'm better at searches than them... 🙂
2019:08:26 18:20:21 unbalanced no hahaha I thought it was hilarious 😄 I loved it
2019:08:26 17:33:23 alexmiller I've seen people do both directions, and also start from a 3rd "source of truth" and generate the Datomic schema and spec too
2019:08:26 17:33:37 unbalanced mmm chimeratomic
2019:08:26 17:39:23 kenny Attaching schema data directly to the specs is something that interested us too @goomba. From what I remember when we wrote that lib, it wasn't immediately obvious what the best path to doing that would be. Ideally it would've been something simple like attaching metadata to the spec's name (e.g. (def ^{:db/isComponent true} my-spec map?)). That wouldn't work for specs because they are typically defined as keywords which do not accept metadata.
2019:08:26 17:42:50 kenny There were 2 other options IIRC. 1) Create a new s/def which accepts a new 3rd parameter. 2) add a function similar to s/with-gen which would attach metadata to the spec itself.
2019:08:26 17:42:52 unbalanced nod... in my lazy imagination it would be something like
and then specs and schema would be gen'd from that. But listen I'm a lazy application layer dev asking for handouts, what you did is great it's just not what I would've thought of immediately
2019:08:26 17:43:43 kenny I mean, that's basically a complete schema map.
2019:08:26 17:43:56 unbalanced I'm not even sold on the :spec/spec with a function there because now that's not even a value anymore
2019:08:26 17:44:32 unbalanced yeah I'm excited to check out spectomic!
2019:08:26 17:44:49 unbalanced I do a ton of datascript/spec work so this is great
2019:08:26 17:49:42 kenny I haven't looked at this at all but I'm assuming that this sort of thing would be easier to do with Spec2, given Spec2 allows specs to have docstrings.
2019:08:26 19:18:41 ikitommi hello all. is there a reason that pos? and neg? don’t have built-in generators?
2019:08:26 19:28:03 alexmiller yes, they are not specific to one numeric type
2019:08:26 19:28:15 alexmiller use (s/and int? pos?) or whatever numeric type you need
2019:08:26 19:39:20 ikitommi ok, thanks!
2019:08:26 19:51:02 alexmiller or something like pos-int?
2019:08:27 15:44:14 joefromct Hi,
I’ve read the spec guide and couldn’t really come up with the correct way to do the following; Apologies if i had missed something.
I am looking to define a spec for a ::paragraph that consists of “::words” , “::space(s)” , and “less often” ::punctuation. It (of course) does not need to be grammatically correct.
It seems like i need something like s/cat however i want the results to be not a vec but a larger string (that consists of smaller “spec” strings...“).
I’m interested in having a spec for ::paragraph as well as a generator to play with a natural language processing app i’m working on.
Any tips appreciated,
2019:08:28 20:03:23 ataggart I managed to do something like that by using conformer to tokenize the string first. E.g.:
2019:08:28 20:04:18 ataggart Typed that on my phone from memory, so apologies if I got things incorrect.
2019:08:27 15:57:26 alexmiller I think you’re probably asking for more out of spec than it is designed to provide
2019:08:27 16:01:08 joefromct ok, that makes sense. thanks, i’ll come up with something.
2019:08:27 20:37:32 vemv Is there a version (in some library, etc) of s/keys that allows me do manipulate the spec definition before proceeding?
My current pattern for that is:
...Generally it works fine, but now my needs also include clojurescript, so I can't use eval
2019:08:27 21:57:42 alexmiller I don't think there is an easy pattern for it in spec 1. CLJ-2112 has a sketch of specs for specs which lets you conform, modify then unform, but it's pretty cumbersome due to the structure of s/keys.
2019:08:27 21:58:28 alexmiller spec 2 has several ways to do this and a new symbolic map form in work in particular
2019:08:27 23:16:48 vemv Appreciate the confirmation, thanks!
2019:08:27 21:38:16 johanatan what would the regex to match "one or more forward slashes" be?
#"\/+" doesn't seem to compile or work
2019:08:27 21:38:42 johanatan also tried several other permutations
2019:08:27 21:40:35 johanatan this seems to have worked: (js/RegExp. "\\/+")
2019:08:27 21:40:54 johanatan but pretty sure #"\\/+" did not
2019:08:28 18:12:22 johanatan hmm, ^^ that was in lumo.
2019:08:28 18:12:47 johanatan when it's in my source file and compiled by figwheel-main I get the following js error:
Uncaught SyntaxError: missing ) after argument list
2019:08:28 18:12:59 johanatan so, yea, a compilation error apparently
2019:08:28 18:14:22 johanatan and i just realized this is the very wrong channel for this topic lol
2019:08:28 18:14:31 johanatan [sorry about that]
2019:08:28 19:37:53 chancerussell I noticed that alpha2 supports s/def ing symbols instead of keywords and got really curious about it…then realized that the first version supports the same thing
2019:08:28 19:38:07 chancerussell Really curious about the use case for that though
2019:08:28 19:40:14 alexmiller that's how function specs are registered (s/fdef)
2019:08:28 19:40:21 chancerussell lol duh
2019:08:28 19:40:22 chancerussell thanks
2019:08:28 22:46:22 ag can someone throw at me with an example of a spec for a string that has to be of specic length (within range) and should only allow letters. The catch though: it should not fail generation with Couldn't satisfy such-that predicate message.
I swear, I used to have those, somehow I’m feeling stupid and can’t make them anymore.
2019:08:28 22:51:49 alexmiller you could use test.chuck's support for making generators from a regex
2019:08:28 22:54:16 ag ah test.chuck. Yeah it has a few nice gems there. But I feel I’m missing something fairly small
2019:08:28 22:59:13 ag my best effort for now is to use with-gen with gen/such-that with max-tries jacked-up to a huge number. But it seems to be slow and still not guaranteed to not to fail to satisfy
2019:08:28 23:05:30 seancorfield @ag Another "vote" for test.chuck here since you can use a (string) regex such as #"[a-zA-Z]{1,32}" for example, for pure alpha strings up to 32 chars in length and test.chuck provides a generator that works out of the box. We use it to generate email addresses and a bunch of other stuff.
2019:08:29 20:27:48 alexmiller We have some open tickets about SO on recursive gen. I have not dug into any of them recently
2019:08:29 20:33:51 joefromct ok sounds good, i’ll keep an eye out.
2019:08:29 20:34:11 joefromct i appreciate how you always try to respond when i’m sure your busy with strange loop very soon.
2019:08:29 20:34:29 joefromct i hope i don’t miss it this year because i’m stuck in a tube in the museum. j/k
2019:08:29 20:35:54 alexmiller me too :)
2019:08:30 05:10:31 johanatan does spec-alpha2 support clojurescript?
2019:08:30 12:23:05 alexmiller No, we will probably hold off on that till churn slows down
2019:08:30 19:25:57 johanatan thx
2019:08:30 08:31:13 hkjels Generating from the spec: (s/inst-in #inst "1984" #inst "2019") will just return dates from 1984
What’s the idiomatic way of generating random dates within a range?
#inst only works with string literals and not computed strings
2019:08:30 12:24:03 alexmiller Samples are biased to beginning of the range, but “grow” as you make more
2019:08:30 12:24:41 alexmiller So if you sample a larger number you’ll see more variety
2019:08:30 10:06:10 mpenet (java.util.Date. x) with some random number within the range
2019:08:30 10:47:41 hkjels problem was actually the string I created. Your absolutely correct @mpenet
2019:08:31 20:30:53 seancorfield @j.m.frith Can you share the spec and the function that you're having problems with? (perhaps in a cut down version if it's a lot of code)
2019:08:31 20:32:07 Jazzer Thanks @U04V70XH6 I'm just working something up. In the actual code, the problem bit is fairly deep in a map, so I'm going to try to simplify
2019:08:31 20:32:14 Jazzer Back in a few minutes...
2019:08:31 20:31:13 Jazzer Hi everyone, I'm getting issues when running stest/check while a function is instrumented. It seems to be conforming the input before passing it to the function. I'll try and condense my code into a minimal example...
2019:08:31 20:31:13 Jazzer Hi everyone, I'm getting issues when running stest/check while a function is instrumented. It seems to be conforming the input before passing it to the function. I'll try and condense my code into a minimal example...
2019:08:31 20:37:34 Jazzer Nuts! My minimal example has no problems
2019:08:31 20:37:44 Jazzer I'll try expanding the code until I see an issue
2019:08:31 20:39:57 Jazzer The symptoms I'm seeing are: if I run (stest/unstrument) and then (stest/check 'my-fn) (not sure how to write a backtick in code but my-fun is quoted with a backtick not a single quote) I get tests all passing
2019:08:31 20:41:02 Jazzer If instead I run (stest/instrument) followed by (stest/check 'my-fn) then I get errors that the input has not conformed to spec.
2019:08:31 20:42:47 alexmiller Could you just share the error?
2019:08:31 20:45:51 Jazzer
2019:08:31 20:46:58 Jazzer I'm not sure that's quite as helpful as it might be, which is why I'm trying to scale down to get to the root of the problem
2019:08:31 20:47:18 Jazzer There's a whole bunch of data in there which isn't very helpful
2019:08:31 20:49:08 Jazzer Although right at the front of line 4 it's the part [:decision #:log{:decision {}}] this is definitely a conformed value - the spec has an (s/or ...) with :decision being one of the options
2019:08:31 20:49:09 seancorfield @j.m.frith Are you sure that function works with instrumentation on when called manually from the REPL?
2019:08:31 20:50:22 Jazzer One moment, will re-try now
2019:08:31 20:51:26 Jazzer It's working fine for me now
2019:08:31 20:51:55 Jazzer And it gives and returns a map as expected without the conform path
2019:08:31 20:53:15 Jazzer Have also just checked that (gen/generate ...) gives me a map that conforms fine to the spec.
2019:08:31 20:54:21 Jazzer I think it'll be easiest if I take a bit of time to whittle away all the nonsense so that things are a bit easier to follow.
2019:08:31 20:57:57 seancorfield One approach that I strongly recommend when working with Spec is to write and check each spec as you write each function -- that way you don't get a bunch of code for which you are trying to write a bunch of specs all at once and then testing all at once. The REPL allows you to take small steps and get feedback immediately on each piece of code as you write it.
2019:08:31 20:59:19 seancorfield For example, I have a (comment ,,,) block in each file where I'm working on code + spec and I have (s/exercise ,,,) calls in that comment that I eval to check each spec will generate (not all specs need to generate but I think it's important to check the ones that should as you write them).
2019:08:31 20:59:48 seancorfield That "Rich Comment Form" also includes instrument and check calls as needed.
2019:08:31 21:01:17 Jazzer That is definitely good advice, which I should have taken before getting to this point. I was too eager to learn the language first and built a working prototype and am now going back to learn how to use spec.
2019:08:31 21:02:23 seancorfield Also bear in mind that "nothing everything needs a spec" 🙂 Spec is great for boundaries and for "key functions" that you really need to be sure of. Spec isn't a type system, so you don't need a spec on every function.
2019:08:31 21:03:57 seancorfield At work, we've used Spec since the first alpha builds in the 1.9 cycle and we mostly spec data, rather than functions. We use s/conform and s/invalid? in production code (and s/explain-data to help drive error message generation).
2019:08:31 21:04:02 Jazzer I guess I have a "base layer" of functions which directly interact with my state-map (in a functional way) and I'm currently only speccing those functions. The higher layers I may get to at some point
2019:08:31 21:04:54 Jazzer And the spec is mostly so that I can run generative tests on that portion of the code
2019:08:31 21:05:21 seancorfield We use instrument to support development work, mostly. We use check sparingly.
2019:08:31 21:05:58 Jazzer The problem is that one part of the system now has an (s/or in the spec and that is throwing off the earlier function which worked alright when I didn't have that in the spec...
2019:08:31 21:07:51 Jazzer Again, that sounds about right. I was really using check just to try and find edge cases. Maybe I should stick to either using instrument or check at a time. I'm getting no problems at all when using one but not the other
2019:08:31 21:08:37 seancorfield I'll be interested to see the cut down example that breaks because of s/or -- I've never run into that (in several years of working with Spec).
2019:08:31 21:12:09 Jazzer I'll work it out. It seems even stranger at the moment because the spec is actually saying it's failing on a function that shouldn't even be called. It's failing on game/player-count and I'm checking game/set-lead-player. set-lead-player has two lines, neither of which call player-count. It looks like I've got a bit of digging around ahead of me... I'll be back in the next few days once I've simplified things down
2019:08:31 21:14:08 seancorfield If you instrument a function that takes a function as an argument, and you have a fspec for that argument, it will be generatively tested when it is called.
2019:08:31 21:15:25 seancorfield (that sometimes surprises people)
2019:08:31 21:16:40 Jazzer I don't think that is the case here, but that is good to know. I have to head out now, but I'll be back 🙂
2019:08:31 21:17:20 Jazzer Thank you for your time - I have to say that I'm impressed both by clojure and the community here
2019:08:31 23:46:50 Jazzer I have got almost to the bottom of this! The issue was that I was calling a function within the spec definition, and not realising that spec passes a conformed value into whatever you give
Code looks like this:
I hadn't realised that (:game %) returns a conformed value of the argument passed in, rather than just passing on the value itself
2019:08:31 23:49:38 Jazzer Two questions would follow from this:
1. Is there any way to use a function in this way and pass in the original argument, rather than a conformed value
2. Should I in fact be avoiding this conflation of spec and function? i.e. any specs should depend purely on the data they are presented (and core clojure functions)
2019:08:31 23:53:39 Jazzer As to my question 1 above, the answer is s/unform. Worked perfectly with that in there. Yay! Answered my own question 😛
2019:09:01 00:24:43 seancorfield When you're writing compound predicates like that, it's expected that you take into account the shape of the conformed value -- calling s/unform is kind of a sledgehammer...
2019:09:01 00:25:49 seancorfield I'm a bit surprised you need a function to get the player count from a game tho'... isn't that just an element in the game's map? Or are you calculating it on every call from something inside the game map?
2019:09:01 00:28:48 seancorfield But your :game/game spec has s/or at the top-level? What are the alternatives there? (sorry if you posted it elsewhere, I'm only looking at this thread right now)
2019:09:01 11:39:28 Jazzer Thanks @U04V70XH6. Agreed that s/unform seems overkill, I just thought I’d share that it does indeed achieve my (probably misguided) aim of being able to use the original argument rather than a conformed version.
2019:09:01 11:41:30 Jazzer The reason for the function is that, as I developed things, I kept changing up the structure of the map. Rather than having to change everywhere in the code which refers to this element, I made a function to pull it out and then only need to update that one method if/when I changed my mind again.
2019:09:01 11:46:07 Jazzer And no, the :game/game spec at the top level is just s/keys. One such key is :game/log which is a s/coll-of :log/log-entry :kind vector? and finally :log/log-entry is where the s/or happens to choose between various different types of log entry
2019:09:01 11:47:55 Jazzer Some log entries are for player actions, others are for rules being applied and each has a different structure.
2019:09:03 12:04:20 Vesa Norilo Hi guys! Is there a way to destructure and walk specs? Suppose I have (update-in document [some path] transform). Is there a way to derive a spec for transform output by reaching into the spec for document?
2019:09:03 12:28:24 alexmiller Not right now
2019:09:03 12:53:59 Vesa Norilo right, thanks!
2019:09:04 18:01:43 johanatan i have a function from ::a -> ::b and i am testing it with stest/check. i have set :num-tests to 1 via: (stest/check 'the-sym {:clojure.spec.test.check/opts {:num-tests 1}}). does it make sense that i would see two ::as generated?
2019:09:04 18:03:30 johanatan [i've verified that both generated ::as are non-trivial/non-empty so it shouldn't be stest peeking at the first and considering it "not good enough" then proceeding to the second].
2019:09:04 18:07:00 johanatan hmm... i guess there could be a hidden such-that or some other "retry" mechanism behind the scenes.
2019:09:04 18:07:00 johanatan hmm... i guess there could be a hidden such-that or some other "retry" mechanism behind the scenes.
2019:09:04 18:07:17 taylor got a small repro example?
2019:09:04 18:13:36 alexmiller you may see it gen more than once
2019:09:04 18:14:56 alexmiller even in a single test
2019:09:05 11:28:46 pvillegas12 I have a spec which reuses another, something like
2019:09:05 11:29:22 pvillegas12 However, my problem is that s/explain does not show the ::sub-a spec violations, it just says that ::sub-a is invalid
2019:09:05 11:29:38 pvillegas12 is there a way for spec to “walk” down ::sub-a in this case?
2019:09:05 11:33:06 pvillegas12 I could, of course, copy over the (s/keys) in ::sub-a to the spec of ::a but I was looking for a solution that reuses the ::a declaration
2019:09:05 12:01:30 alexmiller Right now, this is what I would recommend. Much better options coming for this in spec 2
2019:09:05 12:10:09 pvillegas12 I actually got good error messages by doing
2019:09:05 11:48:36 Leon how can i describe a map where a key could either contain a normal value ::some-object or, in an error-case, an error-value?
I don't want to add the error-case to ::some-object as that would greatly decrease the expressiveness and reliability,...
2019:09:05 11:59:09 alexmiller Spec is about expressing the true range of values. If it can contain special error values, then that is part of that truth
2019:09:05 12:00:21 alexmiller If that’s weird, you might consider using an alternate attribute for the error or using exceptions
2019:09:05 12:03:51 Leon exceptions are an extremely ugly solution (at least for me, coming from ADT Either-type land)... and my data cannot contain errors, but the place they're transmitted can. i guess i'll create a wrapper object that can be either of type success or of type failure, and then contain the values respectively
2019:09:05 12:04:23 alexmiller Exceptions are the idiomatic solution on the jvm and in Clojure
2019:09:05 12:06:18 Leon i guess thats an idiom i dislike ^^ i really hate the complexity that braking out of my control flow adds, especially when otherwise working with pure functions that are easy to reason about ;D
and in my case exceptions aren't really a simple option, as im working in clojurescript with re-frame, where im dispatching events and then dispatching other events on success
2019:09:05 12:49:06 potetm Problem is: if you go down the path of “exceptions are bad,” you’re fighting not just the clojure ecosystem, but the Java(Script) one. Leveraging host libs is one of the primary reasons clojure(script) exists at all!
2019:09:05 12:51:01 potetm Not to say you can’t write clj(s) w/o host libs. Many try. But know that you’re giving up a primary value-add for the lang.
2019:09:05 12:53:03 potetm Many have also tried to improve upon exceptions (e.g. slingshot). I’ve found them to be marginal gains at best. Not worth their overhead.
2019:09:05 14:01:38 Leon yea i know, i don't have any direct issue with exceptions, i just really hate using them where not necessary. and as i said, in my case, they are pretty much not a solution, as the one observing the result isn't the same thing that calls the thing that could throw. for such cases, where error-data makes more sense than exceptions, some solution would really be great. but with the current a spec only talks about the keys it gets, the type of the values HAVE to be defined as part of that key really makes something like this nearly impossible, and strongly increases the complexity in knowing what shape your data actually has, because i CANNOT actually spec it without introducing additional complexity (and removing a hell of a lot of expressivity) to my domain model
2019:09:05 14:04:31 alexmiller if your data can contain error values, then spec'ing it means spec'ing those values. spec is just exposing that complexity that you have introduced
2019:09:05 14:10:58 Leon my data cannot contain errors. a book doesn't contain any error. a find-book-response can contain that error. the big problem is that spec forces me to define find-book-response specs for everything, and by that introduces an additional layer of complexity where something like generics could really help.
the whole notion of needing to have a spec for a key in a map makes many things really complicated for no good reason.
say you have a utility function, that takes a (s/keys*) map as arguments (maybe it does some complex math where you want the arguments to be named for readability.
now you need to create spec defs for each int? that the function can take, thus adding extreme amounts of complexity where a
(s/kv :req {:num1 int?, :num2 float?}) would solve your problem perfectly.
2019:09:05 14:12:13 Leon enforcing best practices is a good thing, don't get me wrong. but if these best practices only apply in some areas, but you enforce them in every little situation, that really screws up what could have been an incredibly powerful and nice solution to safety and type-validation in a fully dynamic environment
2019:09:05 14:12:47 alexmiller I don't understand what's forcing you to define anything
2019:09:05 14:13:12 alexmiller keys specs are open - you don't have to spec every attribute
2019:09:05 14:15:01 Leon say you have some complex, user-input based form, where the data doesn't really mean anything specific from a domain-model point of view.
if you want to validate that data in a map context, you'll need to define seperate, single use specs for each entry in that map that you want to validate.
2019:09:05 14:15:47 Leon I'd say that "just don't spec things that would take to much effort" isn't a desirable solution.
2019:09:05 14:16:42 Leon not only does your safety suffer from that, but it also defeats the purpose of spec in general.
because then you could just say "just manually write (if (not (int? my-number)) (throw Exception. "foo")),... thats not the point
2019:09:05 14:16:49 Leon the point is to make validations easier and expressive
2019:09:05 14:17:09 alexmiller well yes, but not for every purpose
2019:09:05 14:17:49 alexmiller given that these are not for your particular domain, but a truly dynamic problem, you're a bit off the goals of spec
2019:09:05 14:18:18 alexmiller you certainly could use a pairing of fields and specs to generically validate (but don't validate the aggregate map)
2019:09:05 14:18:37 alexmiller then no registration is needed
2019:09:05 14:19:09 Leon still, what is the actual reason there is no spec-function that takes keys ( be they namespace qualified or not) and their types (predicates, etc) and validates a map against them?
is it just to encourage good practises?
2019:09:05 14:19:31 alexmiller I don't know that I explain it better than Rich, who has written/talked about this extensively
2019:09:05 14:19:50 Leon could you give me a summary?
2019:09:05 14:20:03 alexmiller the whole idea is to imbue semantics to attributes, and register those for use globally
2019:09:05 14:20:20 alexmiller and to de-emphasize the role of the aggregate
2019:09:05 14:20:31 alexmiller that is, the attributes are the driver, not the aggregation (the map)
2019:09:05 14:20:51 alexmiller this is taken much further in the schema/select work in spec 2 (and s/keys is going to go away)
2019:09:05 14:21:11 alexmiller spec 2 schemas do have inline support for ad hoc un-namespaced key specs
2019:09:05 14:23:34 alexmiller the ability to programatically create and use specs is greatly enhanced in spec 2 as well
2019:09:05 14:35:15 Leon so spec 2 will support that kind of thing i'm requesting?
as i said, i fully understand the "attributes are the driver, not the aggregation" argument, and i think as a base philosophy it's actually rather genius.
but there still are a lot of cases where these attribute-semantics are single-use function arguments. and in these cases, having to give names to things that only make sense in a very specific context is increasing complexity and hurting readability. the "attributes are the driver" argument stops working as soon as you leave your direct domain-model and go into functions that achieve very specific purposes with specific parts of that data. having to describe the shape of every little thing there really makes this a lot more complicated.
especially in my current project, a clojurescript re-frame based frontend application, i really want to spec out every view function. not strictly for validation or domain-expressivity, but actually primarily just to get any kind of good errors. knowing that a view function got its arguments in the wrong shape/type is a lot more helpful than knowing that some reagent-internal had problems....
for these purposes spec would nearly be perfect. dynamic validation, that is actually close to dependent type-systems, would be the perfect solution to greatly specify what each function expects as arguments, no matter how detached that function is from the domain model.
but the way that spec currently prohibits the generation of map-specs inline makes a lot of things hugely more complex.
having a view function that takes 4 arguments look like this:
this is easier to call and read from caller side.... but extremely unnecessarily complex from implementation side.
(yes, title subtitle and content could be domain specific stuff, but maybe im writing a component library that doesn't HAVE any domain specific information.)
do you see where i come from?
2019:09:05 14:40:27 alexmiller those seem almost the same. so not sure I'm getting "extremely unnecessarily complex"
2019:09:05 14:41:00 alexmiller just the extra s/defs?
2019:09:05 14:43:43 alexmiller I don't think spec 2, at this exact moment, helps in this particular situation. however, it's not really easy to judge as we are currently completely reworking how function specs are defined and used and it's probably not going to bear much similarity to spec 1
2019:09:05 14:45:46 alexmiller something like (s/schema {:title string? :subtitle string? :rating int? :content string?}) is sufficient at current moment to spec a map with those (unqualified) keys. can't currently apply that ala s/keys* - that's still a tbd right now (I don't think it's a big deal to address, just not current focus)
2019:09:05 17:42:29 Leon In this case it's not that Bad, but in more complex cases, and especially all over the Code, it gets very anoyying to type and Deal with. It would be great if that could be Adressed in spec2, as it then would be useful in pretty much any Situation without being overly verbose
2019:09:05 17:53:15 andy.fingerhut Sorry, I do not have much constructive to say, but I do recall an explicit statement made by one of the Clojure core team (probably Stu Halloway) that something can be simple (i.e. not complex) but still be verbose.
2019:09:05 19:16:41 Leon yes, thats the java philosophy. "make simple things seem 5 times as complex by introducing unnecessary abstractions, syntactical overhead and overengineered design patterns". that its POSSIBLE to make something simple overly verbose isn't a good thing, and absolutely nothing you should WANT
2019:09:06 02:38:04 potetm the upside of a well-broken-down design is you can always layer on syntax to taste
2019:09:06 02:40:25 potetm the fundamental diff between “complex = verbose” and “complex = a few things glommed together” is the former can turn to the latter, but the inverse is not true 🙂
2019:09:06 02:42:34 taylor >A complex system that works is invariably found to have evolved from a simple system that worked. A complex system designed from scratch never works and cannot be patched up to make it work. You have to start over, beginning with a working simple system.
2019:09:06 02:48:31 potetm this is aligned with my experience
2019:09:06 02:48:48 taylor it took me a long time to learn this lesson 🙂
2019:09:06 02:50:26 taylor I think this quote resonates with my enjoyment of writing Clojure and how easy it is to compose little solutions into big ones
2019:09:06 02:52:17 potetm honestly, the biggest win there is probably immutable data
2019:09:06 02:52:37 potetm but I’m continuously surprised at how much simpler thing get if you design them to be run from the REPL
2019:09:06 02:53:19 potetm every TDDer: It’s like I’ve been saying for yeeeears! troll
2019:09:06 02:54:21 taylor yeah, I felt the same way with other ML-like functional/immutable languages before Clojure. OCaml/F# is nice, but Clojure is more enjoyable for me
2019:09:06 09:28:12 Leon Thats exactly what im looking for! Thanks, @potetm i was about to spin my own solution and dig into how to create custom spec validators for that (might still do, but only for the learning experience)
As i said, i fully agree that well broken down Systems are easier to reason about and work with. But originally coming from Java, and then going to expressive languages like kotlin, haskell, Rust (and now clojure), ive really started to value expressivity. I like to talk in my Code like i would talk to a Person. I like to describe what im trying to achieve, how different Things relate to each other, and what Things represent, and i like to Use expressive Tools to achieve that. The Same way that a for Loop isnt really the best way to describe the relation of a list of numbers to the Same list with every number squared, i feel about using spec. Defining Things that Look and feel like Domain Model/ more complex stuff for simple validations works, but it doesnt Show what i want to express. In my analogy, i want to actually express (map #(* % %) my-list) . Simmiarly, in my spec i dont want to say "the arg title is of type ::title, where ::title is a String." but just "this function takes a String that will be displayed as the tite."
It's not so much about complexity in a Code sense, but just about expressivity, readability and also just typing and reading time. And yes. Lines of Code do matter. Otherwise lisps would put their closing parens on New lines
2019:09:06 11:51:06 potetm Yeah that makes sense to me.
2019:09:06 11:53:10 potetm One thing that I really disliked about spec is that it encouraged creating names where none would suffice.
2019:09:06 12:17:09 Leon yes, that is my exact point ;D glad we agree here, and i hope spec2 will change that
2019:09:06 11:55:34 misha as a datapoint, in ~2 years I specced like 5 functions, and am really amused how much attention s/fdef and instrument get
2019:09:06 12:07:20 potetm whyso? do you just not get a lot of value out of fdef?
2019:09:06 12:13:37 Leon i spec pretty much all my functions, but mainly because its the only way to get immediately helpful error messages. especially in clojurescript, sometimes the errors really dont help. i just get some errors in some G_1235 generated function name and have no idea what happened.
also, I'd say a big help for speccing all functions are the defn-spec feature of orchestra or things like ds/defn from defn-spec that lets you add specs directly into the function signature
2019:09:06 12:28:31 misha there are few functions which benefit from gen-testing. which somewhat rhymes with "you really need just few refs per project".
but other things too:
- read/write overhead,
- maintenance,
- "spec pushes you to have global names where none suffice",
- project size/complexity is not that out of control, so "specing 100% of the code is the only way out"
- wide contact surface with mutable DOM in cljs
- etc.
2019:09:06 12:34:54 Leon maintainance isn't really a big problem, and the error reporting benefits are just huge
2019:09:06 12:39:33 misha I prefer to understand what system is doing, when I can afford it. Admittedly, you are not always in position to have this as an option (project/team size, domain complexity, etc.)
2019:09:06 12:45:21 misha some analogies would be:
- turning off code highlighter to pay more attention to code.
- writing code w/o autocomplete/autosuggest to know better whats there in the project.
these things keep your mind in a tonus
2019:09:06 12:47:03 misha and more stuff you will know you will have to keep track of, more reluctant you will be to let bloat slide in.
2019:09:06 13:15:40 potetm @misha yes, to all that
2019:09:06 13:16:48 potetm I’ve honestly had very little urge to use spec at all.
Though I’ve spent the last 9mo on a novel thing, so nothing is solidified to the point where I want to invest in pouring some concrete.
2019:09:06 13:17:17 potetm Actually, there are a fair number of unit tests, and IMO even those aren’t worth enough to bother.
2019:09:06 13:18:20 potetm (They have to be touched fairly often and have caught very few/no bugs afaict)
2019:09:06 13:25:00 misha and still, pouring concrete on every private function – is questionable
2019:09:06 13:30:10 alexmiller from spec hq: agree
2019:09:06 19:15:25 seancorfield From our code base (World Singles Networks):
Clojure build/config 52 files 936 total loc
Clojure source 299 files 70282 total loc,
3207 fns, 662 of which are private,
441 vars, 29 macros, 62 atoms,
602 specs, 21 function specs.
Clojure tests 330 files 19116 total loc,
4 specs, 1 function specs.
We were an early adopter of Spec in production code (`conform`/`valid?`/`explain-data`) but we mostly spec data rather than functions, as you can see above.
2019:09:06 22:48:20 gerred @seancorfield are you using valid? and conform in production paths or in tests or their own specs.clj file? that is to say, are you paying the performance costs at runtime for the specs you do have?
2019:09:06 23:21:52 seancorfield @gerred We use conform and invalid? mostly, then explain-data to drive heuristics for turning spec errors into codes/messages we can return to client apps. But, yes, that's all production path code.
2019:09:06 23:23:16 seancorfield A lot of our input form data and most of our API input parameters are run through Spec. We have conforming Specs that accept strings and will conform them to simple numbers, dates, Booleans etc. Something that Alex frowns on 🙂 but it has worked exceptionally well for us.
2019:09:06 23:24:33 seancorfield We usually write the specs to accept either the native type we want or a string that can be parsed/coerced/conformed to that native type, and our specs have generators that produce strings (we mostly generate the underlying spec and then fmapstr over that).
2019:09:07 00:29:13 johanatan is stest/instrument known to work for spec'd reagent components (returning hiccup data structures)? once i enabled it, my browser started complaining that it didn't recognize the elements emitted (which were non-standard html elements which had apparently not been interpreted by react/reagent beforehand).
2019:09:07 21:36:06 borkdude @johanatan fwiw, I do have an fdef for one reagent component in our app, but that's a form-3 component: (defn foo [opts] (r/create-class ...))
2019:09:07 22:24:26 johanatan Yea perhaps it would be less of a problem for form-2 or form-3 components. @borkdude
2019:09:09 19:17:13 Nolan may have been discussed before, but is there a way to install a custom randomness source into spec/gen?
2019:09:09 19:21:32 alexmiller gen rides on test.check
2019:09:09 19:21:50 alexmiller test.check does have pluggable randomness, although there are some caveats about it as it's designed to be repeatable
2019:09:09 19:22:10 alexmiller prob best to hit up @gfredericks for the details
2019:09:09 19:22:33 alexmiller from spec side, the big thing is that stest/check reports the seed it used and accepts a seed as an option to pass along
2019:09:09 19:27:53 gfredericks t.c isn't wired up for that, but the protocol is in place, so currently you could pull it off by monkeypatching random/make-random
2019:09:09 19:28:04 Nolan right. i traced it down to the relevant files, and quickly realized that there are a number of features about the existing random interface that id want to keep in almost every case
2019:09:09 19:29:18 gfredericks What's the motivation?
2019:09:09 19:30:07 Nolan using the current spec-alpha2.gen whats the best way to create a generator that basically calls a function and returns the value? looking for something like gen/return that evaluates x in each generation
2019:09:09 19:30:45 Nolan it may sound weird, but im aiming for a generator that produces a fixed number of CSR bytes
2019:09:09 19:32:15 gfredericks Sounds like something better addressed at the generator or spec level, if I'm understanding you
2019:09:09 19:32:42 gfredericks Or is this some kind of security thing where you want High Quality Randomness?
2019:09:09 19:32:43 Nolan yeah, it definitely is. i was mistaken to think that it would be interesting to aim at the test.check level
2019:09:09 19:34:05 gfredericks gen/fmap is the normal trick for making a generator from an impure function, with all the usual caveats
2019:09:09 19:34:50 Nolan i do want a certain quality of randomness, but more generally im looking for something that would let me (gen/let [x (some-fn-producing-x)] x), but gen/let isnt exposed in spec-alpha2.gen. im using fmap now, but didn’t know if that was the right way:tm:. appreciate the input!
2019:09:09 19:38:41 gfredericks There is no right way, because it circumvents determinism, growth, and shrinking; but spec has generally made theses things a lot murkier
2019:09:09 19:40:34 Nolan makes total sense. thats what i quickly realized when i was digging into the test.check randomness. those are some really neat files, way above what im trying to do
2019:09:09 19:40:50 Nolan awesome work!
2019:09:09 19:41:50 Nolan am i correct in labeling something like this (partial gen/fmap f) a “combinator”?
2019:09:09 19:41:58 Nolan or would that have a different name
2019:09:09 19:49:09 gfredericks fmap is a generator combinator, and partial is a function combinator; the result is also a generator combinator I suppose
2019:09:09 19:51:09 Nolan ideal
2019:09:10 14:21:47 Nick I’d like to be able to add a docstring to my specs, to document particular points of interest about one spec versus a similar one. Looks like there’s an issue on the JIRA but it hasn’t been updated in some time. Is there a recommendation as to how to handle this yet? https://clojure.atlassian.net/browse/CLJ-1965
2019:09:10 14:22:16 alexmiller We're planning to work on it for spec 2, but currently, no
2019:09:10 14:44:48 Nick Cool, thanks. I’ll just drop the comment in my code next to the spec def then.
2019:09:11 18:33:37 Nolan once it leaves alpha of course, is it a priority for spec to expose the same api between CLJ and CLJS?
2019:09:11 18:47:47 alexmiller yes
2019:09:11 18:48:56 alexmiller I'm not sure whether it's possible to guarantee 100% fidelity between the two, but certainly should be close. afaik it is mostly the same now (there are some necessary exceptions around the gen namespace, but those should be able to move closer)
2019:09:11 19:08:42 Nolan impeccable!!! thanks for the info. and totally—they are (at least for us) entirely cross-compatible for spec1. ive gotten so fond of s/schema and s/select that now im almost disenchanted when working with our current specs 😂
2019:09:12 18:21:00 leongrapenthin im wondering whether spec2 is missing feedback/adoption because of cljs?
2019:09:12 18:23:50 hkjels I would definitely have used spec2 if it was available in cljs
2019:09:14 18:10:10 johanatan same
2019:09:12 18:21:59 leongrapenthin id love to try it out and give feedback already, but all my projects are fullstack or cljc
2019:09:12 18:29:31 ghadi @leongrapenthin spec2 isn't even done
2019:09:12 18:30:37 ghadi once the design settles, will be ported to cljs
2019:09:12 18:30:46 leongrapenthin cool
2019:09:12 19:17:16 seancorfield Heh, we're about as leading edge as you'll find, for running pre-release versions of Clojure and Contrib libs but even we're not using Spec2 on the backend yet. Still far too "not done". We do have a branch based on it, passing all our tests, but was a bunch of work to get the Spec1 code running on Spec2.
2019:09:12 23:11:27 andy.fingerhut This is perhaps a bit tangential to spec, but I was reminded recently that a handy technique for making generative testing more time-efficient in finding problems in your code, is if you can tweak parameters in your code to make sub-parts of your data structures smaller. For example, a 32-way branching tree structure like PersistentVector takes 32 conj's or pop's before it actually "does something" to affect the structure of the tree, but if you tweak the code to make it 4-way branching instead, you can make more interesting tree structures and test them with generative testing much, much faster, and the bugs are likely to be identical, except for the size of the test cases.
2019:09:12 23:12:40 andy.fingerhut A technique I saw used in hardware simulation tests long ago (where the simulations are much much slower than the real hardware devices), but hadn't been reminded of in a while.
2019:09:13 22:26:15 gerred i must be tired and missing something
2019:09:13 22:26:31 gerred i'm converting a spec from s/? to requiring exactly one argument.
2019:09:13 22:26:44 gerred i'm staring at the source and can't figure out the arg spec I want.
2019:09:13 22:28:43 gerred oh my gosh
2019:09:13 22:28:44 gerred nm
2019:09:15 09:20:13 ikitommi Is it intentional that s/valid? can throw? should it return false in case the validation fails on exception?
2019:09:15 09:20:43 ikitommi e.g.
(s/valid? empty? 1)
; Syntax error (IllegalArgumentException)
; Don't know how to create ISeq from: java.lang.Long
2019:09:17 04:07:34 ataggart Can't speak to intention, but that predicate has an invariant assumption: that the arg is a collection. Neither true nor false would be valid answers. The error can be avoided by first checking that the invariant assumption holds:
user=> (s/valid? (s/and coll? empty?) 1)
false
2019:09:17 14:31:21 dmarjenburgh Is it possible to dynamically generate specs at runtime? For example, based on a runtime value, I call a fn that returns a vector of namespaced keys. I then generate a spec like (s/keys :req vector-of-keys). Since keys is a macro, I haven’t been able to find a way
2019:09:17 14:46:51 alexmiller you'll need to eval somehow, either by explicitly using eval or by wrapping in another macro that uses backtick
2019:09:17 14:47:11 alexmiller (in spec 2, we've made this much easier)
2019:09:17 14:51:04 dmarjenburgh Great, looking forward to spec2
2019:09:19 16:11:45 dominicm Has anyone experimented with making specs for stateful things, with the intention of using mocks for those values?
2019:09:19 16:11:57 dominicm Thinking in the context of dependency injection somewhat.
2019:09:19 16:13:40 Joe Lane @dominicm Do you mean something like
2019:09:19 16:16:26 dominicm oh, wait. That's not quite what I was looking for. But still very interesting in what I'm thinking of 🤔
2019:09:19 16:17:14 dominicm that's really neat. I didn't know you could do that. That's actually really handy for what I'm working on.
2019:09:19 16:19:58 dominicm I'm actually thinking of the case where you have a function like invoke-service, and you want a dumb wrapper which can figure out what to call it with (from it's registry of "stuff") based on the spec.
I'm a bit frustrated with the pattern of passing around a grab-bag of state which inevitably grows into an unrepl-able mess.
2019:09:20 09:21:01 djtango instrument actually lets you go even further and completely replace the function
2019:09:20 09:23:33 dominicm Yeah, that's great at the repl. I don't want to do that in production though 🙂
2019:09:20 09:24:02 djtango ah - sorry, I am still catching up with the discussion though have found the intercepting properties useful at test-time
2019:09:20 09:25:03 dominicm I'm a little unsure on that. Parallelizing tests is valuable for speed (until Rich invents his tool for running only the required subset of the tests). I don't know how much I love that idea really.
2019:09:20 09:25:42 dominicm But yeah, if you're willing to trade those two properties, it's all good.
2019:09:20 09:34:44 djtango mm yeah fair enough.
So is your use-case this:
use fdef to document what deps your function needs
have some kind of wrapper that then knows how to extract and inject only those deps into a calling function?
2019:09:20 09:37:18 dominicm Yeah. I'm coming round to the idea that I might be barking up the wrong tree though 🙂 I'm thinking a little more time in the hammock should help me puzzle out the use cases.
2019:09:20 09:37:44 dominicm I'm trying to exploit maximum leverage without creating unnecessary verbosity. It's a hard balance 🙂
2019:09:20 09:45:21 djtango I guess I can in theory see how you could fetch the fspec then use a generator, to satisfy the s/keys :req portion but feels brittle, if only as I'm not convinced it's an appropriate usage of the tools
2019:09:20 09:47:20 dominicm Me neither 🙂
2019:09:20 09:49:49 djtango sounds tricky - would be interested to know how you get on
2019:09:20 09:51:58 dominicm I'm hoping to publish my results somewhere, I think I'm onto something, I've reduced the initial problem space into something more palatable. But I still have to handle the stateful dependencies part.
2019:09:19 16:22:37 Joe Lane I'm not sure I understand the usecase for the dumb wrapper. What is registry of "stuff"?
2019:09:19 16:24:17 dominicm I suppose something to the effect of, I have a "system" like: {:db db-conn :some-stateful-service statey} and I want to direct things directly to a function defined like:
(defn add-user
[db add-user-data] …)
So the "registry" is that first map.
2019:09:19 16:25:43 Joe Lane So, in component terms, a "system"
2019:09:19 16:25:45 Joe Lane k
2019:09:19 16:25:57 Joe Lane (Or a pedestal context map)
2019:09:19 16:26:35 dominicm Yeah. Exactly. But passing around systems is an anti-pattern. And I also can't figure out what's in my pedestal context map half the time, because there's lots of interceptors messing with it, and it's unclear why routeA gets :mongodb and routeB doesn't. (So I'm trying to come at this with a new angle of positional parameters)
2019:09:19 16:31:10 Joe Lane thinking
2019:09:19 16:33:37 Joe Lane So, is the goal here to assert a property about the dumb wrapper but you're trying to determine the best way to inject actually stateful mocks?
2019:09:19 16:34:59 dominicm I guess I could write a spec parser for s/cat? And that would let me figure out the arguments in order.
Well, actually I was thinking that it would be handy to just start by being able to call add-user by using it's fdef to figure out that it needs a database, and getting one from the system. (and somehow connecting ::db/db as a spec to that)
2019:09:19 16:35:12 Joe Lane (= dumb-wrapper add-user)
2019:09:19 16:35:15 Joe Lane right?
2019:09:19 16:35:39 Joe Lane oh.
2019:09:19 16:36:27 dominicm yeah, right 🙂
2019:09:19 16:37:54 Joe Lane I might be wrong, but it sounds like you want an integration between your DI tool (are you using component?) and the s/select facilities in spec2. If you're using component though, shouldn't that fdef to figure out.... step be happening at system/lifecycle start time?
2019:09:19 16:37:57 dominicm But I'd be using this in production, not just for testing. So I would know that this function only takes 1 non-stateful argument (the "event", or maybe "req" for http). So I just need to get the rest of the arguments.
2019:09:19 16:39:08 dominicm I'm not using component. The reason being that a system probably ends up with ~50 or so handlers, and the boilerplate involved is quickly tedious. Although maybe I should just use a macro for that 😛
2019:09:19 16:39:24 dominicm The way I see it, I have actual stateful things (e.g. db conn) and things that want to use those things.
2019:09:19 16:40:04 Joe Lane This is starting to feel more and more like spring style annotation DI vs data oriented component DI. And further away from a problem related to spec.
2019:09:19 16:42:04 dominicm A little, yeah. And that's somewhat intentional. I see one of two patterns in this space: 1. create a map with everything in, call it "deps" and hope the function you're passing it to understands it (and there's a bunch of bad patterns which fall out of this). 2. roll entire namespaces into being partial applied with their state available, and passed in as arguments.
2019:09:19 16:44:02 dominicm Tbh, I expect there's not a good spec function for figuring this out 🙂 As I have a lot of contextual awareness that spec doesn't have. (e.g. I know it's always a list, and I will always know everything except 1 argument).
2019:09:19 16:45:59 Joe Lane I think the 3rd pattern is component/system/mount oriented, where functions are passed positional arguments (not a deps/req/context map) which were determined by the DI tool. I don't think you can reduce that kind of complexity, only choose where to solve it, 1 large place (component) or in every dumb-wrapper by declaring the deps at the callsite. I definitely don't think spec is going to be helpful here.
2019:09:19 16:47:08 dominicm I haven't seen 3? what does that look like
2019:09:19 16:49:01 dominicm oh, maybe I do know what you mean.
2019:09:19 16:50:17 dominicm I guess pattern 3 here is implemented in terms of my listed patterns 1 & 2.
2019:09:19 19:28:13 jaihindhreddy I just started solving some basic programming puzzles with Clojure, with the constraint that I want to spec the solutions.
I hit a snag right on the first one.
Here's the fn I want to spec:
(defn two-sum
"Returns indices of two elems in ints that add up to sum"
{::url ""}
[ints sum]
(loop [l 0
r (dec (count ints))]
(let [s (+ (nth ints l) (nth ints r))]
(cond (= l r) nil
(= s sum) [l r]
(< s sum) (recur (inc l) r)
:else (recur l (dec r))))))
2019:09:19 19:30:03 jaihindhreddy The puzzle is to find two (different) indices s.t. the elements in ints in those indices add up to sum.
The catch is: there is always exactly one solution. How do I spec my args to get gen right?
2019:09:19 19:31:11 jaihindhreddy Unless I use the fn itself in it's spec (or trusted equivalent), I can't seem to get gen. to work.
2019:09:19 19:52:06 seancorfield @jaihindhreddy That sounds like an external constraint on the data? An arbitrary vector of 2+ ints and an arbitrary sum aren't going to satisfy that condition so the behavior of two-sum on such data will be...? Undefined? nil?
2019:09:19 19:52:39 seancorfield Your :ret spec doesn't account for the function returning nil -- which it clearly can -- so your spec isn't right as it stands.
2019:09:19 19:55:02 seancorfield
user=> (two-sum (into [] (range 10 20)) 9)
nil
So your :ret spec needs to be s/nilable or use s/or and then your :fn spec needs to accept that (:ret %) can be nil (or should satisfy that complex predicate).
2019:09:19 19:58:24 jaihindhreddy It is an external constraint on the data. I thought of using s/nilable there but actually the fn is not supposed to be called with such args and its UB, and I'm trying to get the generation to work in such a way that there exists exactly one answer.
2019:09:19 19:59:45 jaihindhreddy I'll probably look into using fmap to generate the sum from the ints. Thanks for your help!
2019:09:19 20:03:52 seancorfield And it's also an ascending sequence of ints, yes? (based on the < logic in there)
2019:09:19 20:05:47 seancorfield I think what you're attempting is a bit self-defeating: the generator and the :fn spec together are pretty much going to be a re-implementation of the function logic at this point...
2019:09:19 20:07:06 jaihindhreddy Kinda reckoned that myself.
2019:09:19 20:09:18 jaihindhreddy The fn returns a pair of indexes into the ints arg, so I'm checking that the second one in the return value is less than the count of ints.
2019:09:19 20:45:22 vlaaad noob question: why spec forms sometimes have qualified, and sometimes unqualified symbols?
2019:09:19 21:03:40 alexmiller Should always be qualified
2019:09:19 21:03:55 vlaaad agree!
2019:09:19 21:04:00 alexmiller There are some pending patches for this stuff
2019:09:19 21:08:39 vlaaad I'm fascinated by spec, sprinkling it a bit here and there to document/enforce contracts where it makes sense... do you think it's the future of strong gradually static type systems?
2019:09:19 21:16:32 vlaaad like being able to say something along the lines of
let n: int? = (get-some-int)
let x: (and int? even?) = n ;; checks for `even?` during casting
2019:09:19 21:16:37 Joe Lane I think it will push significantly more boundaries than just "type systems"
2019:09:19 21:18:53 Joe Lane I also think the traditional notion of static type systems look very different if you can work in a dynamic system.
2019:09:19 21:19:10 Joe Lane But what do I know :man-shrugging:
2019:09:19 22:05:56 seancorfield I don't consider it "like" a type system (and I think it's a bit misleading to think of it in those terms).
2019:09:19 22:07:38 seancorfield We've been using spec in production code very heavily since it first appeared. We don't use it much on functions (which is the only "like types" part of it -- and even that isn't much like a "type system").
2019:09:19 22:09:12 seancorfield We mostly use data specs and validation/conformance. We use it to generate (random, conforming) data for example-based tests. We use it for generative testing of some things. We use it via instrument a little bit during dev/test.
2019:09:20 05:41:32 vlaaad Well, it's not like a type systems of today :)
2019:09:20 17:51:53 andy.fingerhut Well, the full generality of spec allows arbitrary code in predicates, so it is in general computationally undecidable to statically check that a program meets such a general spec.
2019:09:20 17:53:21 andy.fingerhut spec has been designed with run-time checking (and generation, etc.) in mind, not trying to limit itself to what can be statically checked, although subsets of it can be: https://github.com/arohner/spectrum
2019:09:20 20:50:04 seancorfield Because folks keep asking about our "coercing specs" at work, we just open-sourced them https://github.com/worldsingles/web-specsworldsingles/web-specs {:mvn/version "0.1.0"}
2019:09:21 05:20:07 dominicm Any regrets using conformers?
2019:09:21 05:26:24 seancorfield Nope, none at all. I originally submitted a talk to Conj about it (and it was accepted) but I pulled it because Alex was so "anti" the whole conformer thing. That was years ago. But I still think we made the right decision and I'd stand by it every day.
2019:09:21 07:04:25 jumar Awesome, thanks for that.
2019:09:22 09:56:28 Sebastiano Barrera Hi y'all... I'm a beginner tinkering around with spec (on CLJS) and I was wondering: is there a way to encode a spec so that a certain keyset is required only if another key has a certain value? For example, I might have {:type :assignment, :left ..., :right ...} or {:type :identifier, :name 'some-name}; could I express the requirement that :left and :right must both appear iff :type is :assignment?
2019:09:22 10:03:11 Sebastiano Barrera Wait, I think I figured it out... I'm guessing
2019:09:22 11:02:26 Sebastiano Barrera Oh, let me take a look at that
2019:09:25 17:07:08 sundbp I’ve got a question about spec2, how do I do something like this:
(s/def ::foo (partial bar 1)) - in spec2 that fails. Wrapping the partial in e.g. s/spec is no solution.
2019:09:25 17:57:53 seancorfield I believe function literals are accepted @sundbp -- try (s/def ::foo #(bar 1 %))
2019:09:26 14:02:20 sundbp Thanks. Will give that a go
2019:09:25 18:06:20 seancorfield Yeah, that works:
2019:09:25 19:01:19 andy.fingerhut I can dig into the implementation and find out, but does anyone happen to know whether test.check/quick-check calls clojure.test/is or clojure.test/are internally, and thus is intended to be called inside of a clojure.test/deftest form directly?
2019:09:25 19:01:39 andy.fingerhut The doc string says it runs multiple tests, but isn't clear about whether there is a return value
2019:09:25 19:03:50 andy.fingerhut Perhaps one reasonable way to use test.check/quick-check inside of a clojure.test/deftest would be (is (:pass? (test.check/quick-check ...))) ?
2019:09:25 19:06:34 andy.fingerhut Ah, from looking at test.check's own deftest's, it appears that it often uses (is (:result (test.check/quick-check ...))). I will use that, too.
2019:09:25 19:10:49 andy.fingerhut Is there some existing test.check docs that describe the return value of quick-check that I could learn this from? Or perhaps a small addition to the quick-check doc string might be welcome?
2019:09:25 19:16:05 gfredericks @andy.fingerhut hi
2019:09:25 19:17:00 gfredericks The docstring of the latest version is pretty explicit about the return value
2019:09:25 19:17:49 gfredericks You're correct that the quickcheck function is unrelated to clojure.test; you can integrate manually as you noted, or by using the t.c.clojure-test namespace
2019:09:25 19:24:23 andy.fingerhut ok, I may have been checking the quick-check doc string of an older version that core.rrb-vector is using. Let me look at the latest.
2019:09:25 19:24:56 gfredericks The past was a terrible time
2019:09:25 19:25:18 andy.fingerhut OK, much better now. Thanks!
2019:09:25 19:27:53 gfredericks NP
2019:09:27 10:09:19 lmergen is it possible to somehow tokenize a string using specs, in a s/cat-like way?
e.g.
(which doesn't work for obvious reasons, but that's the idea of what i'm looking for)
2019:09:27 10:11:04 lmergen actually i'm probably looking for a way to compose regexes more than i'm looking for this spec-like functionality... 🤔
2019:09:27 10:21:15 alexmiller You can do this (by seq-ing a string) but I generally think it’s a bad idea. There is actually a regex composition library for Clojure, but I’m blanking on the name
2019:09:27 10:27:22 lmergen i see, that actually looks interesting
2019:09:27 10:27:39 lmergen thanks
2019:09:27 12:37:50 jonpither Hi. is there a generalised pattern / approach for handling config, in that a) defining the specs for the config attributes, b) documenting the individual options, c) providing default values for options? I realise this Q is a bit wide ranging, but I'm wanting to take a step back and to see what others usually do for config input
2019:09:27 13:47:11 taylor I don’t have an answer for your exact question, but I’ve spec’d CLI args before which is kinda like config https://github.com/taylorwood/clojurl/blob/c69cef7c1f2b46ac9034cf75ea1c5f7d3abf6d2b/src/clojurl.clj#L13-L42
Re: documentation, spec doesn’t really support it (yet?), and I probably wouldn’t use spec to influence config default values. That said, there’s probably an opportunity to pull pieces of this together into an “opinionated” config library :man-shrugging:
2019:09:27 14:27:07 jonpither Thanks @U3DAE8HMG will check this out
2019:09:30 08:56:58 jaihindhreddy spec might support docstrings but probably won't support metadata, because that might cause specs to become bloated and less suitable for a variety of usecases. Because ns-qualified keywords come with no strings attached, spec is able to associate global semantics in particular contexts (when calling s/validate, for example). In a same way, maybe something like aero can maintain its own global registry of documentation, defaults etc. and be useful in contexts that make sense for it, perhaps pawning off the job of validation to spec.
@U050DD55V sorry for being vague 😅
And pointing to your own library (`aero`) 😛
2019:10:16 17:02:14 rickmoynihan @U050DD55V we have an inhouse tool that we use to specify service requirements and dependencies in EDN… it’s meant to tie together CI services with what they expect at runtime etc, and help in pulling clojure services and service dependencies together across various environments…
e.g. in dev you want to run the services with repls and various things set… in prod you want packer images built with config files written in the right places that define parameters to be injected into env files and config files etc.
And all environments want to pull in dependent services.
Anyway part of it is aero; and each service declares things like:
:omni/keys {:stardog/stardog-home {:doc "Location of the stardog home directory"
:default [:install "stardog-home"]
:env-var STARDOG_HOME}
:stardog/java-args {:doc "Arguments to the JVM when running stardog"
:env-var STARDOG_SERVER_JAVA_ARGS}
:stardog/temp-dir {:doc "Java temp directory for the stardog process"
:env-var STARDOG_JAVA_TEMP_DIR}
:stardog/logs-dir {:doc "Directory to write the stardog.log file to"
:default "."}}
2019:10:16 17:03:29 rickmoynihan which includes the doc string, and where the variable comes from etc.
2019:10:16 17:04:17 rickmoynihan so services can inform things further up the chain what they need to set; and display a useful error when something is missing
2019:09:30 10:08:01 shan Basic question about spec, but I would expect clojure to complain if a spec doesn’t exist but seem it just seems to ignore the fact:
Any idea if there’s a reason for this or how you usually handle being aware of missing specs?
2019:09:30 10:16:35 jaihindhreddy I'm guessing that's because spec is an open system. When you say I need ::a to be present in the map, s/keys will verify that, and if you have specified what ::a should look like, then it will verify (and generate) that too correctly. This is what allows us to spec as little or as much as we want about our domain.
2019:09:30 10:17:40 jaihindhreddy Also comes in handy when doing interactive development, where spec doesn't become this thing that is constantly bothering you about missing spec definitions. You get to decide how much you want to spec, and accordingly you get back as much value and precision in validation and generation. Hope that answers your question.
2019:09:30 10:29:09 vlaaad Code above looks like a bug to me.
2019:09:30 10:30:21 vlaaad by specifying :req-un, you asked spec to check that :req-does-not-exist key should be checked against ::req-does-not-exist spec during a call to valid?
2019:09:30 10:31:23 vlaaad it's one thing to have forward declarations in spec where you don't check if referenced spec is declared during "definition phase", and another to ignore required keys during "validation phase"
2019:09:30 10:40:40 shan ^^ yep, that’s what I was thinking
2019:09:30 10:58:43 jaihindhreddy Yeah. That does seem like a bug. Didn't read the whole thing. My bad.
2019:09:30 10:42:00 shan it seems to mean you could add a missing spec to :req/opt-un and add it to the data but this wouldn’t actually be checked or give any indication the spec doesn’t exist
2019:09:30 23:04:02 shan Cheers dude. You’re always a good source for clojure knowledge 🙌
2019:09:30 13:02:46 conan This seems fine to me. You're ensuring that a map contains a specific key, but making no claims about the value of that key. I do this frequently when receiving required but complex data from external systems - i want to check that the data is there, but i don't know enough to write a comprehensive spec for it.
2019:09:30 16:55:10 sgerguri Here is my situation: I have a top-level spec that allows for two situations - cluster updated and cluster removed. This is handled by two multispecs as follows:
2019:09:30 16:56:20 sgerguri Each of those multispecs has two implementations. The implementations generate nested maps, where some of the nested maps have generators that generate matching input, but conforming strips away extra data as specified.
2019:09:30 16:58:02 sgerguri Now, I would like to call (s/conform ::cluster (g/generate (s/gen ::cluster))) and get either a :cluster-removed or :cluster-updated data, but conformed recursively all the way through. However, conformance does not seem to propagate to the level of the multispecs, it simply stops at the level of ::cluster.
2019:09:30 16:59:33 sgerguri I can work around it by examining the tag I get from ::cluster, then conforming again against either ::cluster-updated or ::cluster-removed, which does the correct thing against the correct implementation. But ideally, I would like to conform only once and have the correct thing happen. Is there a way to do that?
2019:09:30 17:00:43 sgerguri FYI - I have two multispecs there as unfortunately multi-spec seems to expect a single field as a dispatch tag, even though the multimethods themselves can dispatch on more than one field (through the use of juxt).
2019:09:30 17:01:31 sgerguri @alexmiller ^ Will definitely appreciate some thoughts on this situation! 🙇
2019:09:30 17:05:21 alexmiller well on the last point first, you can use an arbitrary function for the multi-spec, doesn't have to be a keyword
2019:09:30 17:05:41 alexmiller so you could for example use juxt to produce a vector and dispatch from a single multimethod
2019:09:30 17:05:47 alexmiller maybe that solves your whole problem, not sure
2019:09:30 17:06:00 sgerguri What would I use for a retagging function though?
2019:09:30 17:06:14 alexmiller it's used to drive gen
2019:09:30 17:06:48 alexmiller it will pick a random multimethod, gen from that, then "retag" to something passed to the multimethod
2019:09:30 17:07:47 alexmiller here it would need to go from the dispatch value (the vector output of juxt) to modify the generated defmethod map to get back to your original expected input
2019:09:30 17:08:17 alexmiller depending on your method generators, there may be nothing to do and you could just use identity
2019:09:30 17:08:27 alexmiller or you could assoc back in the keys and the values from the juxt vector
2019:09:30 17:08:30 sgerguri With juxt I'd be ideally looking at something as follows:
I'm just not sure what to put in instead of the ?.
2019:09:30 17:09:34 alexmiller as above, identity might work, or you could use (fn [[type action]] (assoc % :cluster-type type :action action))
2019:09:30 17:09:53 sgerguri Ah, identity might just do the trick. Let me give that a try!
2019:09:30 17:10:01 alexmiller or maybe I've got the signature wrong there - does retag fn take 2 args?
2019:09:30 17:10:33 alexmiller yeah, it's value and dispatch value
2019:09:30 17:10:36 sgerguri I think it only takes one? It can also be a keyword, which would support that argument.
2019:09:30 17:10:58 alexmiller so should have been (fn [gen-val [type action]] (assoc gen-val :cluster-type type :action action))
2019:09:30 17:10:58 sgerguri Ah no, it actually takes two. But apparently one can supply a keyword as well.
2019:09:30 17:12:01 alexmiller I think that's right. I haven't done one of these in a while.
2019:09:30 17:12:03 sgerguri Why is the retagging even necessary, when the multimethods return a spec that is narrow enough to always generate the right data? I guess that's one thing that's confusing me.
2019:09:30 17:20:27 sgerguri Hm, this did not really solve the problem - but thanks anyway. I'll post here if I manage to solve this in a satisfactory fashion.
2019:09:30 17:25:26 sgerguri Actually, I take that back - I had a dangling redef on the multispec elsewhere. Doing it as per Alex's suggestion:
2019:10:01 23:01:51 olivergeorge It'd be nice if defn did s/instrument when you're working at the repl.
2019:10:01 23:02:15 olivergeorge Any chance something like that is on the radar?
2019:10:01 23:14:14 alexmiller I think that’d be wrong for everyone sometimes
2019:10:01 23:15:52 alexmiller I’d prefer to have a lot more control over things like that
2019:10:01 23:17:56 seancorfield Agreed. I definitely would not want auto-instrument at the REPL (and how would the compiler tell the difference between REPL and regular running since the semantics are the same?).
2019:10:01 23:18:31 seancorfield (remember that not everyone AOT's code!)
2019:10:02 11:29:03 holyjak @alexmiller I have a tiny improvement to gensub to provide a meaningful error when such-that fails, is it meaningful for send a patch to https://clojure.atlassian.net/browse/CLJ-2097? Essentially (gen/such-that #(valid? spec %) g 100) -> (gen/such-that #(valid? spec %) g {:max-tries 100, :ex-fn (fn [{:keys [max-tries]}] (ex-info (str "Couldn't satisfy such-that predicate after " max-tries " tries.") {:path path}))}) (Since path is the only thing that looks meaningful, contrary to test.check's gen, pred and spec's spec, form)
I have updated the issue, let me know there whether to send a patch. Thank you!
2019:10:02 12:39:06 alexmiller test.check 0.10.0 has some new hooks for this case too
2019:10:02 12:41:27 alexmiller if you have a patch, go for it, but I'm not sure whether we're going to go back and patch anything on spec.alpha. this is tbd, but my current thought is that once we get close to a spec 2 release I would reassess all the spec tickets (I know some have been fixed) and try to do a clean-up wave. this will require re-working any of the patches as I'd expect none of them to apply to spec 2.
2019:10:02 15:05:11 holyjak Thanks, @alexmiller! Question 2: I discovered that it is even more useful to include an example of why the spec failed on sample data - knowing which spec failed to match is good but knowing why is even better. In my case the spec failed was something like ::person because of invalid :person :address :zip. So, to troubleshoot my issue, I include this in the thrown error:
- which was extremely useful for me but not sure whether it would be an appropriate general solution.
2019:10:02 15:06:34 kenny @holyjak YES. The failing args and ret should be easily shown in the error message. I don't think that particular approach will work because there's no guarantee you'll hit the failing case with one sample but I like the idea.
2019:10:02 15:08:30 alexmiller showing an example which isn't the one that actually failed seems more confusing to me
2019:10:02 15:09:34 kenny Yes - it needs to be the failed one.
2019:10:02 15:09:44 holyjak ALL the examples fail
2019:10:02 15:09:51 kenny We have internal code that does this already but it's a hack on gensub.
2019:10:02 15:10:02 kenny Not always @holyjak.
2019:10:02 15:10:53 holyjak Well, in theory you are right. What we call is `(gen/such-that #(valid? spec %) g {:max-tries 1000 ...}) - none of the 1000 generated samples failed. What is the chance that the one we generate in the error does not fail?
2019:10:02 15:11:56 kenny Depends on how you're controlling the size.
2019:10:02 15:12:08 holyjak What do you mean?
2019:10:02 15:12:29 holyjak Of course I could add a check and include :sample-explain only if it really fails. Would that be satisfactory? If not - what is a good way of capturing some generated, failing data?
2019:10:02 15:12:43 kenny IIRC, gen/sample does not use all 200 size values.
2019:10:02 15:13:11 kenny Using quick-check or st/check will.
2019:10:02 15:13:55 holyjak I am not sure how such-that invokes the generator but I would assume that it does so in a way very similar to sample?
2019:10:02 15:15:06 holyjak Anyway, what do you think about the "best effort" approach - generate a sample, if it fails the spec, include :sample-explain in the error. I think that would work in 99.9% cases?
2019:10:02 15:15:56 kenny Anyway, the piece I find valuable is being able to easily see why a check failed, whether it be by generator failure or check failure. When a generator fails, I would like to see the failed samples. This usually lets me figure out which predicate is causing the issue pretty quick. In the case the check fails, I'd like to see the args and the ret.
2019:10:02 15:17:08 kenny You should know which samples failed while generating @holyjak, no need to generate new ones.
2019:10:02 15:20:06 kenny Our hack (not acceptable for a patch but let's us debug generators without going crazy) is to store invalid values in an atom and display that in the error message.
2019:10:02 15:20:33 kenny So we just change the such-that predicate to do that.
2019:10:02 15:22:09 holyjak Smart! The question is: What approach would be acceptible for a patch (and useful for the users!)?
2019:10:02 15:23:13 kenny I don't know. It definitely feels too hacky for a patch. It may be the only way to do it though. When I was looking at this, I don't think I saw any test.check hook to grab the failed values.
2019:10:02 15:24:26 kenny Just speculating but I imagine it'd be somewhat easy to add that hook to test.check if it isn't already there and is the proper solution.
2019:10:02 15:25:54 kenny I would definitely be in favor of adding some sort of patch to spec1 alpha given we don't know when spec2 is coming. Debugging generators is super painful as it is.
2019:10:02 15:25:58 gfredericks Check the such-that docstring in the latest version
2019:10:02 15:26:52 kenny Doesn't appear to have anything that stores failed samples.
2019:10:02 15:27:20 gfredericks Would you want all of them? Just the latest?
2019:10:02 15:27:38 gfredericks Having the generator means it should be easy to generate more
2019:10:02 15:27:43 kenny Probably all.
2019:10:02 15:28:26 kenny If I have all, I can easily only have the latest if I want.
2019:10:02 15:28:34 holyjak I believe a single value is enough (preferably the 1st as it is the simplest one)?
2019:10:02 15:29:07 kenny True.
2019:10:02 15:29:10 gfredericks Keeping all could be a GC issue
2019:10:02 15:29:18 holyjak > Having the generator means it should be easy to generate more
That is what I do in my solution (generate a new value using sample) but it has been argued that using one of the actually failed would be better.
2019:10:02 15:29:46 kenny I guess I only ever work with one of the failed samples anyway.
2019:10:02 15:31:13 holyjak @gfredericks As I explained above, this popped up when working with specs - a custom generator generated value that did not conform to the spec. I can get the name of the spec but want to see why the custom gen failed to provide anything valid. There, having a sample value and running s/explain-data on it is sufficient and very useful.
2019:10:02 15:31:23 kenny Looks easy to add in such-that-helper.
2019:10:02 15:31:47 holyjak I am creating an issue on test.check to pass a sample failed value to ex-fn. Stop me if I should not 🙂
2019:10:02 15:32:04 gfredericks You shouldn't not
2019:10:02 15:36:04 holyjak Conclusion: I create a patch for Spec to include the spec name (or rather path) in the such-that failures and we wait for test.check to expose a sample failed value to the :ex-fn before adding explain-data of it.
2019:10:02 15:39:14 holyjak @alexmiller is there any point in providing also a similar patch for Spec 2 or is it too much in flux?
2019:10:02 15:40:41 alexmiller I'm not going to do the sample thing, so not interested in a patch for that
2019:10:02 15:41:02 holyjak no, not that, just including path
2019:10:02 15:41:06 alexmiller that's fine
2019:10:02 15:41:14 alexmiller I don't think there's any point for spec 2
2019:10:02 15:41:39 alexmiller one possible outcome is that we repackage everything in spec 2 to remove alpha designation, which will (again) break any patches
2019:10:02 15:45:03 alexmiller I think in test.check, while generating in such-that, it would be possible to just retain the first or last gen'ed every time (before the pred check) and report that if you hit the retry limit. that way you're reporting an example that was actually tried.
2019:10:02 15:50:59 gfredericks An example generated after the fact is just as meaningful, philosophically
2019:10:02 15:53:01 alexmiller if only a percentage of them fail, and you happen to generate a sample that doesn't fail, that seems strongly less useful than one of the actual examples that didn't pass the predicate
2019:10:02 15:53:07 alexmiller so I'ma disagree with you on that
2019:10:02 15:53:22 gfredericks I was assuming you'd filter on failure 😛
2019:10:02 15:53:43 alexmiller well you failed to mention that :)
2019:10:02 15:53:48 alexmiller but that would be fine
2019:10:02 15:53:51 gfredericks Which, ironically, could fail 🙄 though presumably is unlikely to
2019:10:02 15:54:23 gfredericks But that might be a good enough reason not to do it
2019:10:02 15:55:58 gfredericks Adding the size to the failure data would assist with that though
2019:10:02 15:57:31 holyjak @gfredericks Will you consider adding a sample failed value to the arguments of such-that's :ex-fn or some alternative improvement?
2019:10:02 16:04:41 gfredericks @holyjak I'm not going to be doing active work on test.check for the foreseeable future, so that'll be up to somebody else; but a jira ticket is definitely the right way to make sure it gets looked at
2019:10:02 16:25:41 holyjak I'd be happy to provide a patch to test.check - and even happier to get any pointers regarding the best solution.
2019:10:02 16:29:07 gfredericks No I meant you should create the ticket
2019:10:02 16:29:31 gfredericks Sorry, I was being cute with English double negatives, it was probably a bad idea
2019:10:02 16:34:49 holyjak I see 🙂 My bad, I should have read more carefully. Also, I believed that English normally doesn't permit double negatives. Always learning 🙂
2019:10:02 16:35:33 holyjak Any pointers regarding the best way to implement this? Is failed-value a good key name to pass to :ex-fn?
2019:10:02 23:11:28 noprompt What I’d like to do is copy ::defn-args and its dependencies into my own project.
2019:10:02 23:14:05 noprompt Do I just put the copy right information above the code I want to copy?
2019:10:02 23:14:55 noprompt Also, my project is MIT licensed, so I’m not sure how all that works.
2019:10:02 23:18:34 noprompt > What are my obligations if I copy source code obtained from http://Eclipse.org and licensed under the Eclipse Public License and include it in my product that I then distribute?
> Source code licensed under the EPL may only be redistributed under the EPL.
2019:10:02 23:19:04 noprompt I’m guessing I need to handwrite my own version of spec.
2019:10:02 23:33:25 seancorfield @noprompt I'm curious as to how you're depending on specs from that namespace since they are intended to support clojure.core macros?
2019:10:02 23:34:03 seancorfield (and, yes, you're going to be on some pretty shaky ground if you copy'n'paste that code into your system and then distribute it)
2019:10:02 23:34:56 noprompt @seancorfield Imagine you want to write a macro that has the form as defn.
2019:10:02 23:36:32 noprompt And imagine that I want to use conform to parse ::core.specs/defn-args.
2019:10:02 23:36:52 noprompt Thats my situation.
2019:10:02 23:37:01 seancorfield Ah, so your code was relying on the internal names used in the those specs?
2019:10:02 23:37:18 noprompt I’m noticing you’re using the word “internal”.
2019:10:02 23:37:34 seancorfield Only two top-level specs changed their names.
2019:10:02 23:38:04 seancorfield If you conform, you have to rely on the implementation of the spec.
2019:10:02 23:38:18 noprompt Yeah but specs are globally public…
2019:10:02 23:38:48 seancorfield We've used alpha versions of Clojure libs in production for over eight years and sometimes Cognitect change stuff while it's alpha and break your code. That's the risk of using alpha libs.
2019:10:02 23:39:11 noprompt I hope you can appreciate the irony in this.
2019:10:02 23:39:22 noprompt 🙂
2019:10:02 23:39:44 noprompt To me its a bit surprising given the critique of SemVer, etc.
2019:10:02 23:39:58 noprompt So suddenly calling it “alpha” makes it okay to make breaking changes?
2019:10:02 23:40:24 noprompt ::defn-args2?
2019:10:02 23:40:28 seancorfield They've been very clear that alpha means "can change/break". Things are only guaranteed additive/fixative once the alpha label goes away.
2019:10:02 23:40:48 noprompt Its not on the README.
2019:10:02 23:40:56 noprompt Where were they clear?
2019:10:02 23:41:25 seancorfield They've said it repeatedly publicly. Pretty sure even Rich has mentioned that in talks...?
But, that is not to say just leave your thing 0.0.967. At a certain point, you are going to have users, and whether you change it to 1.0 or not, they are going to be depending on your stuff.
And that is where spec is going right now IMHO. But his Maybe Not talk and Alex Miller's blog eased my fears of spec remaining in this state forever.
2019:10:02 23:41:42 noprompt Like on Twitter?
2019:10:02 23:41:44 noprompt The ML?
2019:10:02 23:42:20 noprompt I rarely, if ever, watch talks.
2019:10:02 23:42:45 noprompt The README and the source code are far more public.
2019:10:02 23:42:52 noprompt The disclaimer should be there.
2019:10:02 23:43:22 taylor what would it say
2019:10:02 23:43:30 noprompt What Sean said.
2019:10:02 23:44:00 noprompt How about
> Alpha means this library “can change/break” do not rely on it.
2019:10:02 23:44:53 noprompt Because I obviously misunderstood the situation.
2019:10:02 23:45:54 seancorfield I'm sorry you've missed it being discussed on the mailing list and in Rich's talks and here on Slack (where it has come up several times in several situations).
2019:10:02 23:46:36 noprompt I just want answers to my questions.
2019:10:02 23:47:06 noprompt Can/How do I copy the source code?
2019:10:02 23:48:18 seancorfield Not legally into anything you are distributing under an incompatible license.
2019:10:02 23:49:17 seancorfield If you want to use EPL code in your OSS project, consider changing your license to EPL so it's compatible.
2019:10:02 23:49:21 noprompt Okay, that’s what I gathered by reading the EPL website.
2019:10:02 23:50:05 noprompt Its fine, I can write the specs myself in this case. I was just hoping I could take a short cut.
2019:10:02 23:50:07 noprompt 🙂
2019:10:02 23:50:34 seancorfield Given the day I've had working on our Spec 2 branch, I do sympathize (even if it may not sound like it).
2019:10:02 23:51:26 noprompt Honestly, it didn’t feel that way but you saying that makes me feel like you do.
2019:10:02 23:51:50 noprompt I just want to get my users and teammates out of a broken state quickly.
2019:10:02 23:52:58 noprompt I’ve given up on trying to supply feedback or contribute to anything in the clojure repo due to conversations that go the way they do regarding these kinds of frustrations.
2019:10:02 23:53:18 seancorfield Yeah, I can detect a fair amount of frustration/anger...
2019:10:02 23:53:25 noprompt Transparency would be nice on these topics; one can’t be expected to keep up with every feed of data.
2019:10:02 23:53:58 noprompt The repo and source code is about as public and as inclusive as any place to explain semantics.
2019:10:02 23:54:40 noprompt I’ve made the effort in Meander to call out the versioning semantics I employ so there is no misunderstanding.
2019:10:02 23:54:45 seancorfield When Rich introduced spec in 2016, he was pretty clear that it was alpha and subject to change (I just checked the transcript of one of his talks from back then). He also called out alphas as "potentially breaking" in his Spec-ulation talk (which is where his main criticism of SemVer was made.
2019:10:02 23:55:53 seancorfield SemVer itself is broken tho'. While you're in alpha, you can make (breaking) changes. Once you have a non-alpha, you should only make accretive/fixative changes. SemVer doesn't help with that.
2019:10:02 23:55:59 noprompt That makes this assumption that people will be exposed to those things or, if you have my memory, remember them.
2019:10:02 23:56:47 noprompt Just slap a phrase on the README that says, upfront, what the version semantics are instead of relying on them being implicitly understood.
2019:10:02 23:57:11 noprompt What I took from Rich was if I’m going to make a breaking change, I make a new namespace, artifact, etc.
2019:10:02 23:57:21 noprompt And that works fantastic.
2019:10:02 23:57:24 seancorfield If you're not in alpha, yeah.
2019:10:02 23:57:34 noprompt Its a greek letter.
2019:10:02 23:57:45 seancorfield It was an important caveat in his talk 🙂
2019:10:02 23:57:55 noprompt Rich doesn’t get to decide what that means.
2019:10:02 23:58:03 noprompt He can’t have it both ways.
2019:10:02 23:58:05 seancorfield (and it's been the practice in Clojure overall for years)
2019:10:02 23:58:16 noprompt Saying that “alpha” can break is just weird thing.
2019:10:02 23:58:29 noprompt None of these versions can break but this one can.
2019:10:02 23:58:30 noprompt What?
2019:10:02 23:59:01 noprompt But, hey, if the semantics were with the source, fine, whatever, those are Rich’s rules. Cool.
2019:10:02 23:59:43 noprompt My point is they’re not transparent.
2019:10:03 00:00:09 noprompt Put the semantics with the source.
2019:10:03 00:00:15 seancorfield Well, we're just going to disagree on how clear we think the Clojure/core team have been on that... ¯\(ツ)/¯
2019:10:03 00:00:20 alexmiller I've added a warning to the readme of both core.specs.alpha and spec.alpha
2019:10:03 00:00:35 seancorfield Problem solved! 🙂
2019:10:03 00:00:45 alexmiller If you have another suggestion of how to signal this, please let me know
2019:10:03 00:00:59 noprompt I literally have tears in my eyes. 😂
2019:10:03 00:01:02 alexmiller sorry it wasn't clear
2019:10:03 00:01:31 noprompt But they are now! Which is great!
2019:10:03 00:01:33 noprompt 😄
2019:10:03 00:01:57 noprompt You have literally changed my attitude.
2019:10:03 00:02:11 alexmiller I think making alpha ok with breaking stuff is fine, with the caveat that having something in alpha for years and deps of official releases is admittedly weak sauce
2019:10:03 00:02:22 alexmiller yet, that's where we are
2019:10:03 00:02:56 alexmiller I have a burning desire to move these libs out of alpha and stabilize
2019:10:03 00:03:37 noprompt Speaking from my experience with the greek letter style namespaces, its been incredibly liberating to not have to worry about breaking users.
2019:10:03 00:04:05 alexmiller and as fair warning, core.specs.alpha is eventually going to get updated to depend on spec-alpha2 (or some future non-alpha variant) and will necessarily break again. But on the upside, these will be new names, not breaking versions of old names.
2019:10:03 15:49:19 noprompt Is there an appropriate place to discuss this or get an explanation? It doesn’t seem like a necessary requirement for spec alpha to depend on any other version of spec.
2019:10:03 15:53:28 noprompt Put another way: is there a path forward that doesn’t cause breakage?
Breakage should be taken seriously and avoided at all costs. It takes real time and effort on behalf of users to deal with it.
2019:10:03 16:07:17 alexmiller We have had some discussion about it but are deferring until later the full analysis of what we will do for migration. Most likely, there will be a new lib (probably core.specs) with new namespaces and new (spec 2) specs. There are a number of possible paths to continuing to support spec 1 specs (via some adapters possibly) and many choices about to what extent they are supported. Exploring this in full depth is more than I'm going to do in this slack thread.
2019:10:03 16:08:59 alexmiller You're preaching to the minister here on breakage, but as an alpha library the plan was always to ultimately be replaced and we've been pretty vocal about that from the start.
2019:10:03 16:16:03 noprompt Where does the full depth discussion take place?
2019:10:03 16:52:25 alexmiller N months from now, Rich and I will go through a design process to try to flush out all the questions and answers around this
2019:10:03 16:52:41 alexmiller how and what kind of feedback we'll need at that point I'll defer to that point
2019:10:03 16:53:45 alexmiller what the world will look like then, I have no idea so I'm not going to worry about it now
2019:10:03 00:04:27 noprompt Thats the win I’ve experienced.
2019:10:03 00:05:02 seancorfield clojure.spec.alpha -> clojure.spec-alpha2 has some major breakage for users who choose to migrate. Most mainstream stuff is unchanged but there's a lot of "dragons" if you've ventured off that path 🙂
2019:10:03 00:05:28 noprompt Its been incredibly liberating for me to be able to make new versions of my library without worrying about breaking users or, worse, stagnating.
2019:10:03 00:09:07 noprompt Maybe I took what Rich said to the extreme but, in practice, what I’ve come to discover is that strive for a “stable version” is just a withdrawal symptom of breaking away from a bad versioning habit and way of thinking about a project.
2019:10:03 00:10:20 noprompt Like, it no longer bothers me that the next version of Meander, when I get to it, will live at meander/zeta. I don’t care that I can’t (:require [meander.core :as m]). (:require [meander.zeta :as m]) is fine.
2019:10:03 00:11:25 noprompt The whole idea of a “stable version” is odd to me when we all know the only thing we can rely on as software developers is that things are rarely, if ever, stable.
2019:10:03 00:14:56 noprompt So, I would hope, that spec doesn’t devolve to SemVer at some point.
2019:10:03 00:15:37 noprompt I think it would be a step back to one day to have [clojure/spec X.X.X].
2019:10:03 00:16:01 noprompt That just puts you back in the soup.
2019:10:03 00:16:23 alexmiller well, you will have that
2019:10:03 00:16:46 alexmiller but at the point we go there, we will try to maintain only additive changes after that point
2019:10:03 00:17:08 noprompt Right but the point is by not doing that you don’t have to do what you just said.
2019:10:03 00:17:11 noprompt You don’t lock yourself in.
2019:10:03 00:17:28 noprompt Once you go back to SemVer, you’re locked in.
2019:10:03 00:17:36 alexmiller it's not sem ver
2019:10:03 00:17:43 noprompt You have to have the discipline to do what you just said.
2019:10:03 00:17:52 alexmiller it's an increasing set of numbers that make only additions
2019:10:03 00:18:07 alexmiller I believe we have that discipline :)
2019:10:03 00:18:21 alexmiller and will submit the rest of clojure as evidence :)
2019:10:03 00:18:29 noprompt But why put yourself in a situation where you necessarily must have discipline.
2019:10:03 00:18:31 noprompt 🙂
2019:10:03 00:18:55 alexmiller because supporting an ever changing set of namespaces across versions is irritating?
2019:10:03 00:19:11 alexmiller there's a balance of tensions here
2019:10:03 00:19:20 noprompt How would that be any different from just depending on a fork?
2019:10:03 00:19:44 noprompt I can’t speak for others, but I don’t see that as an irritant.
2019:10:03 00:20:39 noprompt The question is “how do you know you’re at a stable point where its possible to only sustain the software on discipline you can be sure you have?”
2019:10:03 00:20:51 alexmiller in general, I would rather just bump versions than bump versions AND update the namespaces in all my code
2019:10:03 00:22:21 noprompt In principal you can do that already with any artifact that uses SemVer in honestly and never has a major version bump.
2019:10:03 00:22:46 alexmiller I'm going to just say experience. but changing to new namespaces or even new lib names is not an option that's lost - we can always do that if needed
2019:10:03 00:23:07 alexmiller I'm suggesting a small thing for users while still retaining the option to do the big thing for users
2019:10:03 00:23:19 alexmiller that seems better regardless
2019:10:03 00:24:34 noprompt Well, Alex, I’m not going anywhere anytime soon as far as I can tell so I’m looking forward to seeing where this goes. 🙂
2019:10:03 00:24:45 alexmiller cool
2019:10:03 00:25:03 noprompt Though, I don’t mind a little typing. 😉
2019:10:03 00:25:46 noprompt I gotta go be a dad but, again, thanks @alexmiller for making that change to the README.
2019:10:03 00:31:45 alexmiller ditto on dad time
2019:10:03 08:53:35 roklenarcic Current spec alpha has a function s/merge which works like s/and in that it will only conform the last spec. I am wondering if spec-tools coercion suffers from same problem?
2019:10:03 09:26:36 ikitommi @roklenarcic there is spec-tools.core/merge
2019:10:03 09:29:19 ikitommi > .. that selects only the specced keys from each conformed result, then merges those results onto the original input. This avoids overwriting conformed values with unconformed values while preserving all unspecced keys of the input. Fixes #90. By Arttu Kaipiainen.
2019:10:03 09:39:25 roklenarcic Cool thanks
2019:10:03 09:39:43 roklenarcic does it work this way with s/conform or only with coerce?
2019:10:03 09:40:24 ikitommi not sure, please test 🙂
2019:10:05 20:44:37 roklenarcic Hm, ended up not using coerce, because it doesn't use any conforms defined in specs.
2019:10:03 11:56:21 practicalli @alexmiller just like to say thanks for all the work you and the team have done and continue to do with Spec. It is much appreciated. Thank you.
2019:10:03 16:08:34 jonathan Hi all. I'm trying out spec alpha2. I noticed I accidentally forgot to include a key in my schema, but s/select does not seem to validate that the key exists in the schema.
(s/def ::first string?)
(s/def ::last string?)
(s/def ::user (s/schema [::first ::last]))
; Should the select blow up? email is not in spec
(s/def ::create-user (s/select ::user [::last ::email]))
2019:10:03 16:09:15 ghadi it doesn't care in spec1 either
2019:10:03 16:09:41 alexmiller s/schema and s/select both support open maps (extra keys are ok), just like s/keys
2019:10:03 16:10:41 jonathan I thought (s/select ::user [::last ::email])) is specifically saying I'm choosing to require ::email from the ::user schema
2019:10:03 16:10:43 alexmiller whether you should get some feedback if s/select requires things not in the schema is a reasonable question, and not something we've talked about
2019:10:03 16:10:57 jonathan :+1:
2019:10:03 16:12:51 jonathan Without any validation it seems like I don't need the schema at all except for documentation purposes
2019:10:03 16:48:06 alexmiller several things - 1) supports gen in both schema and select, 2) basis for closed spec checking if you want to do that, 3) yes, docs
2019:10:03 18:40:09 jonathan Thanks. I appreciate your work on spec!
2019:10:03 16:23:40 seancorfield One question that came to my mind yesterday as I worked through getting our entire code base switched to the latest Spec 2 and our full test suite passing: how will folks deal with the world when they want to use Spec 2 but they rely on a chain of dependencies where one of the downstream libs still depends on Spec 1. https://github.com/cognitect-labs/anomalies/blob/master/src/cognitect/anomalies.cljc is a good example: very small, focused lib that defines Spec 1 specs and could be depended on by any number of libs in your tool chain.
2019:10:03 16:25:06 seancorfield To get our code base passing all its tests, I had to essentially recreate Anomalies' specs in my code, I had to create Spec 2 compatible versions of expectations.clojure.test (one I maintain), and worldsingles/web-specs (work maintains), since those are downstream dependencies using Spec 1.
2019:10:03 16:26:04 seancorfield @alexmiller If Spec 2 becomes clojure.spec (no alpha), how will folks deal with a mixed Spec 1 (alpha) / Spec 2 (contrib) code base?
2019:10:03 16:27:50 seancorfield In particular, in our case, we create specs in our own code (Spec 1 on master, Spec 2 on a branch) that incorporate the specs from Anomalies -- which I suspect is going to be a fairly common situation that folks get themselves into...
2019:10:03 16:49:10 alexmiller I think it's possible to build an adapter layer from spec 1 to spec 2, but we're going to push off answering these questions till we have something good enough to migrate to
2019:10:03 16:54:41 seancorfield Fair enough 🙂 Is there a specific outline of work left to do on Spec 2 at this point? Or is even that aspect not "baked" yet?
2019:10:03 17:46:28 alexmiller there's a rough list at the end of my clojutre talk
2019:10:03 17:56:53 taylor v nice talk
2019:10:03 17:47:13 alexmiller
* More on map forms
* Function specs, ret/fn specs, defn
* Metadata / doc strings
* Open enumerated sets
* Spec traversal
2019:10:03 17:48:21 alexmiller that doesn't all necessarily have to be done to be at a release point if we feel it's additive from that point
2019:10:03 17:49:17 alexmiller so the actual list may be shorter (or longer) but certainly nailing down 1,2, and probably 4
2019:10:03 17:49:35 seancorfield Ah, and I just watched that the other day and forgot about that list. Sorry.
2019:10:03 17:50:07 seancorfield What do you mean by "Open enumerated sets" in that context?
2019:10:03 17:50:54 seancorfield And "More on map forms" means the unqualified, inline s/schema definitions?
2019:10:03 17:55:41 alexmiller re open enumerated sets - using a set as a predicate #{:red :green :blue} is a different form of "closed" predicate. Often you want to say "these constants", but also other things later
2019:10:03 17:56:03 alexmiller "more on map forms" = more work on the "map" form of specs
2019:10:03 17:56:52 alexmiller as the maps currently in there are a first pass and we will likely change the form of some of them and probably automate the form<->map definition somehow
2019:10:03 17:58:46 seancorfield Ah, the "AST" stuff... OK.
2019:10:03 17:59:34 seancorfield Re: enumerated sets. That sounds interesting. I'm not sure I can come up with a use case right away tho', but I can see how set literals are currently "closed".
2019:10:03 18:04:33 alexmiller options are a common case
2019:10:03 21:02:31 kenny Anyone know of a library that lets you strip extra keys of a multi-spec/parameterized map? For example, if a map with :type set to :a requires keys :a1 and :a2, any additional keys would be removed.
2019:10:05 11:57:55 dmarjenburgh You can use select-keys, but the problem is that you define the key set in two places. I had a similar problem and defined the schema as data {:req [::a1 ::a2] :opt [::a3]} . From there, I use the data to generate the spec and the keyset.
2019:10:05 21:54:04 kenny It needs to be a bit smarter than that because multi-spec is used. I’d only like to select the keys that the multi-spec for a particular dispatch defines.
2019:10:05 21:54:29 kenny Ideally it’d also be recursive.
2019:10:03 21:33:20 hmaurer Hello 🙂 Possibly a silly question but what is the recommended way of working with specs whose data they model I want to serialise and deserialise to JSON with unnamespaced keys?
2019:10:03 21:34:04 hmaurer so, in clojure the keys should be namespaced (since, as far as I understand, that’s the only way to work with spec), but in JSON they should not, and when deserialising JSON into a clojure datastructure I want to automatically namespace the keys based on some spec
2019:10:03 21:34:18 hmaurer I thought s/conform would do this if I specified :req-un, but it doesn’t appear to
2019:10:03 22:09:54 seancorfield :req-un and :opt-un are for unqualified key names -- but let you associate qualifiers for the matching spec names.
2019:10:03 22:11:30 seancorfield
(s/def :foo/bar string?)
(s/def :foo/map (s/keys :req-un [:foo/bar]))
(s/def :quux/bar int?)
(s/def :quux/map (s/keys :req-un [:quux/bar]))
;; so you have have {:bar "s"} in one context and {:bar 42} in a different context
Does that help @hmaurer?
2019:10:03 22:18:43 hmaurer @seancorfield thanks! I think it does; initially I was thinking I would want some sort of way to deserialise {"bar": 42} (json) into something like {:foo/bar 42} in a spec-driven way, but I could just use req-un and have unqualified keywords everywhere
2019:10:03 22:29:36 seancorfield Yeah, I'm not sure what to recommend for switching between "JSON" style maps and "internal" (qualified) maps. I think a lot of that is context-dependent / application-dependent so I'm not sure there's a generic approach that suits everyone.
2019:10:03 22:31:04 seancorfield Given the specs, you can always get the form of the spec and get the list(s) of keywords used in the spec and you could create a generic way to call clojure.set/rename-keys or simply process the (JSON) map to qualify the keys yourself. Much will depend on exactly what your application needs and why you need to perform that transformation.
2019:10:03 22:40:08 hmaurer Alright; I'll look into that. Thank you
2019:10:05 21:28:55 seancorfield @roklenarcic Interesting. I tried it with the latest Spec 2 and it's the same. Also the same behavior using s/schema (with [::a] for qualified keys and {:a ::a} for unqualified keys).
2019:10:05 21:38:44 ikitommi I think the reason is that clojure.spec.alpha/merge separately conforms each merged spec, then merges the results, so the unconformed value from the lateer overrides the result
2019:10:05 21:38:52 alexmiller Merge only confirms the last spec
2019:10:05 21:39:21 ikitommi Second works because all namespaced are always confirmed, even if they are part of the spec.
2019:10:05 21:39:33 alexmiller ^^
2019:10:05 21:50:53 seancorfield Ah yeah. That makes sense!
2019:10:05 22:51:21 roklenarcic How? Makes no sense at all to me that it matters whether I use namespaced keys or not.
2019:10:05 22:53:06 roklenarcic Alex says that merge only conforms the last spec, but if I s/merge multiple s/keys specs that use conformers and that use namespaced keys, all the keys are conformed not just the ones from the last merge argument
2019:10:06 00:46:40 seancorfield @roklenarcic If you have a spec :foo/bar and you conform any map that contains :foo/bar, it will get checked
2019:10:06 00:49:41 seancorfield So the s/merge with the unnamespaced key doesn't get conformed because s/merge only conforms the last spec -- ::b -- but the qualified keys all get conformed because regardless of whether they're even in the spec.
2019:10:06 08:16:29 roklenarcic wait, so namespaced keys in maps get checks even if they are not in the spec I'm using to check them with?
2019:10:06 08:46:00 mpenet Yes
2019:10:06 08:46:37 mpenet You can confirm this with (s/keys)
2019:10:07 11:03:42 sgerguri How can I force spec to not validate a namespaced key against an optional spec in a map? I have a multispec with two clauses where the only difference is in the inclusion of a namespaced key, but because the key matches a spec the library will simply go ahead and validate that key against the spec even though it should not for that specific clause.
2019:10:07 12:44:22 alexmiller keys will always validate namespaced keys
2019:10:07 12:45:32 alexmiller That’s kind of central to the whole design of spec
2019:10:07 13:33:47 sgerguri It just feels inconsistent to how un-namespaced maps are treated - though I guess one could technically argue that if I'm saying a namespaced key should have a particular shape then it should have it whether or not it is required in a map. I have since tweaked it so that this is indeed the case, though given how namespacing is a particularly burdensome thing in our service (I have stripped it from everywhere except for one namespace, and even there it feels a bit redundant) this was quite the surprise in behaviour.
2019:10:07 13:37:54 mpenet if you think about them (kinda sorta) like Types it makes sense. a :animal/lion is always the same thing.
2019:10:07 13:38:54 favila Rich has a philosophical position that maps should be a bag of keys, and type information comes primarily from keys not from the bag
2019:10:07 14:13:46 dominicm I think is inaccurate to characterise it as philosophical. Having consistent meaning associated with names in a program makes your program easier to reason about.
2019:10:07 16:37:29 sgerguri I found that in our specific use case, where we have lots of things that look the same up to a certain level and can diverge in some specifics, namespacing became a burden both during testing and during main program flow. Our use case might just be a special scenario, though I suspect I would probably do it the same way again if I was facing largely similar scenario - i.e., internal entities with roughly similar shapes, as I really don't like the idea of mixing different namespaces in a single map.
2019:10:07 16:48:55 dominicm I hate working on programs where the thing I'm looking at shares a name with something else but is different. E.g. if "products" in one place referred to the ids of products you're allowed to see, but elsewhere referred to a list of product entities from the database.
2019:10:07 16:50:58 dominicm Mixed namespaces don't create any bother for me, I believe that exists so you can have separate pieces of code which operates on the same entity as long as it fulfils the data contract. E.g. It's easier to understand code which operates on "Twitter account" which is a limited number of details, rather than "user" which is much larger and requires more juggling in your head.
2019:10:07 17:19:04 sgerguri Yeah - and I'd imagine that would work well if the maps truly mixed contexts. In our case, they mostly mix different datatypes that tend to look similar to one another, with a common "envelope" map around them.
Given the benefits we gain out of being able to shove everything into a common processing pathway stripping the namespacing ended up making things a lot less unwieldy than it was.
2019:10:07 17:30:46 dominicm Sounds like stamp coupling
2019:10:07 17:31:57 dominicm When taken to an extreme, these ideas are still coupling. Functional connections are the only kind that's free from this, but that's a lot of typing and not much abstraction.
2019:10:07 18:10:10 favila I don’t disagree it’s nice when keys = types and they are well namespaced and shared across entire application. I think Dominic took “philosophical” as a slur which I absolutely do not mean
2019:10:07 21:34:13 dominicm I didn't think you were necessarily disparaging it, but I hear the term "philosophical" a lot when people feel a stance is without pragmatics and more rooted in perfectionism at cost to real software. I just wanted to attempt to explain the importance of this stance.
2019:10:07 21:34:47 favila fair enough
2019:10:07 21:35:01 favila yeah, I think it is slightly impractical, but aspirational
2019:10:07 21:36:41 favila the impracticality for me is entirely centered around those two use cases: typing someone else’s data which doesn’t share that philosophy (which, honestly, is the vast majority of systems! most think property meanings should come from the record type they are in), or cases where the key is used “as a value” in e.g. a DSL instead of its “natural” meaning.
2019:10:07 21:38:26 seancorfield For me, that's why there is the split between unqualified and qualified keys: the former are "out there" and you have to assign meaning on a per-case basis; the latter should be unique keys within your domain and should have a single meaning for each name (in terms of the data type).
2019:10:07 21:38:56 dominicm If you're a piece in a pipeline and you need to keep the data fairly untainted, I can appreciate that difficulty.
2019:10:07 21:41:52 seancorfield (and as an adjunct to my comment: if you can't assign a single meaning for a name "globally" then don't use a qualified keyword for it -- stick with unqualified keys that you have more flexibility with)
2019:10:07 18:10:38 favila However, keys in fact do not always have the same type, e.g. a map key in a DSL vs a map key in data
2019:10:07 18:11:00 favila e.g. datomic pull expression map vs the value of the pull
2019:10:07 18:11:30 favila the “bag” sometimes imparts meaning to the keys, IOW
2019:10:07 18:12:13 favila not to mention having to deal with other systems which don’t share your “maps are only keysets” philosophy, but I think the spec2 select machinery is an acknowledgement of that problem
2019:10:07 18:13:27 alexmiller spec 2 select just says that the notion of required/optional keys is contextual
2019:10:07 18:13:40 alexmiller it still maintains strong semantics for attributes across aggregates
2019:10:07 18:13:46 favila but it also has contextual key renaming, no?
2019:10:07 18:13:56 alexmiller no, it doesn't
2019:10:07 18:14:43 alexmiller it has the ability to do binding of un-qualified keys to specs, but un-qualified keys only
2019:10:07 18:18:42 favila that’s all I meant
2019:10:07 18:19:07 favila {:first :my.app.name-part/first-name} for example
2019:10:07 18:20:24 favila I think schema is what I am talking about, as being able to impart some meaning to the aggregate
2019:10:07 18:20:30 favila instead of being “merely” a keyset
2019:10:07 18:21:22 alexmiller the intent here is not that attributes can be assigned many different meanings, but that unqualified keys are missing the link to attribute semantics
2019:10:07 18:21:53 alexmiller it's still just a collection of attributes though
2019:10:07 18:27:52 favila yes it does not solve that problem
2019:10:08 08:42:33 misha yeah, I felt tension speccing pull-patterns vs pulled data too, but I think pull-pattern should not be specced with s/keys, which kinda resolves the situation
2019:10:08 09:29:18 sgerguri In my case, we only do two layers of transformation before passing the data out. Input is big, mangled and flaky. Given all that it felt the domain - if there is one indeed - was rather thin, and thus there were few benefits to be gained in namespacing everything. But then we're using it for data engineering over Kafka, so we're basically talking about transit transformations, rather than domain manipulations.
2019:10:08 09:36:29 ikitommi > spec 2 select just says that the notion of required/optional keys is contextual
... but there is no explicit syntax of marking some keys as optional in select? Rather implicit "If the key if not part of the select, it's optional", right?
2019:10:08 10:07:30 valerauko there's no equivalent for the opt declaration in spec 1? "if present, it has to be this shape" thing
2019:10:08 12:08:40 alexmiller Schemas in spec 2 are all optional
2019:10:08 12:10:17 alexmiller Select specifies what’s required in a particular context
2019:10:08 21:25:22 rapskalian I have a question that I think is related to the conversation above. If I have the following datomic-style schema:
How would one spec :user/email?
The difficulty stems from the fact that I can transact {:user/email "myemail"} or {:user/email ["myemail"]}, but when I query the db, I would always receive {:user/email ["myemail"]}.
Perhaps it’s recommended to define contextual specs:
(s/def :email/address ::non-empty-string)
(s/def :user/email (s/coll-of :email/address)) ;; matches the schema in name and shape
(s/def :user.input/email (s/or :one :email/address :many :user/email)) ;; matches what the boundaries of the system would accept
I’m especially interested in the use-case of integrating specs with datomic’s :db/ensure feature in order to validate entities before they enter the db.
2019:10:08 21:26:56 alexmiller with specs it's always best to try to "state the truth" so I'd go with something that took either
2019:10:09 08:08:26 dominicm I think the truth is that these are semantically different uses of the same key. One is a DSL, and one is data.
Requiring all my functions to handle a collection or singular because spec doesn't allow me to distinguish the contexts makes pointless additional work for me.
You could move the spec to the aggregate, applying additional constraints. I'm not sure how well that would work on a conventional spec 2 system.
2019:10:08 21:27:21 alexmiller for the last question, might be best to ask that in #datomic, I'm not well-versed enough to have a good answer
2019:10:08 21:28:19 rapskalian That makes sense. The truth here being that, indeed, the input is a different thing than what comes out of the database?
2019:10:08 21:29:12 rapskalian I suppose I could restate the question as: if I want my specs to “follow” the schema, should I spec what comes out, or what goes in?
2019:10:08 21:29:38 rapskalian By “follow” I mean “possess the same semantics”
2019:10:08 21:34:23 rapskalian I think I may have answered my own question…I’m realizing that spec’ing input is much more useful than having a truthful “spec+schema” (the latter of which is likely redundant)
2019:10:08 21:35:33 rapskalian In other words, spec’ing what comes out of the database is pointless…at that point it’s too late. Spec efforts should be focused at the input boundaries of the system.
2019:10:08 21:36:59 rapskalian Interestingly tho this makes spec and the :db/ensure feature of datomic feel a bit incompatible, because predicate functions given to :db/ensure have the signature (defn my-pred [db eid])…and so they can only validate what comes out
2019:10:08 22:19:07 favila I think it’s better to think of the transaction as a DSL. You can spec a dsl’s grammar, but not its meaning
2019:10:08 22:20:02 favila so you can’t spec :user/email specifically as input to d/transact, only as an opaque key in a transaction map
2019:10:08 22:22:56 favila you should either spec a level higher, e.g. some function that takes well-formed input and produces a set of transaction assertions/retractions/maps that do what you want; or a level lower with data-aware assertions, e.g. perhaps the DSL produces an ast which is easier to analyze, or in the case of datomic :db/ensure to guarantee your invariants.
2019:10:08 22:23:23 favila it’s a little less obvious how to leverage spec in the “level lower” cases
2019:10:08 22:24:29 favila combining spec with datomic, I’ve had more luck specing what you would want to see out of d/pull, because that’s what’s flowing through the functions in an application
2019:10:08 22:24:43 favila that feels like where spec shines the most
2019:10:08 22:25:17 favila and I’ve just accepted there’s no silver bullet for the cases where the data approaches the edges, e.g. to serialize/deserialize or to transact to the db
2019:10:09 00:54:42 rapskalian Thanks @favila that’s very helpful
>combining spec with datomic, I’ve had more luck specing what you would want to see out of d/pull, because that’s what’s flowing through the functions in an application
I’m reaching this conclusion as well. It actually works very well for spec’ing at that “lower level” you mentioned, because :db/ensure can easily make use of pull results.
2019:10:09 18:05:10 rapskalian Thinking some more on this. I gave up on trying to use spec “near” the database. The schema is sufficient for specifying data at rest, coupled with some basic structural constraints (e.g. required keys, or in my case, ensuring no cycles are formed). Structural meaning only the relationships are constrained, not their actual values. I think this is what you mean by opaque @favila.
With that in mind, spec seems to really flex its muscle upon data in motion; the transitions between valid data-at-rest so to speak…
Thinking out loud 🙂
2019:10:09 18:06:54 favila Perhaps. I was thinking of it more in terms of speccing the grammar vs analysing the result
2019:10:09 18:07:52 favila for a tx, [{:my-map myval}] the spec is really not anything about :my-map, but a vector of things, some of which may be maps, the keys of which are arguments to some transformation
2019:10:09 18:08:47 favila so the key is “opaque” in that sense, it’s not specced as itself; map values are independent (gramatically) from the keys
2019:10:09 18:09:32 favila think about what this would conform to: it would produce some kind of normalized ast with {:attribute-key :my-attr :attribute-scalar-value myval} for example
2019:10:09 18:10:34 favila it’s hard to spec “both” at once, i.e. both the generic grammar and also some specific constraints on terms in it (constraints which would be enforced by an analyzer stage in a typical compiler)
2019:10:09 18:19:24 rapskalian I see. So to apply your compiler analogy to an “application”, you’re referring to the fundamental difference between, say, the description of some mutation (as data), and the new database that results (assuming db-as-a-value semantics). The keys in both could also be said to be grammatically separate (?)
2019:10:09 18:23:53 favila yeah
2019:10:09 18:24:34 favila your key inside the transaction mini-language means something different than your key in a map with your data in it
2019:10:09 18:25:22 favila in the first, it’s a parameter to some invisible ast’s map key, in the later it’s the data itself
2019:10:09 18:25:42 favila maybe map-as-syntax vs map-as-data
2019:10:09 18:26:04 favila you can’t s/keys the keys inside a map-as-syntax, only s/map-of them
2019:10:09 18:30:35 rapskalian That’s where my mapping of these concepts to spec was failing. s/keys seems to imply that a key must mean the same thing everywhere it’s used, but there’s an assumption that you’ll be checking against map-as-data. It is inappropriate for map-as-snytax. s/map-of was the 💡 for me, the data there is purely syntax.
2019:10:09 18:31:30 rapskalian For some reason I seem to have this desire to write “the base specs” of a system, but specs really don’t make sense without a context…
2019:10:09 18:36:11 favila yeah, for me that hurts most when the context is the type of record it’s in
2019:10:09 18:37:11 favila I keep running into cases where a key has a wider domain, but can be narrowed/refined contextually by the type of its record.
2019:10:09 18:37:34 favila this is a common feature of type systems inspired by class based inheritence or xsds
2019:10:09 19:04:20 rapskalian Interesting, I haven’t run into that yet. How have you been handling it? Seems you would need some ability to compose spec registries into hierarchies.
2019:10:09 19:07:34 rapskalian Or perhaps express that inheritance in the key itself, eg :slack.user/email
Is this expected behavior? From reading the docs on select I expected it to select from the schema and provide a more restrictive spec. Is this the wrong way of thinking about it?
2019:10:11 21:02:08 seancorfield @mattparker I think it's a known issue (I think I remember Alex mentioning this).
2019:10:11 21:02:43 mattparker ah ok. Thanks!
2019:10:11 21:02:53 seancorfield It would probably be nice for the above to throw an exception (that :first-name isn't present in the schema).
2019:10:11 21:26:21 alexmiller schemas and selects are both open, so this is the expected behavior
2019:10:11 21:27:16 alexmiller if doing closed spec checking, it would make sense for that to error (I'm not at a repl but I suspect it doesn't right now due to some lingering work there)
2019:10:11 21:27:31 seancorfield Ah, that might be what I'm thinking of...
2019:10:11 21:27:34 alexmiller select is about requirements
2019:10:11 21:27:58 alexmiller you must supply ... , not you can only supply
2019:10:11 21:28:53 seancorfield (so select can force key presence for things that are not aligned with specs for their values)
2019:10:11 21:29:25 mattparker wouldn't that easily be achieved by expanding the schema with union?
2019:10:11 21:30:29 mattparker that would make the first arg what's possible and then the second arg the restrictions
2019:10:11 21:30:42 alexmiller that's not what select is
2019:10:11 21:31:07 alexmiller select tells you what is required in a context
2019:10:11 21:31:37 alexmiller "restrictions" are about negation / closed world
2019:10:11 21:32:41 mattparker what got me thinking that was in the wiki s/select is a spec op that uses a schema to define the world of possible keys
2019:10:11 21:33:00 mattparker in this case :first-name is in the world of possible
2019:10:11 21:33:29 alexmiller well, there's a separate wrinkle here of qualified vs unqualified keys
2019:10:11 21:33:37 mattparker but I guess what this means is schema is a subset of what's possible
2019:10:11 21:34:03 alexmiller probably this is mostly that I could be better at writing words :)
2019:10:11 21:34:47 mattparker it's a wiki for an alpha. I definitely don't take it as a contract haha
2019:10:11 21:34:51 alexmiller anything on that wiki is me writing, and not vetted by Rich
2019:10:11 21:37:17 mattparker thanks for helping me understand!
2019:10:11 22:18:30 Steven Zhou A question around clojure.spec.test.check/check for its num-tests opt.
I’m trying to check how many times a snippet has run based on the print values as below.
(s/fdef foo :args (s/cat :s string?) :ret string?)
(defn foo [s] (prn "s") s)
(s/fdef bar
:args (s/cat :x string?)
:ret (s/fspec :args (s/cat :x string?)
:ret string?))
(defn bar [x]
(prn "x")
(fn [y] (prn "y") y))
(alias 'stc 'clojure.spec.test.check)
(st/check `foo {::stc/opts {:num-tests 1}})
(st/check `bar {::stc/opts {:num-tests 1}})
s => printed 1 time
x => printed 1 time
y => printed 21 times
Could anyone give any insights here that why y has been printed out 21 times? And is there a way to control the number of tests run for a s/fspec block?
2019:10:11 22:25:35 gfredericks fspecs are validated using their own test.check run
2019:10:11 22:25:49 gfredericks I doubt it's configurable
2019:10:11 22:29:58 Steven Zhou Thanks for the quick response! That makes sense, will you able to point where that's configured in the source code? It will be really nice if I could configure that, since one of my fspec is running super slow and I don't want it to eat up all my resources.
2019:10:11 23:21:37 wilkerlucio @alexmiller hello, I just watched your video from the clojuretree. nice talk! I noticed one thing about the new schema / select thing, in the current spec1, its valid to create anonymous selections, like calling (s/keys) directly as a parameter or a return value, this possibility seems to be vanished with the requirement to name the things with s/schema before doing the structural selection with s/select, did I got that right? if not, is there gonna be a way to define nested selections without having to name it?
2019:10:11 23:23:05 alexmiller There’s no requirement to name either
2019:10:11 23:24:52 wilkerlucio ok, so the idea is to use like: (s/select [] [{::my/selection [::deep-value]}]), like this?
2019:10:11 23:25:13 alexmiller well, vector there, but yes
2019:10:11 23:25:26 wilkerlucio fixed 🙂
2019:10:11 23:25:45 alexmiller yep
2019:10:11 23:25:51 wilkerlucio cool, thanks for the clarification
2019:10:11 23:26:11 alexmiller schemas can be put in the registry to reuse them, which we think is useful, but not required
2019:10:11 23:26:28 alexmiller selects are probably less useful to put in the registry
2019:10:11 23:26:55 wilkerlucio yeah, what makes it important to me, is that more and more I'm going in a modeling direction that I don't name groups of things most of the time
2019:10:11 23:27:13 wilkerlucio instead, just say what "shape" a fn needs directly, so I can get very specific every time
2019:10:11 23:27:54 wilkerlucio do you think a select with arity 1 makies sense? so it would use empty keyset
2019:10:11 23:50:46 alexmiller Dunno
2019:10:11 23:52:29 kenny I have a map with a ::type key and a ::tags key. ::tags is a map. I want the keys in the ::tags map to depend on the ::type of the "top-level" map. Is this possible to spec? e.g., Where the ::tags map is spec'ed as (s/keys :req [::a]) for ::type ::a.
{::type ::a
::tags {::a 1}}
2019:10:11 23:56:21 kenny It seems like the options are either:
1) restructure the map so the type and tags are at the same level (this won't work because there needs to be a way to get the tags map easily)
2) Copy the ::type key into the ::tags map & use multi-spec (this isn't great because the data is redundant)
3) Create a s/and predicate on the top-level map that checks the ::tags using multi-spec after assoc'ing the ::type onto the ::tags map. (this feels icky but may be the best way)
4) Unqualify ::tags (I don't really like this because every other key in the map is qualified. The key :tags may be used all over the place so finding usages of the key becomes harder therefore making refactoring harder.)
2019:10:12 03:15:59 alexmiller well I would prioritize how you're going to use it as data over how to spec it - that's worth 10x more thought than how to spec it
2019:10:12 03:16:41 alexmiller and multi-spec does not require that you use a top-level key - the dispatch function can be anything
2019:10:12 15:23:35 kenny There’s not way to change the spec for the nested ::tags map with multi-spec though.
2019:10:12 15:24:07 kenny The data definitely belongs in a nested map like above.
2019:10:12 21:16:55 tianshu How about the current status about spec2? Will it replace the spec1?
2019:10:12 22:15:24 seancorfield @doglooksgood That is my understanding, yes. Spec 2 will probably just become clojure.spec once it moves out of alpha and Spec 1 will be left as-is.
2019:10:12 22:34:42 alexmiller They may coexist for a while
2019:10:12 22:49:00 seancorfield Spec 1 will "never go away" completely because a lot of code out there depends on it, so it depends on your definition of "coexist".
2019:10:12 22:49:21 seancorfield (Maven Central is forever 🙂 )
2019:10:12 23:27:55 alexmiller By coexist, I mean that core continues to support both in critical touch points (error handling, registry, doc)
2019:10:14 12:01:10 yuhan Is there a regex op which expresses "n to m repetitions of a spec"?
2019:10:14 12:51:17 jaihindhreddy I believe that cannot be done with a regular language.
You can do (s/coll-of ::s :min-length n :max-length m) though.
2019:10:14 14:21:23 yuhan hmm, I'm not so familiar with automata theory, but isn't the above string-based regex constructed using a regular language?
2019:10:14 14:22:02 yuhan Good to know that I'm not missing anything obvious built-in though
2019:10:14 15:49:32 andy.fingerhut Something like a{1,2} can be hand-expanded into what it means, which is a|aa. The expansions can get pretty long, of course.
2019:10:14 14:00:01 WhittlesJr Hi! I have some s/keys specs in which the keys' specs use s/or, and the map is only valid for certain combinations of s/or paths. I ended up writing a function, keys-conform?, which allows me to give a map of keywords to spec path, and check those spec paths against the conformed values:
This works, but as I've been using it, I'm finding I also want access to more subtle conditions for valid paths, like "or" and "not", and the logic is getting more complex.
So I'm wondering if there's already a built-in way to do what I'm doing, or if someone has written a library like this? I can't think of a good way to use multi-specs for this, but maybe someone else can?
(Also, I'll preempt any possible objections that ::foo and ::bar should be split up into different keys. In my use case, they really do have to be the same key with different possible paths.)
2019:10:14 20:04:10 seancorfield @alex.joseph.whitt Is there any sort of disciminant you could use to turn this into a multi-spec?
2019:10:14 20:06:31 WhittlesJr The problem seems to be that the branching at the level of ::specific-map is sort of multi-dimensional. So there's not necessarily one keyword that changes everything. The options are sort of mix-and-matchable.
Do you think there might still be a way?
2019:10:14 20:08:05 WhittlesJr The domain in question is parsing a rather nuanced binary protocol called BACnet. The packets have a number of valid shapes, but just as many invalid combinations of fields.
2019:10:14 20:08:40 WhittlesJr And the fields are obnoxiously polyvalent
2019:10:14 20:12:29 WhittlesJr Perhaps there could be a solution in nested multi-specs? Like if the flag in the header says this is a foo-packet, check foo-spec, and foo-spec has a number of other options under it based on the given values?
2019:10:14 20:22:54 WhittlesJr Well actually, it seems like multi-specs would just be replacing the use of s/or in ::specific-map... I don't think that would change the use of keys-conform?...
2019:10:14 20:44:33 misha @alex.joseph.whitt do you have specification document url?
2019:10:14 20:45:25 WhittlesJr Do you mean you want to see my real use case?
2019:10:14 20:45:29 misha yes
2019:10:14 20:47:57 misha but I think the issue here is your attribute tries to be too much different things. I'd try to model it with many attributes instead, e.g.
(s/def : int?)
(s/def :bac.str/foo string?)
,,,
2019:10:14 20:48:31 misha might save you some ifs down the road
2019:10:14 20:50:09 misha another question is: "why exactly do you want to spec that?"
2019:10:14 20:50:39 misha and "how many valid combinations there are vs. how many possible ones?"
2019:10:14 20:55:14 misha lol, it's 150$ for pdf
2019:10:14 20:58:43 WhittlesJr I can put a portion of my code up in a gist in a bit.
I have a few main reasons why I want to spec it:
1. To validate packets and my parsing logic
2. To have the ability to generate random but valid packets for automated test purposes (testing an embedded device that supports the BACnet protocol).
3. Generative testing for code that touches BACnet
How many valid combinations: Depends. For most parts that would use this pattern, probably less that 6 for each level of abstraction. The possible combinations are probably 2-3 times that number.
Heh, yeah it's a closed standard.
As for splitting the keys, that's a pattern I can and do utilize, but at some point it does boil down to one key representing multiple possibilities due to the nature of the fixed positions in the binary protocol. I'll keep noodling on it though.
2019:10:14 21:00:24 misha > BACnet currently defines 35 message types, or "services," that are divided into 5 classes.
sound like you can put em into multispec
2019:10:14 21:06:07 WhittlesJr I'm working at a different level than the message types at the moment. At the message type level, absolutely a multi-spec works (and that's what I use.) Here's my code for the generic header:
https://gist.github.com/WhittlesJr/4571fed1596400e242724972ee39b2d4
2019:10:14 21:06:35 misha so the general advice is to reduce combinatorial explosion, and spec only things you need to spec. how exactly – depends on that protocol, and what exactly you are doing with it.
2019:10:14 21:07:10 misha can you show raw header/message/whatever?
2019:10:14 21:10:52 WhittlesJr Here's an example capture of BACnet packets. Open in Wireshark and apply the bacnet filter. You'll probably also need to add UDP port 47824 as a BVLC decoding. (Analyze -> Decode as... -> +)
2019:10:14 21:11:37 WhittlesJr I used the s/or there for testing branching in ::attributes
2019:10:14 21:12:04 WhittlesJr Alternatively I suppose you could use a multi-spec, but seems to be 6 of one 1/2 dozen of the other
2019:10:14 21:21:12 misha well, looking at raw data did not help opieop
2019:10:14 21:22:05 WhittlesJr Merp. The BACnet spec is really dense and nuanced, takes a ton of contextualizing to grok the issues involved.
2019:10:14 21:25:38 misha this is what I dislike about it: context. spec works great when attributes are the same regardless of context. and it (still) seems to me, flattening out amount of valid attribute combinations with different attribute names would be simpler solution, than trying to replicate 1to1 all the branchyness in specs (don't forget to write custom generators hehe)
2019:10:14 21:27:39 misha looking at spec names (all ::/) namespaces of attributes don't mean much for you, so I'd try to encode types in those, to keep attribute name "polymorphic" (if you need for something)
2019:10:14 21:28:57 WhittlesJr Yeah, I have to wrestle with the high-context nature of the spec when writing the parsing logic. It's horrible.
I'll look into breaking it up, but something tells me that it won't be a fully satisfying solution. Part of the purpose of my code is to represent the packet fully, much like Wireshark does. It will make it less obvious what the actual binary fields are saying if I have three different optional keys rather than one key that can have three different types. But that's more of a human-level issue rather than a code-level issue.
2019:10:14 21:30:27 misha what are you parsing exactly? what is your input to spec?
2019:10:14 21:33:53 WhittlesJr If you want to look at the capture, check out packet 337 -> "Building Automation and Control Network APDU" -> "list of Values:" -> "{[4]". You should be able to recognize the header in there, which is the part of the spec that my gist is dealing with.
2019:10:14 21:34:38 WhittlesJr Everything under the "list of Values:" has one of those headers (or is a standalone "header" acting as an opening or closing tag)
2019:10:14 21:38:49 WhittlesJr Anyway, the approach I'm taking is working, I was just curious if there was already a solution out there or if I was coming at this from the wrong angle. Depending on how this goes, I may just make a library out of keys-conform? with some well-defined syntax. If my employer will let me open source anything, that is... :c
2019:10:14 21:40:20 misha does it have generator tho? :)
2019:10:14 21:43:45 WhittlesJr Yeah, actually I'm doing pretty well with generators, surprisingly. I'm finding that the coercion logic a la spec-coerce is synergizing really well with generators.
2019:10:14 21:44:31 WhittlesJr Especially as you climb up the abstraction chain and you get more and more fields that are interdependent, the coercion approach is helping to keep generation sane
2019:10:14 21:45:53 WhittlesJr But maybe I have yet to hit a real barrier and it's lurking somewhere in the thickets, waiting to spring an ambuscade on my productivity
2019:10:14 21:46:59 WhittlesJr ::attributes in the gist generates out of the box
2019:10:15 11:52:29 mpenet is there a way to express negation of other specs in a regex spec other than something like (s/+ #(not (some (fn [spec] (s/valid? spec %) [::foo ::bar])))
2019:10:15 11:52:55 mpenet I am sure I am missing the obvious
2019:10:15 12:23:02 alexmiller Not missing anything
2019:10:15 12:23:43 alexmiller Negation is fairly unusual in specs
2019:10:15 12:24:33 mpenet yeah it doesn't even work for my use case actually, I am better of with a simple predicate
2019:10:15 14:06:24 misha @mpenet is that pseudocode or actual code?
2019:10:15 14:13:18 mpenet pseudo
2019:10:15 14:13:52 mpenet I went with something else btw. I was specing a form similar to try/catch/finally
2019:10:15 14:14:10 mpenet getting the body accurately was the target
2019:10:16 17:22:56 rickmoynihan In spec 1 keywords are privileged with regards to s/keys. I’d love for spec 2 to let me define other types of values to name key specs… e.g.
(s/defkey "foo" integer?)(s/keys :req-key ["foo"])
I’m not actually so bothered by string keys, as what I’d really like to do is use URI’s as keys. My use case is that we work with RDF, and it’s really annoying having to convert between URI predicates and clojure keywords… I’d like to essentially do:
(def rdfs:label (URI. "http://,,,,#label"))(s/defkey rdfs:label (s/or :xsdstring string? :langstring langstring?)(s/def ::resource (s/keys :req-key [rdfs:label]))
Any ideas on whether this sort of thing might be possible in spec2?
2019:10:16 17:23:34 alexmiller no
2019:10:16 17:24:43 rickmoynihan implementing my own s/keys seems like it’ll be hard work
2019:10:16 17:25:07 rickmoynihan might there be an easy way to do it that can leverage stuff in spec?
2019:10:16 17:25:16 alexmiller the idea of keywords as names for specs is deeply embedded
2019:10:16 17:25:34 alexmiller not just s/keys but every spec impl relies on that
2019:10:16 17:25:36 rickmoynihan I wouldn’t mind having a layer of indirection between the name of the spec and the URI/key
2019:10:16 17:25:55 alexmiller we have no plans to make that
2019:10:16 17:28:40 rickmoynihan so would doing:
(s/def :rdfs/label (URI. ""http://,,,,#label"))(s/def ::resource (rdf/keys :req [:rdfs/label]))
not be feasible?
2019:10:16 17:29:00 alexmiller no
2019:10:16 17:30:55 alexmiller it is possible that we might allow arbitrary metadata for specs. if we did that (and I'm not committing to it yet), then you could hang your mapping to a uri on that.
2019:10:16 17:35:03 alexmiller in the meantime you could create such a mapping outside spec if desired
2019:10:17 07:17:38 vlaaad @alexmiller can you share rationale behind decision to limit spec names to keywords? (and symbols, right? for fdefs...)
2019:10:17 13:53:36 jaihindhreddy I believe, it's more like: spec allows us to spec data and functions. We spec functions by their name and pieces of data by their name (keywords), symbols are already used to name vars in the context of the language, which is why symbols can't be used. All other stuff (strings, characters, aggregates, etc) is not namespace qualified and hence those cannot be "precise" names. That causes collisions and is a recipe for brittleness IMHO.
2019:10:17 08:04:33 rickmoynihan @vlaaad: Though I’m asking whether it’s possible or suggestions on how to do it; I totally understand the reason to name specs with keywords. I suspect the main reason is that they support namespaces; and can therefore be unambiguous and named in a global context, but don’t require a dependency on being loaded like vars. Allowing the use of a key to be decoupled from its spec/definition, whilst also allowing the same key to have multiple specs. Like with RDF it means it’s open for extension, anyone can spec anything etc.
The reason I’d like to do it, is because when I convert RDF into a clojure structure, it’d be nice not to have too coerce URI’s (which are used to identify properties & classes in the same way spec uses keywords) into keywords. However it’s probably not so bad, maintaining that mapping. I was just hoping it’d be possible to write my own version of s/keys that used URI’s directly.
2019:10:17 14:45:31 mpenet it's super easy to make your own in the meantime. Just use atom as metadata registry + write few fns (alter-meta, doc, meta, etc). then it's possible to chain (-> (s/def ::foo string?) (stg/with-doc "bla bla bla")) since specs returns their key.
2019:10:17 18:48:30 eskemojoe007 I'm trying to create a spec around a time characterstic in a cljc file and getting confused. I generally try to use tick to manage time within cljs and clj. But their api doesn't have any tests for type of time that I'd like to put into the spec. What do others do for cljc spec that involve times?
2019:10:18 18:09:50 ag okay, but let’s say it returns nil and I want to print that out with expound, I guess I’m gonna have to run st/check one more time?
2019:10:18 18:10:05 taylor not sure about the expound part
2019:10:18 18:12:40 ag
(expound/explain-results (stest/check `foo))
would print out the reason in human readable form, right? But it doesn’t return anything, so no way to assert it.
and I can’t do something like:
(let [chk (stest/check `foo] ,,,
to re-use it for both
2019:10:18 18:13:13 ag for assertion and for human-readable output
2019:10:18 18:13:18 taylor I’ve never tried to use expound with test.check output
2019:10:18 18:13:45 taylor which part doesn’t return anything?
2019:10:18 18:14:29 taylor explain-results? you could use
(doto (stest/check `foo) expound/explain-results)
if so
2019:10:18 18:19:05 ag ah yeah, nifty doto. Cool, thanks!
2019:10:18 18:41:39 ag ah, the problem with doto approach though - stest/check throws on failure… damn
2019:10:18 18:52:18 taylor hmm really? maybe I’m thinking of something else but I thought it returned a map with useful failure info
2019:10:18 18:52:29 taylor or maybe that’s in the exception
2019:10:18 18:06:15 ag can someone show me an example of deftest where combination of stest/check used with expound
2019:10:18 18:16:19 bbrinck Haven’t tried it yet, but it’s on my list.
2019:10:18 18:21:02 ag Really awesome ideas, thank you guys!
2019:10:18 18:36:22 kszabo we use Kaocha’s feature, it works very well
2019:10:20 12:48:13 abdullahibra Hell everyone,
2019:10:20 13:08:25 abdullahibra can i pass the spec to function?
(defn validate [data spec]) ?, or this isn't right, because the spec is globally defined so i can use it inside the function directly ?
2019:10:20 15:07:41 rascio yes you can, it is what the default validate does.
specs are just values if you want to do a function that can work with any kind of spec you should do it.
2019:10:24 18:48:35 ag do we have any libraries that can generate specs based on DB schema, e.g.: postgress DDLs?
2019:10:24 20:27:10 ag has anyone dealt with type of a problem where you need a spec for a map with underscored keys (for data coming from DB) and absolutely same spec with the only difference being that keys are hyphenated? Maybe someone wrote a macro?
2019:10:24 20:37:00 ikitommi There is no way to close a Spec2 Select? e.g. dont’ allow any extra keys, right?
2019:10:24 20:43:29 ikitommi So, if i have s Schema with ::id, ::name, ::description and ::secret keys and I would like to make an api where you must send ::id & ::name, optionally ::description, but not ::secret, I would first have to make an (api) Schema with just ::id, ::name and ::description close that (so the ::secret becomes illegal) and make a select for that schema with just ::id and ::name defined so the ::description becomes optional. Right?
2019:10:24 20:55:51 seancorfield Spec 2 uses an option hash map to specify which specs are closed -- at checking time (`s/valid?` s/conform etc)
2019:10:24 20:56:09 seancorfield select is always open. It just says what is required.
2019:10:24 21:02:20 ikitommi Thanks, that’s my understanding too. So, to support a use case that one can’t send the ::secret, I need to create a concrete new Schema to be able to mark is as closed.
2019:10:24 21:03:39 seancorfield The checking code can determine which specs are treated as closed -- independent of schema / select.
(s/def ::id int?)
(s/def ::name string?)
(s/def ::description string?)
(s/def ::secret boolean?)
;; the internal schema
(s/def ::user (s/schema [::id ::name ::description ::secret]))
;; the api schema
(s/def ::user2 (s/schema [::id ::name ::description]))
;; the api select (id & name mandatory, description optional)
(s/def ::user3 (s/select ::user2 [::id ::name]))
;; closed validation
(s/valid? ::user3 {::id 1, ::name "kikka"} {:closed #{::user2}})
2019:10:24 21:28:04 vlaaad i wonder why closed specs are implemented as options instead of as another type of spec
2019:10:24 21:38:26 seancorfield @vlaaad Alex talks about that on the Inside Clojure blog as Spec 2 went down the "closed spec" path first and then changed to "closed checking".
2019:10:24 21:42:14 vlaaad (s/close-specs ::s) this is some mutability thing, I was talking about having closed specs as different specs, no mutability attached
2019:10:24 21:45:08 vlaaad I think that makes sense: being able to refer to your data specification by name, instead of by combining a name and an arguments to s/valid?
2019:10:24 21:47:49 seancorfield I'll leave it up to @alexmiller to respond. I prefer the path they've taken where "closedness" is just an option on checking.
2019:10:24 21:48:52 vlaaad If you have it as another spec it's still opt-in, but now it's also composable
2019:10:24 21:50:37 alexmiller Not going to do that
2019:10:24 21:51:46 alexmiller It’s not composable - negation is inherently problematic as specs evolve
2019:10:25 07:20:45 vlaaad I don't get it, can you explain? I thought about closed specs for a bit, and the way I see them, they can be just a s/select counterpart, where s/select describes lower bound of data shape ("be at least that"), while s/closed describes upper bound ("be at most that").
By composable I mean using same specs both as open and as closed in same s/valid? etc. checks. I think it might make sense, for example, if I were to spec a function that cleans user input before putting it into db, it's spec would look like that:
Speaking of specs evolution, do you mean being able to tell if change of spec from a to b is accretive or not? I don't see problems here as well:
- changing (s/closed (s/keys [::name])) to (s/closed (s/keys [::name ::age])) in return position provides more — it returned ::name, now it also can return ::age;
- changing (s/closed (s/keys [::name])) to (s/closed (s/keys [::name ::age])) in argument position requires less — it didn't accept ::age, now it accepts it;
- changing (s/closed (s/keys [::name])) to (s/keys [::name]) in return position provides more — it used to return only ::name, now it may return more.
2019:10:25 12:39:45 alexmiller those words do not match what you're saying in the specs. the closed version says "MUST NOT contain ..."
1/3. is probably ok from the consumers point of view but is contextual to use
2. if you provide age on the first one, then forbid on the second one, you've broken users that previously passed age
2019:10:25 12:42:32 alexmiller this notion of context is critical and is driving the changes in schema/select/fdef, so I wouldn't rule out some way to make a more restrictive statement at the point of use, although we currently think of that again as more of a "check", not part of a spec
2019:10:25 13:04:56 vlaaad > those words do not match what you're saying in the specs
what words?
2019:10:25 13:07:47 vlaaad maybe we have different definitions of closed? My (s/closed [::name ::address]) says that map may be empty, or contain ::name, or contain ::address, or contain both, but no extra keys are allowed
2019:10:25 13:11:13 vlaaad like select, that requires at least this and allows more, but in different direction: requires at most this and allows less. in different contexts select, closed and a combination of both might be useful
2019:10:25 13:12:52 vlaaad so in "2" you could not provide ::age before: this is invalid, and version after allows it
2019:10:28 16:56:53 vlaaad @alexmiller bump
2019:10:24 21:57:51 ikitommi What if they were part of the select?
2019:10:24 22:32:17 alexmiller well, again the principle here is that "closedness" is part of the checking, never part of the spec
2019:10:25 04:43:31 ikitommi I agree on that, but have thought that select is a utility of making checking context (values!) from schema. It would allow one place & syntax to easily close any of the nested maps too. The current syntax of closing specs by name in s/valid? with different (nested maps) syntax adds more things for the developer to keep in his/her head. Need to jump to both schema and select definitions to see what subkeys are including in the select and need to be closed. Also, if schemas & selects get refactored, the closed options might get out-of-sync.
2019:10:24 22:05:25 ikitommi In the code I pasted above, keysets need to be introduced in three places: the root schema, the api schema and the select. Also, partially in the s/valid? call to make it closed.
2019:10:24 22:15:45 seancorfield I was a bit surprised by this:
user=> (require '[clojure.spec-alpha2 :as s])
nil
user=> (s/def ::thing (s/schema [::a ::b ::c]))
:user/thing
user=> (s/valid? (s/select ::thing [::a ::b]) {::a 1 ::b 2 ::c 3} {:closed #{::thing}})
true ; fine -- ::c is optional but present
user=> (s/valid? (s/select ::thing [::a ::b]) {::a 1 ::b 2} {:closed #{::thing}})
true ; fine -- ::c is optional and omitted
user=> (s/valid? (s/select ::thing [::a ::b]) {::a 1 ::b 2 ::d 4} {:closed #{::thing}})
true ; huh? -- ::d is not in ::thing -- why doesn't this fail?
user=> (s/valid? (s/select ::thing [::a ::b]) {::a 1 ::d 4} {:closed #{::thing}})
false ; because ::b is required but omitted... see next...
user=> (s/explain-str (s/select ::thing [::a ::b]) {::a 1 ::d 4} {:closed #{::thing}})
"#:user{:a 1, :d 4} - failed: (fn [m] (contains? m :user/b))\n"
user=>
2019:10:24 22:29:55 alexmiller there's some lingering tech debt in the schema/select code where some things are duplicated rather than having select lean on schema's validation code. I think once that's cleaned up, these will fail as you expect
2019:10:27 23:01:11 kenny Is there a way to write s/ands without having the conformed value passed to the predicate? All of our s/and predicates call s/unform on the value before calling the actual predicate function. This seems unnecessary. For example:
2019:10:28 00:15:08 alexmiller currently, no. we will probably have both flowing and unflowing forms in spec 2
2019:10:28 07:53:49 misha you might get away with moving spec to the last position in the s/and, depending on resilience of your predicates to random input. @kenny
or, if this is the only place you use predicates – make them expect conformed value
2019:10:28 16:05:29 kenny Hmm. Can't do the latter because the predicates are used on the regular form of the data too.
2019:10:28 16:19:09 alexmiller you could wrap s/nonconforming around each pred
2019:10:28 17:15:34 ataggart Per the description of every, the :kind pred "must generate in order for every to generate". I see that preds like set? can generate, e.g., (gen/sample (s/gen (s/spec set?))), but I'm at a loss as to how to provide generators for custom preds.
2019:10:28 22:51:45 alexmiller there is no open mechanism to bind generators to custom preds currently
2019:10:28 22:53:30 alexmiller you can provide the :kind pred as a separate spec outside s/every too
(s/and (s/with-gen my-pred ...) (s/every ...))
2019:10:28 22:54:44 alexmiller that said, what kind of pred are you using that's not a core pred?
2019:10:29 15:32:23 ataggart The specific case of wanting to use every over a priority queue.
2019:10:29 15:51:09 alexmiller prob easier to attach a generator to the overall spec
2019:10:29 16:30:14 ataggart Yup
2019:10:29 19:59:43 ataggart I assume the same goes for predicates like uuid?
2019:10:29 20:00:06 ataggart In that we can't provide generators directly for custom predicates
2019:10:29 20:00:26 alexmiller I think uuid? has a generator? or maybe that's just one we've talked about
2019:10:29 20:01:04 alexmiller yeah, uuid? gens
2019:10:29 20:01:34 alexmiller you can generally wrap specs with custom generators around preds with s/with-gen or s/spec
2019:10:29 20:01:57 alexmiller the particular case of :kind in s/every is a place where you don't have that opportunity
2019:10:29 20:46:34 ataggart I've been fighting to successfully use with-gen for about an hour now
2019:10:29 20:47:36 ataggart And I just figured it out.
2019:10:29 20:48:07 ataggart spec.gen wants fns for everything, but I'm used to test.check which doesn't.
2019:10:29 20:48:33 ataggart I'm good now, thanks for your help
2019:10:29 20:51:41 alexmiller yep, it's thunked to make the load step dynamic and optional
2019:10:29 20:54:48 ataggart Yeah, makes sense
2019:10:29 20:55:02 ataggart It's a one-time problem
2019:10:29 20:55:04 ataggart Now I know
2019:10:29 07:46:48 jumar Looking at spectrum, I can't figure out how to use it: https://github.com/arohner/spectrum
I know it's (still) not recommended for any kind of use, but I was curious about what it can do.
However, I don't even know what kind of output can I expect from it.
Trying the most basic example doesn't produce anything useful:
2019:10:29 21:34:56 ataggart I'm not yet sufficiently fluent in spec-error-ese.
2019:10:29 21:48:40 andy.fingerhut @ataggart Most likely an older version of clojure.tools.logging from before Clojure 1.9 was released, that contains some old syntax that Clojure 1.9's and later's stricter checking rejects, and someone has since fixed with a later tools.logging release.
2019:10:29 21:49:13 ataggart I'm using 0.5.0, which is the latest version. And that "someone" would be me. 😛
2019:10:29 21:49:56 ataggart deps hell once again
2019:10:29 21:49:57 andy.fingerhut Sorry, I was going for an answer appropriate for 99% of similar questions. Looking at the line of code now...
2019:10:29 21:51:11 seancorfield (and I did wonder if a version conflict was bringing in an older version)
2019:10:29 21:52:37 seancorfield FWIW, we use tools.logging 0.5.0 at work with Clojure 1.10.1 -- which includes all the Spec checking on defn -- and we don't see any problems.
2019:10:29 21:52:51 seancorfield What other dependencies have you got in play?
2019:10:29 21:53:52 ataggart Yep, I'm confident it's a deps issue. I'll keep digging. Aside, any guess on what that error means?
2019:10:29 21:54:32 seancorfield It's a badly formed defn. Expecting an argument vector or sequence of pairs of arg vectors / bodies.
2019:10:29 21:54:39 andy.fingerhut It mentions param-list, so my first guess would be it thinks the defn parameter list is not a vector
2019:10:29 21:54:44 ataggart weird
2019:10:29 21:56:07 seancorfield Are you using lein/boot/clj? What does your deps tree show? Could it be a badly-behaved plugin patching stuff?
2019:10:29 21:56:37 ataggart I'll find out and report back.
2019:10:29 22:03:53 ataggart So, I changed the version to 0.4.1, same error. Changed it back to 0.5.0, and the error is gone....
2019:10:29 22:04:38 andy.fingerhut Please keep your gremlins away from my computer.
2019:10:29 22:05:36 ataggart The vision of a world of spec'd libraries without versions can't come soon enough.
2019:10:29 22:06:06 ataggart Back to work....
2019:10:29 23:05:18 alexmiller maybe a cached classpath? changing the deps file would invalidate the cache
2019:10:29 23:05:41 alexmiller so any change would have forced an update
2019:10:29 23:06:08 alexmiller if you're ever not sure with clojure, adding an -Sforce can force that regardless
2019:10:30 13:26:29 ataggart Is there a canonical way to generate only a subset of the “types” from a multi-spec?
2019:10:30 13:33:43 sgerguri You can force a multimethod clause to be selected with the following:
(s/gen (<multispec> <dispatch-arg>)))
2019:10:30 13:34:36 sgerguri To pick a subset of the multispecs you can make a collection of the valid dispatch args, then randomly sample that.
2019:10:30 13:35:06 ataggart Thanks!
2019:10:30 14:22:29 ataggart Hmm... doing that is yielding maps that have the correct shape for the type, but the value of the dispatching type key is random, not the dispatch value.
2019:10:30 14:24:14 ataggart I'm guessing it's because the result of (s/gen ...) isn't flowing through the multi-spec's retag.
2019:10:30 14:25:56 sgerguri That all depends on how tight your multispec clauses are. You can override what you get with fmap like this:
2019:10:30 14:26:31 ataggart >That all depends on how tight your multispec clauses are.
What do you mean?
2019:10:30 14:28:04 sgerguri For example, I have a multispec where I dispatch on two keys in the map I generate. The keys are unqualified, meaning I can strictly enforce that for a particular clause, I always get a particular value. Like this:
2019:10:30 14:29:25 sgerguri So even if the s/keys is part of a multispec for a dispatch value that involves :foo, because the dispatch is done based on the value of a function applied to the data this way I make sure that when the above clause is selected, :foo will be the value of the key :type.
2019:10:30 14:29:49 sgerguri In general though, if you do the fmap above it should be fine.
2019:10:30 14:30:10 sgerguri Irrespective of whether you do something similar to what I just described or not.
2019:10:30 15:27:28 ataggart @U064X3EF3 Continuing the multi-spec example from the official guide, does this seem reasonable for the simple case of a kw retag?
2019:10:30 14:33:49 ataggart The intention of the second one is to only yield :event/search.
2019:10:30 14:35:25 ataggart That's why I think just calling the multi-method isn't enough. Generating via the multi-spec would flow through retag, but the code above doesn't.
2019:10:30 14:37:29 ataggart The master approaches...
2019:10:30 14:37:44 alexmiller I think going back to your original question, the answer is no, there is no way to (easily) generate a subset of the multispec values right now
2019:10:30 14:38:06 sgerguri If you fmap, however, you should still get the right thing.
2019:10:30 14:38:10 ataggart Man, I was all braced to receive your wisdom, Alex.
2019:10:30 14:38:15 alexmiller there are a variety of hacks :)
2019:10:30 14:38:19 ataggart Yeah, I can manually retag
2019:10:30 14:39:01 alexmiller you can choose a particular one and retag, or you can dive into the guts of the multimethod dispatch value map (which is what the multispec generator does)
2019:10:30 14:39:15 sgerguri Also :event/type is defined as a keyword in the example, whereas in mine above I restrict that particular spec to a single value.
2019:10:30 14:39:34 alexmiller or you could s/and and only take events of a particular type
2019:10:30 14:39:41 sgerguri It just so happens that I have the same unqualified key in all the multispecs defined difrerently per multispec.
2019:10:30 14:40:04 ataggart Every time I try to use s/and for this issue I end up with such-that errors
2019:10:30 14:40:23 sgerguri s/with-gen is your friend there.
2019:10:30 14:40:46 sgerguri Don't be afraid to use it - I do in strategic places, typically around complex s/and cases.
2019:10:30 14:40:56 ataggart problem is I can't tell what's causing the such-that error
2019:10:30 14:41:14 ataggart Well, I can eventually, but it's not obvious when it happens
2019:10:30 14:41:28 ataggart ok, I'm going to beat my head on this a little more
2019:10:30 14:41:31 sgerguri Well if it's an s/and then simply provide your own generator by wrapping that in s/with-gen.
2019:10:30 14:41:49 ataggart The issue tends to be using s/and when I should use s/merge
2019:10:30 14:42:28 sgerguri Possibly. Though if you rely on custom conformers, s/and is a very frequent thing to end up using.
2019:10:30 14:43:51 ataggart yep, just wasn't helping in this case
2019:10:30 14:44:06 ataggart retagging via fmap over the generated values from the multimethod is working though.
2019:10:30 15:06:39 orestis I know it's a vague question, but what's the upgrade path from spec -> spec2 look like? I know there's changes coming in s/keys and I guess the functions that validate/conform/explain might change, but I was wondering how much of the spec definitions will change?
2019:10:30 15:08:08 orestis I'm trying to figure out if we should start with spec2 directly, and deal with the breakage as it comes (I guess it's pre-alpha), or start with spec and deal with the upgrade breakage when it comes.
2019:10:30 17:05:46 seancorfield We've been trying to keep a branch of our codebase at work up to date on Spec 2 but it's required quite a few changes to some of our Spec 1 code and at this point, we've essentially "given up" on a straightforward migration. We plan to adopt Spec 2 for new code once it is out of pre-alpha status and run a mixed Spec 1 / Spec 2 codebase for a good long while.
2019:10:30 15:16:20 alexmiller I would use spec right now
2019:10:30 15:17:50 alexmiller the api functions - validate/conform/explain etc have gained some capabilities, but I expect will continue to work in spec 2 as is
2019:10:30 15:19:31 mpenet any news on eta? we're also waiting on v2 release for some new features (looking in particular at new data spec format for "interpretation" of specs)
2019:10:30 15:19:34 alexmiller the spec op forms will mostly be the same, although keys -> schema/select will be a significant change and there are some differences around the use of sets, preds, functions, etc at thee top level
2019:10:30 15:20:05 alexmiller I don't have an eta but I'd guess it's on the order of months
2019:10:30 15:32:59 alexmiller I guess?
2019:10:30 15:34:51 ataggart heh, ok, thanks
2019:10:30 16:59:22 orestis Thanks for your reply Alex, much appreciated.
2019:10:30 17:07:33 seancorfield @orestis As Alex says, use Spec 1 for the time being. Conversion to Spec 2 may be easy or hard depending on how you use Spec 1.
2019:10:30 17:17:15 orestis The thing is I need to do closed map checking, recursively. I think that it can be hacked on top of spec1, but not sure if it’s already done and if not, if worth the effort to do myself.
2019:10:30 17:22:32 seancorfield Is the project a production-level one, or likely to be one within the next couple of months?
2019:10:30 17:22:46 seancorfield If not, I'd say give Spec 2 a go. Otherwise, stick to Spec 1.
2019:10:30 17:23:51 alexmiller I wouldn't, I'd say stick to spec 1 :)
2019:10:30 17:25:47 alexmiller there is no release of spec 2 available, because I do not think it is release-worthy yet. until then, I would not recommend for anything other than tire-kicking and feedback (which is very welcome)
2019:10:30 17:28:08 seancorfield Right, that's what I meant about not production-level -- if this is just an exercise project, go kick the tires. Otherwise, stick to Spec 1.
2019:10:30 17:28:52 seancorfield I really like where Spec 2 is going but it's a different beast in many ways from Spec 1...
2019:10:30 18:26:38 orestis Production-level. Ok, I’m hearing you loud and clear :)
2019:10:30 20:59:51 mloughlin The book Secure By Design from Manning writes about using Java classes with immutable properties and data contracts in the constructor, then only using this defined class (instead of using, say, a string for a bank account number) in order to prevent business logic bugs. The nice thing about this is the type system enforces a degree of defensive programming.
My question: is there an clojure.spec analogue, or convention for this?
2019:10:30 21:29:50 alexmiller mostly I think that's trying to patch problems that are endemic to use of Java, rather than an actually good idea. however, I'd say the closest thing is to separate specs into two levels - domain definitions (::domain/bank-acct-no) and uses in specific contexts that alias those domain definitions
2019:10:30 21:30:30 alexmiller but this is really mostly just making complicated definitions once rather than repeating them
2019:10:30 22:40:43 mloughlin would you validate as a pre-condition every time you were to use ::domain/bank-acct-no? I suppose it's super cheap computationally
2019:10:30 22:41:24 alexmiller No, I would not
2019:10:30 22:41:52 alexmiller I would use instrument to check during dev/test
2019:10:30 22:43:19 mloughlin thanks for clarifying
2019:10:30 22:43:21 mloughlin 🙂
2019:11:01 09:51:21 Nazral hi
2019:11:01 09:52:43 Nazral I'm trying to write a macro that among other things needs to generate new specs by wrapping or merging existing ones, and it doesn't work. I made the simple following example
I'm not sure why doesn't this work? I tried with a symbol instead of a keyword too, to no avail
2019:11:01 10:12:56 Nazral I also tried with
n-name `(symbol (str (.name *ns*)) '~base-name)
to no avail
2019:11:01 12:51:26 alexmiller did you look at the expansion?
2019:11:01 12:52:27 alexmiller just as a general technique it's important to (pprint (macroexpand-1 '(wrapspec 'abc int?))) or whatever to see if it's doing what you want
2019:11:01 12:53:30 alexmiller here s/coll-of is going to be a spec object, which is not what you want
2019:11:01 12:53:58 alexmiller
new-spec `(s/coll-of ~base-spec)
2019:11:01 12:54:09 alexmiller probably closer but I'm just eyeballing this
2019:11:01 21:25:17 Flo Anyone aware of issues with clojure.spec.alpha/gen in cljs (using shadow-cljs)? This snippet works well in clj repl:
(s/def ::foo string?)
(s/gen ::foo)
but fails in cljs with: Var clojure.test.check.generators/simple-type-printable does not exist, clojure.test.check.generators never required at eval (/js/compiled/cljs-runtime/cljs.spec.gen.alpha.js:1854:8)
2019:11:01 21:26:30 Flo My project.clj has a dependency entry for [org.clojure/test.check "0.10.0"]
2019:11:01 21:28:38 Flo Nevermind, after I explicitly required [clojure.test.check.generators] in the same file I'm (s/gen ..)ing, it works.
2019:11:02 08:35:58 kurt-o-sys I'm having a pretty extensive spec, with quite some s/or parts. When any data structure doesn't conform, it shows the issues, as expected. Conforming structures show that they conform... so all is well so far.
However, what I'd like to have, is to know which 'path' has been taken to conform:
How can I ensure with spec that a) a :parent is never the same key as the current line or ideally b) that :parent is a valid key in the map? I read through the website and some tutorials and I don't know if that's even possible with spec. Any advise would be super helpful
2019:11:02 20:02:19 andy.fingerhut You can write arbitrary functions that return true or false for a given value, and use those in your specs, so if you can write a normal Clojure function to do it, you can write a spec to do it. Some of those reduce the amount of help you get from random generation of values, etc.
2019:11:02 20:04:21 coder46334 This sounds useful, can you hint me what this is called in the docs so I can look it up? Also, I thought it would be cool to have the test cases auto-generated, so that won't work?
2019:11:02 20:05:39 andy.fingerhut This property of functions is mentioned very early in the spec guide article here: https://clojure.org/guides/spec under the heading "Predicates"
2019:11:02 20:07:04 andy.fingerhut I am not a knowledgeable enough user of spec to give you the best answer to how to auto-generate things with custom predicates, but there are places in the docs that describe how to create custom random value generators and use those instead of the default built-in ones.
2019:11:02 20:10:21 coder46334 I guess where I'm stuck at is how I bring that into the s/def of a map-of. I can't put it onto the smaller pieces, because they'd have to reference each other
2019:11:02 20:11:45 coder46334 Ok, I thought of looking into the generators, but was hoping there is an easier solution to define it on the specs. There are not really a lot of examples out there that are more complex than one-liners. Thanks for your help!
2019:11:02 20:12:30 coder46334 And if anyone has an idea about how to link it with the s/def of the map-of I'm still curious. Will be back at the computer later and will try to find out
2019:11:02 20:14:35 andy.fingerhut I believe s/and can combine arbitrary specs into one?
2019:11:02 20:17:59 coder46334 I'll try later - thanks!
2019:11:03 18:03:23 alex-dixon Anyone interested in spec as a parser combinator library?
2019:11:04 10:06:58 acron Is there a preference to using either s/* or s/coll-of when we just want 'a collection of x`? I realise coll-of has more tuning, but besides that, is there difference between (s/* ::foo) and (s/coll-of ::foo) ?
2019:11:05 13:41:18 jeremys Hey guys I ran into something I’d like to ask you about. I wanted to “merge” to s/keys specs using s/and. It didn’t work as I thought it would. Here is an example showing what I wanted to do:
Do I use s/and in an unintended way? Is ::v2 the correct way to do it? Did I miss something in spec’s api?
2019:11:05 13:48:09 sgerguri @jeremys Sounds to me like you want s/merge.
2019:11:05 13:57:32 jeremys @sgerguri as usual I missed something in the api... Thanks a lot mate
2019:11:05 14:07:30 jeremys It’s so logical also, as you merge maps, you merge specs about maps...
2019:11:05 17:46:46 colinkahn Is it a good idea to define your generators in the same file as your specs? Sometimes I have generators that need my specs and vise-versa. Couldn’t tell if it was good practice to keep them together though.
2019:11:06 02:02:23 sgerguri I (and my colleagues) tend to do that, then things are nicely collocated. Doubly useful when your spec is a complicated s/and - if you wrap with s/with-gen it's all neatly in one place.
2019:11:06 21:21:04 telekid Is there a functional difference between (s/spec some-pred :gen some-gen) and (s/with-gen some-pred (constantly some-gen))?
2019:11:07 01:52:30 alexmiller No
2019:11:07 21:30:56 pablore is there an expected date for spec 1.0.0 ?
2019:11:07 21:32:59 alexmiller no
2019:11:07 21:39:20 vlaaad will there be spec 1.0.0?
2019:11:07 21:39:35 vlaaad and tools.deps without alpha
2019:11:07 21:42:23 alexmiller yes
2019:11:07 22:18:20 seancorfield spec 1.0.0 will be Spec 2 when it's "baked", right @alexmiller ?
2019:11:07 22:36:59 alexmiller most likely
2019:11:07 22:37:18 alexmiller I don't know that it will be 1.0.0 exactly but it will be some non-alpha version
2019:11:08 00:45:15 pablore what are the main differences between spec 2 and alpha?
2019:11:08 00:48:04 kszabo mainly how they handle aggregates of attributes. You only specify required/optional attributes at the context of the usage site
2019:11:08 00:48:48 kszabo ‘bags’ of attributes are now the primary way of mapping domain entities in spec2, they don’t talk about requirements
2019:11:08 00:49:05 kszabo also some programmatic convenience improvements
2019:11:08 00:49:09 kszabo of course I am oversimplifying
2019:11:08 16:23:52 wilkerlucio hello, with spec1, I'm having issues to define a spec that is not loaded, I get Unable to resolve spec: ..., consider this is file B, the spec in question is defined by A, but A requires B, so I can't load A before loading B (circular reference), with that constraint, is there a way to tell spec to ignore that the spec is not defined at that time?
2019:11:08 16:26:37 alexmiller most spec types delay resolving references so don't run into this, but there are a few that don't
2019:11:08 16:26:58 alexmiller for those that don't, you'll need to break that cycle somehow
2019:11:08 16:43:46 wilkerlucio thanks, I got rid of the cycle by moving those spec definitions to a new namespace (since I we can define things from other namespaces, so still have the same name, just defined in a different file)
2019:11:08 21:01:53 didibus What's the best way to restrict a string to a min/max length, so that the generator still works for it ?
2019:11:08 21:04:58 alexmiller I don't think there is a general way to do that without writing a generator
2019:11:08 21:06:20 alexmiller (and to do that, I would generate strings of length up to (max-min) and a string of length min, then concat)
2019:11:08 21:06:48 didibus Hum, alright, I'll use a custom generator then
2019:11:08 21:06:50 didibus thx
2019:11:09 17:17:35 cddr I made a thing to capture one side of that constraint as part of a toolkit for generating test-data to match a SQL schema....
2019:11:09 20:18:43 bertofer Is there any way to add a docstring to specs?
2019:11:09 20:56:11 alexmiller no, not currently
2019:11:09 23:51:03 bertofer Are there any plans to add it? I think it could be interesting to have docstring when using spec to model the domain of an application
2019:11:10 00:28:48 alexmiller yes, it is the highest voted ticket in the tracker :)
2019:11:10 00:29:17 bertofer great, thanks :+1:
2019:11:10 00:30:12 alexmiller it's on the list for spec 2, but we haven't gotten to it yet
2019:11:12 00:49:22 didibus FYI, its pretty easy to generate a string generator of min/max length using: #(sgen/fmap str/join (sgen/vector (sgen/char-alphanumeric) min max))
2019:11:12 14:18:43 coder46334 Hi! When I start shadow-cljs with :browser-test , insead of :app, my spec instrumentation doesn't work. The specs work with (exercise fully-qualified-ns/spec), but I can't instrument functions either through code or repl. Any ideas?
2019:11:12 14:22:36 aisamu Is something from the test namespaces requiring the files containing the specs?
2019:11:12 14:24:20 coder46334 Yes. I also have an instrument command in that namespace and also tried it through REPL while in that namespace
2019:11:12 14:26:11 coder46334 I can even copy the specs into REPL, run the instrument command there and it's not working
2019:11:12 14:47:48 aisamu This is probably something with your code.
We have specs on test ns's running with :browser-test, and there's nothing special about the invocation/setup:
(stest/instrument `ns/component)
2019:11:12 14:49:25 aisamu It's worth noting that the instrument call must happen after the fdef, so mind the namespaces loading order
2019:11:12 15:00:47 coder46334 I just drop-in replaced it with orchestra-cljs.spec.test and it works
2019:11:12 15:01:12 coder46334 yes, the instrument calls are after the fdefs
2019:11:12 15:02:09 coder46334 So with orchestra it works, but no idea why it doesn't with normal spec. It's hard to debug when you don't get an error, it just doesn't work 😞
2019:11:12 22:51:13 olivergeorge I find myself wringing my hands when writing specs in my CLJS code. Particularly choosing namespaces for my specs. I'd like something like (s/def ::list-options/options (s/coll-of ::option)) to mean (s/def :current-ns.list-options/options (s/coll-of ::option)) where the alias isn't defined in the current ns.
2019:11:12 22:51:13 olivergeorge I find myself wringing my hands when writing specs in my CLJS code. Particularly choosing namespaces for my specs. I'd like something like (s/def ::list-options/options (s/coll-of ::option)) to mean (s/def :current-ns.list-options/options (s/coll-of ::option)) where the alias isn't defined in the current ns.
2019:11:13 01:12:37 ataggart I've found it convenient to always define specs as ::foo, and reference specs with ::foo or ::x/foo. It eliminates a lot of bikeshedding and subtle bugs. I want it to fail when the ns alias isn't defined.
2019:11:13 01:21:22 ataggart Oh, I see what you mean. You could just move the dot segments to the other side of the slash, e.g., ::list-options.options.
2019:11:13 04:58:45 olivergeorge That works sometimes but for s/keys you need the name part to match so it's the namespace part which needs to be adaptable/unique.
2019:11:13 15:46:36 ataggart Ah, yeah, if you're using unnamespaced keywords
2019:11:15 05:00:52 valerauko i wanted a similar thing for db enums so i made it.
this validates keywords like :my-model.status/ok against ::my-model/status to see if "ok" is a valid value
2019:11:12 22:51:24 olivergeorge Or something similar and less complected of course.
2019:11:12 22:53:13 olivergeorge Perhaps something like clojure.core/alias for CLJS would solve the problem.
2019:11:14 04:10:22 danielcompton Is there a ticket for spec to bring in the rest of the generators in clojure.test.check.generators?
2019:11:14 05:05:58 alexmiller we did not have a general goal to replicate all aspects of test.check.generators in spec.gen
2019:11:14 05:06:09 alexmiller so, no as we do not intend to do so
2019:11:14 05:06:26 alexmiller you can just use them from test.check if you need them?
2019:11:14 16:33:21 kenny The problem is that you want the lazy loading spec.gen ns provides.
2019:11:14 17:02:35 alexmiller is there something specific you're looking for?
2019:11:14 17:24:12 kenny No runtime dependency on test.check.
2019:11:14 17:25:16 alexmiller I mean which generator functions?
2019:11:14 17:41:50 kenny The ones that we have copied over are: generator?, let, vector-distinct-by. I believe there may be some others scattered throughout the code though.
2019:11:14 17:42:54 alexmiller is the dynaload stuff in gen exposed enough to just use?
2019:11:14 17:44:07 kenny We had to copy over lazy-combinator macro because dynaload was private.
2019:11:14 17:44:24 alexmiller lazy-combinator is public though, right?
2019:11:14 17:44:39 alexmiller ah, so you can't get to the expansion, right
2019:11:14 17:44:44 kenny Right
2019:11:14 17:47:00 alexmiller let's a macro - so I presume that's not something you can dynaload?
2019:11:14 17:47:10 alexmiller there actually is a ticket about that one in particular
2019:11:14 17:47:41 kenny Yes. But let is used a ton so having that in the spec gen ns would be very helpful.
2019:11:14 17:47:47 alexmiller vector-distinct-by could be done through (s/gen (s/coll-of ::foo :distinct true :into [])) ?
2019:11:14 17:48:02 alexmiller oh, distinct-by
2019:11:14 17:48:04 alexmiller nvm
2019:11:14 17:49:34 alexmiller generator? is prob weird given the thunking done in gen isn't it?
2019:11:14 17:50:19 kenny This is what we are using:
(def generator? #'gen/generator?)
2019:11:14 17:50:29 kenny where gen is clojure.spec.gen.alpha
2019:11:14 17:52:04 alexmiller so these each seem like distinct issues
2019:11:14 17:52:43 alexmiller there's a ticket for let already, generator? could just be made public, and vector-distinct-by could be added to the dynaload list
2019:11:14 17:54:07 kenny Yep. @U051KLSJF may be interested in some other missing ones. I'm pretty sure there's a few others I've run into.
2019:11:14 18:02:39 alexmiller I've made the generator? and vector-distinct-by changes in spec-alpha2
2019:11:14 18:03:14 kenny Thanks!
2019:11:14 20:04:08 danielcompton I was missing byte which prompted me to ask, there are also more generators around ints which aren’t exposed
2019:11:14 15:31:47 ataggart I find that the things in clojure.spec.gen.alpha are sufficient to define generators for specs, and I only need the things in clojure.test.check.generators for generating "interesting" data for tests.
2019:11:14 16:03:36 alexmiller you can get pretty far by using (s/gen <some-spec>) too
2019:11:14 16:03:58 alexmiller often I find that's easier that constructing the generator from scratch
2019:11:15 18:14:09 dpkp How do folks usually choose between req and req-un ? Namespaces are great for organizing, but do folks usually drop them after validation w req-un? Or maybe coerce to namespaced keys? Or require upstream users to pass namespaced keys?
2019:11:15 19:27:25 seancorfield @dpkp "It Depends". We tend to use :req-un for input validation specs (coming from JSON or URL/form input) but :req for quite a bit of internal stuff, including database-related data (where we either use clojure.java.jdbc's :qualifier option to add a generic "namespace" to all columns, or next.jdbc's default table-qualified column names).
2019:11:15 19:29:03 seancorfield It does mean that at system boundaries you have to decide where (or even whether) to switch from unqualified keys to qualified keys -- but I think that's reasonable since I would expect input data to be fairly "flat" and domain data to be a more interesting shape, i.e., I would expect a mapping from input -> domain to exist anyway in your code, even in the absence of Spec.
2019:11:15 22:12:38 cjmurphy When you believe a spec is not being picked up for whatever reason (I'm using guardrails/ghostwheel - therefore expound and spec are picked up), is there a way to clear the registry, just to test the assumption?
2019:11:15 22:18:38 ghadi what do you mean by "not being picked up"?
2019:11:15 22:50:05 cjmurphy Thanks that helps. But I just verified they are as they are written in the file that defines them. The problem is unlikely to be with spec itself. But if I could somehow 'invalidate the cache' I'd be able to force the other tools to recalculate, hopefully...
2019:11:15 22:33:16 cjmurphy @ghadi 'not being picked up": I have a spec for essential-rule and another for rule. One of the function parameters has been changed from rule to essential-rule, yet there's still a spec failure and the expound message thinks that the function parameter is rule rather than the new and more lax essential-rule. I've touched various files, restarted the JVM/REPL. But still somehow 'it' still thinks the function is different to the way it is written.
2019:11:15 22:52:06 kszabo if you restarted the JVM then it’s not a cached thing
2019:11:15 23:02:13 cjmurphy I worked it out, and its not the first time I've fallen for this 😞. The function in question is multi-arity and I was changing one of the arities and not the other. Thanks for your help kszabo and ghadi.
2019:11:17 13:37:43 tianshu in spec2, should {:closed true} be available for schema spec?
2019:11:17 15:40:37 alexmiller It’s a set of schema names, not a boolean
2019:11:17 22:28:12 coder46334 Hi! Is there a way to spec a return type of Promise<something> ? I would need this in quite a few places using cljs
2019:11:17 23:19:41 alexmiller Not usefully, no
2019:11:17 23:22:31 coder46334 Thanks!
2019:11:18 04:28:51 tianshu @alexmiller thanks!
2019:11:18 21:05:52 colinkahn I’ve used with-redefs around an st/check in ClojureScript code and it has worked fine, but trying in Clojure code it doesn’t. I had to redefine a my-function-with-redefs and spec that with the redefs inside the body. Is that expected? I’m assuming it’s something to do with multiple threads vs single thread in JS
2019:11:18 21:24:50 alexmiller might be - st/check uses pmap which is multi-threaded in Clojure (or possibly it's really the laziness of pmap)
2019:11:18 21:25:38 alexmiller make sure you realize the result of pmap with doall or something inside the lexical scope of the with-redefs
2019:11:19 16:41:12 colinkahn Awesome! That worked
2019:11:19 17:09:25 Lukasz Hall Data generation question -
I have a top-level (map) spec composed of many nested specs and their associated generators at a leaf level. Some of the leaves of the nested structures use the same spec as other leaves, and I need those that share a spec to produce the same values when they generate. Is there any way to have this happen without explicitly writing a generator for the top level spec?
2019:11:19 17:37:02 alexmiller No
2019:11:19 17:37:42 alexmiller To guarantee properties at the aggregate level, you need to control generation at the aggregate, as a general principle
2019:11:19 20:21:50 telekid I’m curious how people integrate instrumentation with testing. Do they create an instrumentation fixture and call it in a :once block at the top of every test ns? If so, how do you restore instrumentation to its initial state at the end of that fixture?
2019:11:19 20:39:52 alexmiller unstrument exists to do that
2019:11:19 21:43:24 seancorfield @jake142 I instrument directly at the top of a test file (after the ns form) and don't bother undoing it. I do it separately from my test fixtures because those are often functions I reuse across multiple test namespaces, whereas the instrumentation that I do (if any) is usually limited to the functions under test in a specific namespace.
2019:11:20 10:21:18 abdullahibra hello everyone
2019:11:20 10:21:32 abdullahibra what is the spec for checking file data type ?
2019:11:20 10:53:39 vlaaad you mean like that #(instance? java.io.File %)?
2019:11:20 13:33:01 Daouda Hey Folks, I am learning spec and need help to understand the registry parte. They say that it help to avoid name collision, but since we already have namespaces, they aren't enough to avoid that? I don't get to see the real advantage of using :: in spec declaration. Can someone help me to understand please with example will be wonderful
2019:11:20 13:58:47 alexmiller ::keywords are “auto-resolved” to have the current namespace so it’s just a useful tool for succinctly making qualified keywords
2019:11:20 13:59:39 alexmiller They create the exact same result as using :my.ns/keyword so use that if you prefer!
2019:11:20 14:00:10 alexmiller You’ll see them in a lot of examples as it just makes the example shorter
2019:11:20 16:13:34 Daouda Thank you very much @alexmiller
2019:11:20 17:00:55 Daouda I think I`ve got the big picture now. When you want to use a spec/def, first spec look in the spec-registry if exist then take from there. Else it create it and store it in the spec-registry. A very naive description of course hehehe
2019:11:20 19:33:27 Daouda Hey Folks, when should I use qualified or unqualifiedkeys in clojure spec?
2019:11:20 19:43:53 alexmiller qualified
2019:11:20 19:44:30 alexmiller unless you are working with existing data, don't want to transform it, and it has unqualified keys
2019:11:21 11:12:55 Daouda Hey folks, can you help me to understand why I can't gen/generateor gen/sample:
(def email-regex #"^[a-zA-Z0-9._%+-]
2019:11:21 11:18:31 guy Have you tried exercising your specs?
2019:11:21 11:18:36 guy Couldn't satisfy such-that predicate after 100 tries
2019:11:21 11:18:41 guy Thats the key part to the error message
2019:11:21 11:18:54 guy I would just go through each def and see if they can produce a result
2019:11:21 11:19:14 vlaaad my bet is on email and phone
2019:11:21 11:19:20 guy yes i would agree
2019:11:21 11:19:51 guy If the predicate is too hard to er fufill? based on random string input, don't you usually just create a generator for those complicated predicates?
2019:11:21 11:20:54 guy also i'm not all that sure how (spec/def ::phone #"^\d{12}$") works
2019:11:21 11:26:17 guy If you exercised ::phone what would you get? thats a pattern isnt it 🤔
2019:11:21 11:28:06 guy Unable to construct gen at: []
2019:11:21 11:28:11 guy Thats what i get
2019:11:21 11:28:18 guy for (s/exercise ::phone)
2019:11:21 11:29:10 Daouda Just saw now, the answers, will be right back after doing what you said 😄
2019:11:21 11:29:30 guy So then I think for the email, its just as simple as, the string generator can't create a random entry that matches your regex
2019:11:21 11:29:50 guy which makes sense right? as even after 100 times its going to be difficult to randomly create an email i think haha
2019:11:21 22:18:19 sgerguri I am trialling offloading some extra logic onto s/or and would like some collective opinion, if y'all were so kind.
Suppose I have a spec defined like this:
(s/or :x :x/item
:y :y/item)
Now suppose I add x1 and want to combine it with x at some point; so let's say I start making the semantic distinction between what this thing is and what it can be combined into:
Now, suppose further that I want to encode the order in which the respective operation should be defined. Now I'm thinking it would be really useful to be a bit more descriptive in my tags, so why not use a map?
Now I'm thinking, should I simply add this via a conformer, keeping the tags simple (and thus transforming them at a higher level elsewhere in the spec) or do I keep this?
This might all seem like a contrived example - and it is, because my use case isn't algebraic operations, but rather data merging, and thus it's really important to define what order each data types should be merged in, and what the parent type is, so that the correct merge strategy is applied.
My question is - am I really using s/or in a way I shouldn't be or would you still consider this ok?
2019:11:22 06:27:05 jaihindhreddy My 2c is that: this is trying to do too much with spec. I'd keep the tags simple keywords, and do the manipulation later, using spec only to suss out the implicit ordering or types into explicit data (via cat and or respectively).
2019:11:22 09:59:43 sgerguri I'm always torn between that and doing more. Spec feels like it's one step away from automatic transformations, with conformers being the key enabling element for this.
2019:11:22 10:42:44 jaihindhreddy I can sympathise 100%
2019:11:22 10:43:23 jaihindhreddy But spec's neither a kitchen sink, nor a silver bullet 😀
2019:11:22 11:06:57 sgerguri I think if specs supported metadata one could use simple tags and just indicate code-level stuff there. Perhaps we'll get that functionality in version 2.
2019:11:22 11:38:34 jaihindhreddy I dunno if spec wants to do that. Because the awesome thing in Clojure is that, we already have namespaced keywords, which are (when correctly used) globally precise names.
And this means just like spec maintains a global registry of specs attached to these names for the purpose of specification, we can write our own things which maintain global registries of other stuff attached to the same names, for use in other contexts. I think this is one of those instances where making things global makes sense, and alleviates parochiality. For this kind of a thing to be realistic though, I feel like the idea of providing and requiring contexts needs to be first class and written down in code, so that automatic verification of breakage can be detected and avoided, making way for an ecosystem that can integrate various things seamlessly while still being dynamic and "growable"
2019:11:22 13:48:22 alexmiller conform is designed to tell you why something was validated and separate data into parsed components
2019:11:22 14:00:59 alexmiller It is not a general “data transform meat grinder” and you’ll be disappointed if you try to use it that way
2019:11:25 16:58:08 vemv Given a ns that goes like this:
...how to generate :some-events only with :some-dispatch-value type? i.e., exclude all other dispatch values (i.e. the specs coming from other defmethods)
and ::organization would be specced to have the namespaced nested data? I'm a bit unsure on how to proceed since the official guide doesn't really have explicit examples of nested data as far as I can see
2019:11:27 11:51:41 vemv I'd say you're going in the right direction
try the pattern you have posted, and verify if you are getting the desired validations.
Else post the exact failed attempt here
2019:11:28 00:24:47 vemv I authored https://github.com/nedap/speced.def and we're happy with it at work. Since it compiles to a clojure/script defn it has zero limitations or new syntax, guaranteed.
it doesn't preclude creating generic/reusable specs: you can author those and still use them here.
It has no :fn equivalent (only :`args` :ret ones), but honestly I have yet to see how that is not equivalently achieved with the occasional :pre condition.
2019:11:28 00:25:33 vemv I haven't campaigned much about it so far, but I plan to post something to #news-and-articles later this week :)
2019:11:27 22:56:28 seancorfield @bbloom I don't use it -- I often put fdefs in a separate namespace in libraries so they are optional and the code can then run on older Clojure versions.
2019:11:27 22:57:10 seancorfield (and I don't like that syntax so I wouldn't want to use it even when I would put fdefs above the function for which they exist)
2019:11:27 22:58:58 bbloom thanks for the thoughts - i’m not using anything like it yet b/c i like to kinda use the primitives until i grok them fully and then and only then abstract more
2019:11:27 22:58:58 seancorfield Those limitations alone would make it non-viable for most of my fdef use cases I think.
2019:11:27 22:59:17 bbloom i’m still trying to figure out where/when to utilize spec
2019:11:27 22:59:57 bbloom i’ve been doing a lot of TypeScript for work lately, and all the inline annotations are simultaneously: laborious, helpful, and hurtful
2019:11:27 23:07:19 bbloom thanks @seancorfield, that all makes sense to me - to some extent i’m trying to decide how far to do w/ development time checking
2019:11:27 23:09:36 bbloom my current thinking is basically just-in-time specing, where i spec stuff as part of my debugging process and leave the specs around for future use
2019:11:27 23:13:13 didibus I've got a library like that in the works, probably 80% done at this point. Don't know if or when I'll finish it. But my focus is on fn specs, since those are the painful ones to write in my opinion. As well as multi-arity, var-args, et all. Those are the hard ones to write, so my DSL intends to make these way quicker to spec, as to encourage more spec writing. Right now the machinery attaches itself as metadata on defns, so it also is compatible with older Clojure versions.
2019:11:27 23:14:34 didibus Some edge cases can be tricky, which is why it's taking me longer than I initially thought to put together.
2019:11:27 23:15:52 didibus But I do think I'd write more fn specs if defn had a easier syntax for writing them. So that's what I'm trying to achieve. I also recognize it is hard to come up with one that is simpler but not limited. Which is why there probably isn't an official one.
2019:11:28 00:29:19 alexmiller That’s on the table for spec 2
2019:11:28 00:29:54 alexmiller But they are likely to be significantly different than spec 1
2019:11:28 02:01:33 bbloom @alexmiller Could you please clarify: “That’s” == what exactly?
2019:11:28 02:04:37 alexmiller specs integrated into defn
2019:11:28 02:05:12 bbloom ah, ok, just wanted to be sure that’s what you meant 🙂 very cool. looking forward to seeing what you folks come up with
2019:11:28 02:34:28 didibus Why is there no support for and/or inside of s/keys :opt ?
2019:11:28 02:35:27 didibus
(s/keys :req-un [::a]
:opt-un [(and ::b ::c)])
2019:11:28 02:35:59 didibus In my case, ::b and ::c are optional, but if anyone of them is there, the other has to be as well
2019:11:28 02:45:07 alexmiller Because optional things are ... optional
2019:11:28 02:45:31 alexmiller You’ll need to s/and another predicate if you have additional constraints
2019:11:28 02:46:19 didibus Hum.. its just more annoying, because the combinations to list out with s/and can quickly grow
2019:11:28 02:46:52 didibus but that's what I'll do for now
2019:11:28 16:14:38 gws You could do something like this, which doesn't require you to list all possible optional combinations
2019:12:02 00:16:25 frederic Hi. Do you have guidelines, or rules of thumb for naming specs in the following two situations
1. do you rather choose a name that describes the role that the data plays in the enclosing structure (say, ::confirmation-url), or a name that describes the intrinsic kind of data that key holds (say, ::url). I tend towards the former, because I think the more descriptive keys read better, but I'm wondering if by doing so I'm not inadvertantly creating private DSL (fragmenting keys when I could have one large overarching key instead and maximise the number of functions that can operate on the data)
2. when do you use symbols rather than keywords to name specs? I tend to use symbols as a kind of zero-arg spec-op, specs that I think of as building blocks, reserving keywords for specs that I would expect to actually appear in maps. Is that a personal quirk of mine?
2019:12:02 00:21:40 alexmiller 1. I would say both. Name the generic thing, then have the specific one refer to the generic one.
2019:12:02 00:22:07 alexmiller 2. You should never name specs with symbols (those are reserved for function specs)
2019:12:02 00:22:56 alexmiller Lots of places in spec make that assumption and you are likely to be burned by using symbols.
2019:12:02 00:27:16 frederic Thank you! That answer was fast and very clear cut, I appreciate that 🙂
2019:12:02 02:09:05 favila By “symbols” do you mean vars with def or literally (s/def ‘symbol ...)?
2019:12:02 02:11:06 favila I go back and forth on whether I should register specs or just def the spec object. I’m on a “don’t register” swing now
2019:12:02 02:23:38 seancorfield @favila (s/def ::keyword ...) vs (s/def 'symbol ...) -- the code assumes that only function specs use symbols (s/fdef 'foo ...)
2019:12:02 02:26:08 favila I know this, I am asking OP what he meant
2019:12:02 02:31:22 seancorfield Both Alex and I made the same assumption and the OP didn't correct Alex so... ¯\(ツ)/¯
2019:12:02 11:12:59 frederic You were correct about what I meant, @seancorfield. Thanks for suggesting using a plain def when there's no need to s/def a keyword, @favila.
2019:12:03 09:13:30 Quest Anyone have a convenient solution for defining clojure specs at runtime? Channel logs say either wrap with another macro OR to use eval.
Macro approach requires everything else to become a macro & still doesn't allow full runtime extension.
Eval works but creates more complexity to get it working in CLJS.
2019:12:03 09:18:44 Quest Interestingly, I can simply copy my own version of res (private FN from clojure.spec.alpha) & suddenly the below works.
(s/def-impl k (res v) v)
While hackish it proves that dynamic definition works just fine, but seems to have been intentionally disallowed? Though I still prefer over the eval nonsense.
(Which for completeness sake is included below:)
(eval (list 's/def k v))
2019:12:03 09:31:20 Quest Looks like CLJS still hiccups on the def-impl approach due this a difference in how clojure.core/resolve works. It's a macro in CLJS but a FN in Clojure? (I'm curious how the impl differs under the hood if anyone knows...)
The offending line:
(symbol? form) (c/or (-> form resolve ->sym) form)
In my case I'm just passing namespaced keywords at runtime, so this part is unnecessary for me. Simply commenting it out obviates the issue.
I'm sure this breaks many best practice recommendations, but if this is the most direct solution then I'm happy with it. I understand that spec2 should support this more easily & development is focused there, meaning that spec1 is unlikely to change & break this approach.
2019:12:03 10:33:58 alexmiller Correct. There are multiple non-macro ways to make this work now in spec 2
2019:12:04 13:38:43 Vincent Cantin Hi, I am trying to define a cross-recursive spec (i.e. 2 specs defined using the other one) using spec-alpha2. How to do that?
2019:12:04 13:40:16 Vincent Cantin My repl complains on the first definition, saying that it does not find the second one (which is defined after)
2019:12:04 13:57:55 alexmiller That won’t work at the moment
2019:12:04 13:58:39 alexmiller There’s a pass needed to ensure every spec properly delays resolution of named specs
2019:12:04 14:11:09 vlaaad will it work in spec-alpha2?
2019:12:04 14:20:04 alexmiller yes
2019:12:04 14:20:10 alexmiller it's just work I haven't done yet
2019:12:04 14:20:19 alexmiller it (mostly) works in spec 1
(gen/sample (s/gen (s/select {:a int? :b keyword?} [:a])) 5)
Error printing return value (ExceptionInfo) at clojure.test.check.generators/such-that-helper (generators.cljc:320).
Couldn't satisfy such-that predicate after 100 tries.
2019:12:05 03:22:58 seancorfield I expect that's just a bug in master -- or a doc bug.
2019:12:05 03:23:58 seancorfield @lukaszkorecki Spec2 is very much a work-in-progress right now.
2019:12:05 03:24:22 seancorfield @hall ^
2019:12:05 09:35:38 Quest Took a closer look at spec2. Really like ease of programmatic definition & similar API is a plus. Good vision & look forward to moving to it after CLJS support 🙂👍
2019:12:05 10:21:01 rickmoynihan In lieu of spec 2, is there any advice on how to structure spec 1 specs, so that they’re more likely to be easily portable to spec 2's schema / style?
2019:12:05 10:21:17 rickmoynihan For example, defining essentially the schema (leaf) keys, and then essentially defining what would be the select ions on the fdefs.
2019:12:05 10:21:20 rickmoynihan Obviously that can lead to some duplication… is there a way in spec 1 to dissoc an s/key:req from an s/keys spec?
2019:12:05 10:22:19 rickmoynihan I can’t see such a thing; so I’m thinking it’s better to build the composites from multiple s/keys at the fdefsites with s/merge?
2019:12:05 10:28:53 rickmoynihan i.e. it seems you can kind of simulate spec2's idiomatic structure with spec1
2019:12:05 12:52:05 kszabo I had this discussion with Alex before, your ‘record types’ should all use :opt/`opt-un` , at usage sites (endpoints/fdefs) you can tighten them down by (s/merge ::user (s/keys :req [:user/name :user/age]))
2019:12:05 13:16:36 rickmoynihan @thenonameguy: great, that’s what I’ve been doing. Thanks for the clarification. Is ‘record types’ a phrase Alex mentioned, or is it one you’ve coined (hence the bunnies?)
I ask because I would’ve thought he would’ve said ‘schema types’ or just ‘schemas’, given that is the terminology used in spec2.
2019:12:05 13:17:12 rickmoynihan Or is there some more nuance to that?
2019:12:05 13:24:06 kszabo I just came up with it, case class/record type/schema, there are lots of names for arbitrary groupings of attributes. That is my conclusion from these spec2 developments: Aggregates are incidental and are context-dependent.
2019:12:05 13:33:34 kszabo And you can see the industry trend towards systems that are attribute-focused and not aggregate focused. Think of SQL vs Datomic for instance, folks appreciate the multi-dimensional flexibility of EAV* stores vs predetermined tables. There is also REST vs GraphQL where the predetermined ‘resource’ is inferior to getting just the attributes that you want.
2019:12:05 13:34:56 rickmoynihan Yeah, I do a lot of work with RDF; which is pretty much the poster child for property-based open-world thinking.
2019:12:05 14:08:08 alexmiller and the inspiration for parts of spec
2019:12:05 14:24:01 rickmoynihan yeah and it really shows… same for clojure too actually 👍
2019:12:05 14:31:24 rickmoynihan Incidentally I’m not sure if you or Rich had seen SHACL or SHEX; but they’re remarkably similar to spec (but for RDF):
https://www.w3.org/TR/shacl/https://shex.io/
I’ve often wondered if they were part of the inspiration behind spec, or whether spec and SHACL/SHEX just ended up in the same sort of place because of the nature of RDF / OWL.
Either explanation seem plausible to me.
2019:12:05 14:34:25 rickmoynihan Incidentally this (free) online book describes both of them and their usage, and their differences: https://book.validatingrdf.com/
2019:12:05 13:35:11 kszabo Although you can take one more step and go to Pathom which is even more attribute-focused, there are no predetermined ‘types’ at all, very RDF-like
2019:12:05 13:37:16 rickmoynihan I should check out pathom… it’s been on my radar for a while — but I’ve not really dug into it, so thanks for the pointer.
2019:12:05 13:38:24 kszabo We are in the process of adding Pathom to all of our APIs so that we can later have a unified ‘maximal’ graph, it has been great so far
2019:12:05 14:58:02 eggsyntax I thought there was a way to say in a map spec that a key could be either qualifed or unqualified. I misremembered that :req-un behaved that way, but apparently not:
Is there any good way to say that either qualified or unqualified is fine? I'd like to be able to reuse the same spec both in internal contexts where keys are qualified and at the system boundary where keys come in unqualified. The simplest thing I've thought of is to use :req-un and always strip namespaces before checking for validity. But I thought that spec itself had a way to express that...
2019:12:05 15:04:56 alexmiller it doesn't
2019:12:05 15:05:08 eggsyntax Oh well. Thanks, Alex!
Might be an interesting feature to consider at some point, it'd make spec reuse easier and encourage people to use qualified keys when possible rather than defaulting to the least common denominator of unqualified keys (in cases where they have to deal with unqualified keys somewhere in their system).
2019:12:05 15:31:51 alexmiller the intent with spec is to strongly encourage you to use qualified keys
2019:12:05 15:32:05 alexmiller spec 2 has more support for working with unqual in schemas though
2019:12:05 15:32:30 alexmiller unqualified that are not tied by name to qualified, but just directly to specs
2019:12:05 16:23:20 eggsyntax Meaning something like this?
2019:12:05 17:46:17 eggsyntax In a spirit of spec 2 brainstorming -- it seems like this case is similar to closed spec checking: something that it's better to avoid, but in some specific contexts turns out to be necessary. It would be cool if rather than write the boilerplate I gave above (which would be tedious and noisy in the case of an edge function that takes a map with a lot of unqualified keys) you could use an :unqualified setting with conforming API calls (just like with :closed):
(s/valid? ::foo {:bar "a"} {:unqualified true})
This also somewhat parallels the use of :keys in map destructuring for the common case where you want the binding names to be the same as the key names -- ie it handles the common case where the unqualified keys are the same as the names of the qualified keys.
Just a thought 🙂
2019:12:05 18:05:45 alexmiller again, trying to encourage qualified keys
2019:12:05 18:06:50 eggsyntax Fair enough 🙂
2019:12:05 18:06:54 alexmiller the settings are really for conform ops (valid? / conform / explain)
2019:12:05 18:07:07 alexmiller unqualified would also need to be in play for gen
2019:12:05 18:10:01 eggsyntax That makes sense. I was only suggesting it for valid? / conform / explain, but I see how gen breaks the parallelism with :closed in an important way.
2019:12:05 16:19:49 eggsyntax > the intent with spec is to strongly encourage you to use qualified keys
For sure, and I'm 100% bought into that idea. But sometimes at the edges you have to accept unqualified keys, and it'd be nice to be able to say, even on a case-by-case basis, "I'm looking for qualified keys, but I'm willing to accept unqualified" rather than (as currently) either
a) making a separate unqualified spec for the same sort of data structure (eg a user), for use at the edges; or
b) using a :req-un spec across the board and internally stripping namespaces from keys before you check for validity.
What would you say is the best way to handle that situation?
2019:12:05 16:23:11 alexmiller I think I'd either do a) or go the other way and add a pre-transformation to qualify the attributes so they match the qualified specs
2019:12:05 16:23:41 alexmiller b is stripping good information so I don't think I'd do that
2019:12:05 16:24:00 alexmiller if you're going to change the data, at least make it better :)
2019:12:05 16:24:33 eggsyntax Good point, and I should have thought of that since I do have places where I already do that 😆
2019:12:06 11:55:21 Flo How can I check whether the (optional) keys are within a set? e.g. valid keys are #{:foo :bar :qux} and would (s/valid? :myspec {:foo true})
2019:12:06 11:58:17 Flo ah! map-of is my friend
2019:12:06 14:45:35 bertofer When using spec in the context of domain modelling, sometimes I see that some of the require I do are only for specs, as some entities would have properties from a different entity (sometimes to define relationships). This makes it very easy to end up with circular dependencies, is there any way on the namespace declaration to have only ns aliases, without requiring them to avoid circular dependencies?
2019:12:06 14:46:32 alexmiller no, but you don't need to require them at all to use keywords with a certain namespace
2019:12:06 14:46:41 alexmiller you just have to write the full namespace out w/o aliases
2019:12:06 14:48:33 bertofer Ok, I thought about that, I wanted to know if there was any way to avoid having to write the full namespace
2019:12:06 18:59:56 seancorfield That looks a lot more work and complexity than a simple alias/`create-ns` like I pasted in the main channel. Could you explain what your macros are trying to do, beyond just the alias/`create-ns` call I showed?
2019:12:06 20:43:49 telekid Yeah, I think my def-synthetic-ns fn is overly complex, probably because of a combination of “my first macro” confusion and not noticing that create-ns returns the ns
2019:12:06 17:14:31 vemv not sure if I already mentioned this one to you last time we talked, but I tend to use a namespace called kws.clj (for "keywords") which contains only specs, and zero implementation details (other than very simple predicates)
since kws definitionally depend on nothing, you can always depend on a "kws" ns, and no circular deps will arise
2019:12:06 17:59:59 bertofer That could be useful, but I prefer to have different namespaces as I want to keep ns keywords on the namespace that contains function that operate on them, for now I am using keywords under namespaces that do not correspond to the files, like :product/id instead of ::product/id , even if I define it’s spec in ::product namespace. I prefer this to writing the full namespace, for that I’ll probably wait until there is a better way of doing so
2019:12:06 18:09:04 vemv I see!
In case it wasn't clear, I actually use one kws ns per model (or 'module', depending on the chosen arch)
e.g. product.clj + product/kws.clj
2019:12:06 18:18:19 bertofer Oh ok, I thought you meant one big kws. Although this still does not solve interdependency between models where you want to reference each other through the specs, as you might need to import to define specs from another place. Image orders from a product, you might have ::product/id inside an order, and have ::orders referencing orders from product
2019:12:06 18:24:59 vemv that design sounds problematic per se tbh: the DB design might not be sound, and in any case the generative part of spec might choke on the mutual recursion
2019:12:06 18:30:21 bertofer But databases have relationships of this kind, by the product_id of an order you are able to get all orders from a product (::orders) . It would be the case when you want to hydrate a one-to-many relationship, the many has id of the one , and the one has an array of the many
2019:12:06 18:33:46 bertofer I am not very familiar with the generative part of spec, but a problem I can see in generative is that it wouldn’t generate the same :product/id for all orders when generating a set of orders, aside from that the specs per se are not recursive, no spec end up calling itself, but both reference things from the other
2019:12:06 18:36:29 vemv one-to-many is easily modeled as:
product/kws.clj # zero references to orders; since products exist independently from orders
order/kws.clj # references product specs; this is a simple (non-cyclic) dependency
and even a fancy many-to-many relationship can be modeled as:
product/kws.clj # zero references to orders
order/kws.clj # zero references to products
order_product/kws.clj # depends on orders and products
2019:12:06 18:38:48 vemv I wouldn't let a trivial concern like ids (normally ids have the same type across all SQL tables) introduce cyclical deps
IMO that's true regardless of the way you decide to use spec
2019:12:06 18:55:31 bertofer I still think could be useful to have some product info on the order itself (without directly referencing a product). Maybe this scenario is not the ideal one for this use case, but I still think you might want to reference inside your schema specs from other ns, I don’t think it’s bad to have cyclical namespaced keywords inside specs, as it’s not a dependency itself, and allows a lot of flexibility for how you retrieve your domain entities. In my case the spec are not directly translated to database tables or columns (at least not a database I control), and sometimes properties from a parent entity are hydrated directly on the child entity, not always referencing by id
2019:12:06 18:58:23 bertofer > I wouldn’t let a trivial concern like ids (normally ids have the same type across all SQL tables) introduce cyclical deps
Namespaced keywords do not need a require, so it’s not strictly a cyclical dep, that’s only for aliasing them, and I think it is better to have the flexibility of being able to do that (or at least as it is now just using full namespace)
2019:12:06 19:00:36 vemv Could be, I'd have to check it out carefully
Still, keep in mind that specs are composable.
so you can have non-cyclical, discrete product and order specs, and then completely aside, a hydrated-order spec that composes both.
keeps things non-cyclical, without compromising DRY either :)
2019:12:06 19:02:56 vemv also keep in mind that at write time, order should only have thin references to products. so a full-blown spec could be an impediment, as it could require more product keys to be present.
the hypothetical hydrated-order spec keeps read/write concerns separate
2019:12:06 19:04:29 bertofer schema/select allow to handle those contextual dependencies
2019:12:06 19:05:05 bertofer Basically I am going for these specs with schema like “This is everything an order might contain”, then by context I could select some properties
2019:12:06 19:05:39 vemv yeah. it's in the radar for spec 2 as you probably know
2019:12:06 19:06:41 vemv must be said, considering graphql-style responses, a monolithic spec can end up being problematic. there can be N "read models" of a given thing
2019:12:06 19:06:44 bertofer Yeah, as this is a personal project I am already using it, so my concern was on that context
2019:12:06 19:07:42 bertofer > must be said, considering graphql-style responses, a monolithic spec can end up being problematic. there can be N “read models” of a given thing
I am not familiar with graphql, what do you mean N read models?
2019:12:06 19:09:54 vemv there are N possible schemas ("models") in which a client might want a response
e.g. [[1 2] [3 4]] or {1 2 3 4} (same data, different representation)
a monolithic spec cannot reasonably define all schemas that client could request
2019:12:06 19:16:07 bertofer Why not? And what do you mean by monolithic schema?
2019:12:06 19:18:11 vemv monolithic: a single spec that can be used for describing both reads and writes, relying on select for flexiblity
2019:12:06 19:19:48 vemv problem with that:
different clients (e.g. web, android, ios) might want different data shapes, pagination, camelCase, referenced models, etc
probably a single spec defining all possible combinations of those would not be maintainable
2019:12:06 19:55:26 bertofer What would be best in that case, one per read model and one per write?
2019:12:06 19:56:27 bertofer I think I would have 1 spec for the read you retrieve from the DB, and then map it to whatever the client expects, probably not event having a schema for the specific data shape of the client, or if you want, then a spec for each of those too
2019:12:06 19:59:40 bertofer It’s far from my use case though
2019:12:07 03:29:48 vemv > What would be best in that case, one per read model and one per write?
at work I advocate the following:
* there's one write model (since there should be only 1 acceptable way of persisting things)
* by default, that also serves as the read model (since one always can read a subset of what he wrote)
* there also can be N additional read models, as your needs grow
so, rule of thumb is 1:N
2019:12:07 03:31:52 vemv > It’s far from my use case though (edited)
yeah I don't even particularly like graphql, and in a personal project you likely won't have 10 different clients ;p
but I like finding patterns that scale, that are universal. that way I don't have to worry later, or have different patterns per-project
2019:12:07 08:45:15 bertofer > or have different patterns per-project
I disagree here, depending on the scope of the project this could make sense
2019:12:07 08:56:59 vemv morning :)
sure thing. Personally I enjoy having a minimal amount of patterns in my head. The tradeoff being that sometimes 'scalable' solutions can certainly feel as overkill
2019:12:07 09:13:20 bertofer Morning 👋
I think though that scalable could mean different things on different projects, to justify different patterns to achieve such scalability
2019:12:06 18:12:23 seancorfield You can also do this to get a local alias without needing the full namespace to actually exist as code:
(alias 'm (create-ns 'ws.domain.member))
2019:12:06 19:03:23 bertofer Thanks! This looks really good and something that solves my use case 🙂
2019:12:06 18:12:49 seancorfield Then ::m/country will expand to :ws.domain.member/country for the spec name.
2019:12:06 18:13:44 seancorfield (in this particular case, that ns does exist, but we don't need to require it everywhere we use the specs)
2019:12:07 03:49:31 vemv Genuine question (and not an exercise in snobbery ;p), I wonder if many people's seeming disfavour for ns-qualified keywords comes from lack of (properly setup or understood?) IDE tooling.
With cljr-slash (https://github.com/clojure-emacs/clj-refactor.el/blob/50d2d8aad5e0bd8002173b300f8419d72ceab7af/clj-refactor.el#L1968 ), I type ::foo/ and the right :require will be inserted in my ns declaration, and the requiring will be actually performed into the runtime
that's an instantaneous op, and after that, tab completions for ::foo/ will be 100% accurate
I can imagine that if lacking this tooling, using ns-qualified kws could be more tedious.
2019:12:07 03:53:18 eggsyntax Oooh, I hadn’t stumbled on cljr-slash before, thanks for pointing that out!
2019:12:07 03:54:56 eggsyntax I can say that for me personally, I found it annoying using the more verbose namespaced keys, until I started to feel the benefits directly. Now I’m the person who’s pushing people toward namespacing keywords everywhere…
2019:12:07 03:54:16 seancorfield I don't find auto-completion/hinting to be any obstacle at all to using ns-qualified keywords, to be honest. My set up is Atom/Chlorine with its bare-bones completion (without Compliment). Stu Halloway says he doesn't use auto-complete/hinting at all in his editor setup -- and he seems to use ns-qualified keywords just fine 🙂
2019:12:07 03:57:40 seancorfield As I'm converting code at work from clojure.java.jdbc to next.jdbc I'm working more and more with (table)-qualified keywords and I like it. I'm even changing some old code that still uses c.j.j to use the :qualifier option to add a table-qualifier to the column keywords 🙂
2019:12:07 03:59:12 vemv interesting design, thanks for sharing!
2019:12:07 04:01:57 seancorfield It was this that caused me to update my CFML/Clojure bridge library to support namespace-qualified keys in hash maps for interop, BTW 🙂
2019:12:08 22:25:53 frederic Is this supposed to work in spec 2?
(s/def ::some-keyword ::some-other-keyword)
When I try to validate against ::some-keyword I get this exception:
java.lang.IllegalArgumentException: No implementation of method: :conform* of protocol: #'clojure.spec-alpha2.protocols/Spec found for class: clojure.lang.Keyword
I can fix the symptoms by doing this instead of s/def, but the fact that I have to do a special dance makes me suspect that I might be working against the tool's grain
2019:12:08 22:34:09 frederic I get that spec2 is stricter than spec1 in terms of separating symbolic specs from spec objects.
From the documentation (https://github.com/clojure/spec-alpha2/wiki/Differences-from-spec.alpha) (emphasis mine)
> s/def is a helpful wrapper macro for s/register that understands how to interpret all kinds of symbolic specs (not just spec forms), including symbols and sets, which will be wrapped in the helper s/spec spec op.
I was hoping that ::some-other-keyword would qualify as a symbolic spec, but maybe not.
2019:12:08 22:34:09 seancorfield I believe aliased Specs are supposed to work but there are situations where you can forward reference in Spec 1 but you cannot yet forward reference in Spec 2.
2019:12:08 22:34:31 seancorfield I assume ::some-other-keyword is defined after ::some-keyword here?
2019:12:08 22:35:30 frederic Actually, it's imported from another ns in my real code, shouldn't have tried to simplify that away
2019:12:08 22:35:54 frederic There must be something else going on then
2019:12:08 22:36:42 frederic Let's see if I can build a minimal example that reproduces the behaviour I observe
2019:12:08 22:36:52 seancorfield In that simple case, it does let you forward reference FYI -- I tried it in a fresh REPL session with the defs of ::one and ::two swapped and it still worked.
2019:12:08 22:40:44 frederic Interesting. Thank you for the handholding 😊, will try to figure out what I'm doing differently and report back
2019:12:08 22:52:34 seancorfield With the caveat, again, that there are numerous bugs in Spec2 still at this point. And it sounds like function specs will change dramatically before they're done, according to Alex's latest blog point (and s/and may become non-flowing -- which would be another departure from Spec1). While I was working against the Spec2 branch for a while, things would break between various updates on master, sometimes intentionally but often unintentionally. It's very much a moving target right now.
2019:12:08 22:54:05 seancorfield I had hoped to migrate our (large) codebase from Spec1 to Spec2 but, in reality, I think we're "stuck" with Spec1 in our existing code at this point and we'll just adopt Spec2 for new code (depending on how Spec1 and Spec2 interop -- because they don't, right now).
2019:12:08 22:54:31 seancorfield Eventually, we'll convert everything over but I suspect we'll do it piecemeal over time.
2019:12:08 22:58:00 frederic Mine is just a hobby project, but yes, I might have been over eager there 🙂
2019:12:08 23:00:23 frederic Maybe I should just switch back to spec 1, and follow the advice above (<https://clojurians.slack.com/archives/C1B1BB2Q3/p1575541261064300>) to make all my specs :opt rather than :req, merging in :req at the last minute to emulate schema and select.
2019:12:08 23:00:59 frederic That way I'd also get orchestra and expound, which would be nice
2019:12:08 23:01:59 frederic And probably other libraries and tooling integration that I'm not aware of yet.
2019:12:08 23:04:07 eggsyntax > make all my specs :opt rather than :req, merging in :req at the last minute to emulate schema and select.
I hadn’t seen the discussion of that above, but that’s pretty much what I’ve come to independently for a large-ish set of domain specs I’ve been working on. Seems to be a decent approach.
2019:12:08 23:06:22 eggsyntax Along with a couple of helper functions to eg build a selection from a schema less verbosely by passing in the base schema and the list of attributes that should be required (in the simple cases), and by passing in the base schema and a full tree of requiredness (for complex cases).
2019:12:09 01:51:07 frederic Converting to spec 1 fixed my problem. I must have run into some corner case that wasn't covered yet.
2019:12:09 15:52:59 arohner I have a multi-spec which works fine, but for a test I want to gen messages of a single type. If I do (defmethod my-spec ::foo [_] ::foo), then the messages are missing retag. If I include the retag key in ::foo, the keys are auto-generated and don’t correspond to the message type. What’s the best way to make the correct retag show up in the individual messages?
2019:12:09 17:48:44 alexmiller if you hit the backchannel here, several people have walked through this recently - zulip has the archive
2019:12:10 09:33:01 bertofer In spec2, can you create a subschema of a schema, like doing select-keys on schema keeping the optionality?
2019:12:10 10:21:05 vlaaad isn't it what select is for?
2019:12:10 10:32:20 bertofer select returns the required keys from a schema on a specific context, I am looking for something to generate schemas of optionals from a subset of keys of another schema
2019:12:10 13:44:29 alexmiller Schema keys are all optional. I don’t understand what you’re trying to do.
2019:12:10 14:05:47 bertofer Imagine I have an entity, for which I have a spec/schema with a set of keys that it can have.
Now, I would like to spec the request parameters to modify the given entity. You can only modify half of the parameters of that entity, although each of them is still optional.
How would I approach this case? Does it make sense?
2019:12:10 14:16:01 alexmiller a select is exactly the case of specifying a more constrained structure based on a schema, intended specifically for "in a context", like request parameters
2019:12:10 14:17:02 alexmiller if the keys are optional, then the existing schema suffices
2019:12:10 14:41:54 bertofer I see, in terms of validation and instrumentation it makes sense, just use the “big schema” with all possible keys and select if there is any requirement. I guess what I was trying to achieve was more aligned with a “specification / documentation” use of schema, so I could tell which of the keys are supported by a specific function, rather tan required
2019:12:10 14:52:31 bertofer Is there in spec any concept or would it make sense to have something to specify that a property is supported? with a meaning of The property will be used if it exists, but it's not required , or is it something that should be left to documentation / outside of the code specification?
2019:12:10 14:59:31 alexmiller nothing like that in spec, no
2019:12:10 15:12:40 bertofer Thanks alex. Would it make sense in the context of spec? I think it could be something very close to the concept of select . As I imagine it, would be:
• schema: All keys that travel together, outside of function context (all are optional, although wouldn’t optional also be contextual? optional meaning supported or used)
• select: All keys that are used in a specific context (With the slight change that it would allow to select optional and required keys for the case mentioned above)
Is this something that makes sense? Am I missing something in how I conceive spec?
2019:12:10 15:17:16 alexmiller schemas are all optional, select is all about what's required
2019:12:10 15:58:24 bertofer I (think I) understand, what I am trying to achieve maybe fits more into the metadata space (saying what is used if specified, for auto generated doc or generative test) rather than in spec.
I am trying to wrap my mind around the philosophy behind the schema/select approach, I already find it quite useful how it is now, although sometimes I find myself in a 3-concepts space rather than two (schema/select) when thinking about the information model:
1. What travels together (big schema with all related kw, e.g. domain entity)
2. What a context uses (subset of kw used by the fn. Not everything used is required, but it’s not ignored)
3. What a context requires to work properly (required set of kw for the fn to work)
Maybe it makes sense to have smaller schemas in 2. with shared kw with 1. , or to have 2. just as metadata of the fn outisde spec. Any ideas on this ?
Seems a good time to re-watch Maybe not , I might be missing something about the philosophy of the tool
2019:12:10 16:01:57 alexmiller On 2, you don't have to use a predefined schema, you can just restate specifically what attrs are allowed, which may be a smaller set
2019:12:10 16:02:35 alexmiller or you can just make a schema that is a smaller set
2019:12:10 16:17:49 bertofer > or you can just make a schema that is a smaller set
I’ll go for that. Which brings me to the beginning, create schema from subset of another schema, (or join schema’s together). Probably something that can be done as tooling outside spec2.
Looking forward for a final release 🙌 cheers!
2019:12:10 17:28:54 seancorfield s/merge could merge smaller schemas to make a bigger schema, yes?
2019:12:10 17:31:24 bertofer ^ Indeed !
2019:12:10 17:33:12 bertofer By <https://github.com/clojure/spec-alpha2/wiki/Schema-and-select> I think s/union would be the one to use on schemas
2019:12:10 17:45:10 bertofer Thinking about my previous comment of generating schema from subset of schemas, it doesn’t make any sense :man-facepalming: Schema’s syntax is just the set of keys (when ns-kw at least)
2019:12:10 17:46:46 alexmiller We may actually re-purpose “union” for nonflowing s/and so don’t get too attached to that name :)
2019:12:10 18:07:03 seancorfield Usual caveat: Spec2 is a moving target whose function names may change and/or whose semantics may change, potentially several times 🙂
2019:12:10 18:26:55 bertofer Yeah, luckily it is a personal project with a small codebase, refactors aren’t very painful (I do lots of them to try new approaches). Having that into account, I was avoiding to go much further than def’ing schemas, which it feels already useful to define my domain entities and properties (and understand them) as a common ground between a set of external services that all of them use similar entities but each one of them with different set of properties and property names and formats
2019:12:10 11:13:25 mpenet basically spec will happily validate against stuff that was never declared, just checking for presence of the key as if any?
2019:12:10 11:15:31 mpenet the docstring is not really clear about that one
2019:12:10 13:45:47 alexmiller s/keys checks that required keys exists and that all registered keys have values that conform
2019:12:10 13:47:07 alexmiller So, not a bug. I think you’ll get exceptions if you try to gen though
2019:12:10 14:03:12 mpenet yes, it throws
2019:12:10 20:35:17 danielcompton Is there a way to validate a "closed select"? I can see how to check a schema with closed settings, but it doesn't seem like that works for selects?
2019:12:11 00:44:47 alexmiller I think the select linkage is not fully implemented yet, long story, wip
2019:12:10 20:36:17 danielcompton The use case is for API boundaries, where you want to make a selection of keys that you will accept, fail if extra keys are provided, and especially to not validate the extra keys before failing
2019:12:11 02:56:41 bbrinck Slightly off-topic, but I have a question about what people would prefer in their error messages. I’d appreciate a vote if you have a minute https://twitter.com/bbrinck/status/1204595098207444993
2019:12:11 22:43:00 kenny Does anyone have a keys spec but for a map with string keys?
2019:12:11 23:00:00 seancorfield @kenny Not sure how you'd do that -- s/keys requires keywords I believe? You could use s/map-of for string keys (and add some custom predicates depending on how much checking you need)...
2019:12:11 23:00:35 kenny Yeah that's what I have. Was hoping someone had written something already that did that & generated valid values.
2019:12:11 23:17:13 alexmiller not that I'm aware of
2019:12:11 23:17:39 alexmiller you can get some of what you want by combining s/and and an s/conformer that keywordizes keys
2019:12:11 23:18:13 alexmiller not sure it works on all the api operations the way you want though
2019:12:12 01:05:31 olivergeorge Hi Alex, I’m super excited about the new spec. Just trying to moderate my expectations… Do you think it will be a complex port to clojurescript? From memory the first version took a while (months?) to come out.
2019:12:12 01:10:36 alexmiller ¯\(ツ)/¯
2019:12:12 01:12:27 alexmiller I think there were a couple tricky parts to the cljs port in gen/instrument/check - I suspect those tricky bits aren't going to change, at least in the tricky parts. And I think the increased clarity around symbols vs objects is probably likely to make the cljs code larger but simpler
2019:12:12 01:13:35 alexmiller so I don't see any reason it should be particularly hard, but there is a lot of internal structural change, enough that if it were me doing it, I'd probably start fresh and pull from the old code as needed
2019:12:12 01:14:54 alexmiller that said, we're not done yet :)
2019:12:12 01:24:20 olivergeorge Thanks
2019:12:12 01:25:50 olivergeorge While I’ve got you can I ask about the idea of including specs in defn. Do you think we will see :args and :ret keys added to the pre/post condition map?
2019:12:12 03:11:22 alexmiller Haven’t looked at that yet
2019:12:12 03:12:18 alexmiller Ret and fn specs are likely going to change a lot
2019:12:12 16:13:31 gariepyalex I am trying without success to generate clj-time dates using specs in clj-time.spec . (s/exercise ::spec/date-time) always returns dates close to January 1rst 2011 by a few millis. The spec in the lib is using:
which always returns numbers close to 1293840000000.
Under the hood, it looks like (gen/large-integer* {:min 1293840000000 :max 1924991999000}) . When I run this directly, I have the same issue.
Is there a way to generate dates that are more spread from 2011 to 2030?
2019:12:12 16:22:00 ghadi are you sure that it doesn't start near 2011, then move away progressively? @gariepyalex
2019:12:12 16:22:11 ghadi if you sample more values
2019:12:12 16:28:19 gariepyalex You are right. With more samples some are different. Still, most date are very close to the minimum value. This is expected behavior? If I want something else, I need to write a custom generator?
2019:12:12 16:30:17 ghadi This is expected.
2019:12:12 16:30:32 ghadi there's a control -- I don't have a link handy, but see the test.check wiki I think
2019:12:12 16:32:23 gfredericks @gariepyalex the expected behavior of large-integer is that many values are close to 0
2019:12:12 16:33:00 gfredericks So any date/time generator naively implemented with large-integer will have the sort of behavior you're describing
2019:12:12 16:33:53 gfredericks There's no datetime generator that doesn't privilege some span of time
2019:12:12 16:35:18 gariepyalex > From https://clojure.github.io/test.check/growth-and-shrinking.html
> gen/sample starts with very small sizes in the same way that the quick-check function does. This can be misleading to users who don’t expect that and take the first ten results from gen/sample to be representative of the distribution of a generator. Using gen/generate with an explicit size argument can be a better way of learning about the distribution of a generator.
Thanks @ghadi and @gfredericks!
2019:12:12 16:36:05 gfredericks :+1:
2019:12:13 20:32:44 kvlt Is it not possible to use with-redefs inside of defspec?
I have a relatively simple test that seems to suggest that it's not working:
The function I'm testing is here:
(deftest abc-test
(with-redefs [sut/gen-uri (constantly "")]
(is (= ""
(sut/send-email2 "String is ignored because of the redef")))))
(abc-test) => nil
2019:12:13 20:44:19 kvlt I see how it was poorly named for this example, sorry for that
2019:12:13 20:44:43 kvlt What I really want to mock is the http call to the service that sends the email. BUt wanted to keep things simple
2019:12:13 20:45:05 gfredericks Then try reversing the order of prop and with-redefs
2019:12:13 20:45:38 kvlt Oh I feel silly
2019:12:13 20:45:46 kvlt I didn't expect that to throw things off
2019:12:13 20:46:07 kvlt Thanky ou
2019:12:13 20:47:03 gfredericks The prop call is essentially creating a function that gets called later, at which point your with-redefs has already exited
2019:12:13 20:47:15 gfredericks Similar to if you put it around a defn
2019:12:13 20:47:42 kvlt That makes sense.
2019:12:13 20:47:56 kvlt Thank you again!
2019:12:13 21:40:19 alexmiller I don't know if you've looked at it, but clojure.spec.test.alpha/instrument has the ability to mock and stub
2019:12:14 11:32:48 kwrooijen Hey, I was wondering if there was a shorthand for creating fdefs , For example I have:
Which works, but it’s quite a lot noise (seeing that the fdef is larger than the function itself
I know I shouldn’t be comparing spec to type signatures, but that’s basically what I’m doing here. Something like this would be pretty useful for me:
2019:12:14 14:19:10 kwrooijen Thanks, these sort of do what I want. But they replace defn which I don’t like.
I couldn’t find anything so I ended up writing a quick macro:
2019:12:14 13:53:10 alexmiller Stay tuned for spec 2.... :)
2019:12:14 14:21:28 kwrooijen Looking forward to that!
2019:12:14 16:02:55 alexmiller Rich is working on the design for this right now and it’s starting to look pretty good
2019:12:15 14:14:56 valerauko I'd be interested to hear what your workflow (feedback loop?) is for developing the language. Are there any blogs about that?
2019:12:15 14:48:16 alexmiller I did a talk at Clojutre that talks about it some
2019:12:18 06:30:31 valerauko in your Nice and Accurate Counsel?
2019:12:14 16:23:41 kwrooijen That would be such an amazing improvement for me. I’d honestly want to spec every function at my day job (I guess that’s the static typing mindset in my head talking), but it’s just too verbose at the moment
2019:12:14 16:24:25 kwrooijen it’s also difficult to convince my colleagues 😄
2019:12:16 13:46:35 kwrooijen Wrote a small library which does what I need: https://github.com/kwrooijen/spec-signature
Hopefully spec2 will make this obsolete, but for now this works 🙂
2019:12:17 16:24:44 vemv > Hopefully spec2 will make this obsolete
I also hope the current alternative landscape goes mostly obsolete - obviously not good to have N competing solutions, or syntaxes. If browsing a given general-purpose library (e.g. a file unzipper, whatever), casual readers shouldn't struggle to understand its defns. It goes in the opposite direction of having a common language that we all speak :)
There are some interesting ideas around though, like ghostwheel's.
https://github.com/nedap/speced.def which I authored is quite minimalistic, in that regard. Although it has advanced features like spec -> type hint inference, which at this point would be painful to stop benefiting from
(at work we have essentially zero reflection warnings, other than those coming from external libs. Our lib makes this easy and meaningful)
2019:12:17 19:01:08 vemv The money quote being, I presume:
> I'm not gonna go code up this big thing, because you know, some of the things that you're talking about as being standard, they also have a bunch of known shortcomings
Fully agreed. I don't look forward to a big solution that solves everyone's problems
But I do advocate small-yet-full solutions that actually solve one specific problem.
Else the landscape you have is a Lisp Cursehttp://www.winestockwebdesign.com/Essays/Lisp_Curse.html .
There's actually a pretty strong analogy between Common Lispers doing OOP and Clojurists wrapping spec :)
2019:12:18 21:46:06 ag how can I ns-resolve symbol a spec sitting in a different namespace based on a given string?
2019:12:18 21:47:32 alexmiller can you more clearly state inputs and output?
2019:12:18 21:49:16 ag I want something like:
(ns-resolve
'my-specs
(symbol "foo"))
but for a spec. How can I resolve ::my-specs/foo when given two strings “my-specs” and “foo”
2019:12:18 21:49:57 alexmiller resolve to what?
2019:12:18 21:50:02 alexmiller a spec object?
2019:12:18 21:50:29 alexmiller a fq keyword?
2019:12:18 21:51:09 ag yeah, let’s say I want to validate a spec, but instead of spec I have its name as a string
2019:12:18 21:51:13 alexmiller my-specs is an alias only meaningful in the context of a namespace. is that the current namespace or some other one?
2019:12:18 21:54:40 ag so let’s say I have a bunch of fields that I extracted from e.g. SQL query, it’s a vector of ["foo" "bar"]. I have a namespace my-specs with two specs in it: (s/def ::foo string?) and (s/def ::bar boolean?)… now I want to validate data, or generate data, basically I need to “get” those specs
2019:12:18 21:55:03 ag all that programmatically
2019:12:18 21:55:54 alexmiller do you know that all of these specs are in my-specs?
2019:12:18 21:57:21 ag sure… let’s assume they are 100% there
2019:12:18 21:57:26 alexmiller if so (s/get-spec (keyword "my-specs" "foo")) should work?
2019:12:18 21:58:05 ag OMG… there’s a literally a function called get-spec… LOL
2019:12:18 21:58:19 ag Thank you Alex… sorry for being so lame at explaining
2019:12:18 21:58:34 alexmiller np, just trying to impedance match :)
2019:12:18 21:59:00 ag however… what if I want to check if the spec is indeed there? 🙂
2019:12:18 21:59:15 alexmiller well if you get nil, it's not there :)
2019:12:18 21:59:28 alexmiller you can also call (s/registry) to just get the full map too
2019:12:18 21:59:44 ag ah… okay… awesome… exactly what I needed. Thanks again!
2019:12:18 22:01:28 seancorfield (the dynamic require/resolve is so the code can run on Clojure 1.8)
2019:12:18 22:01:35 ag Oh… that’s nice. I’ll give a gander as well. Thank you Sean!
2019:12:18 22:48:52 ag hey friends… another dynamic lookup related question:
if I have a (s/keys) spec and string representation of one of the fields how do I get-spec of that?
e.g: I have:
::foo-spec/foo defined as:
(s/def ::foo
(s/keys :req-un [::bar-spec/bar]))
and in bar-spec ns I have:
(s/def ::bar (s/keys :req-un [::name]))
and I have strings “foo-spec”, “foo” and “bar.name” what’s the best way to get-spec of ::bar/name ?
How can I make it work for multiple levels of nesting?
2019:12:18 22:53:50 ag meaning I can get-spec/foo but now I need to “analyze” its :bar field, I have no idea where it sits, is there a way to find it out?
2019:12:18 22:53:59 ag dynamically?
2019:12:18 22:54:16 alexmiller there are multiple independent questions here
2019:12:18 22:55:20 alexmiller re understanding the structure of a keys spec, you can use s/form to get a fully resolved form representing the spec (so you'd see (clojure.spec.alpha/keys :req-un [:bar-spec/name]) )
2019:12:18 22:57:37 alexmiller finding the subspecs inside that spec is a matter of fishing for it (made somewhat complicated by the and / or support in s/keys). There are a variety of ways to tackle that, all a little meh, that's something we're looking at having better answers for in spec 2.
2019:12:18 22:58:02 alexmiller if you go that path, you have only fully-qualified subspecs, so you can just pass them to s/get-spec
2019:12:18 22:58:55 alexmiller and re nesting, there is no such thing as ::bar/name in this case - you've just smooshed together two independent things there
2019:12:18 22:59:45 ag > you’ve just smooshed together two independent things there
ehmm… I’m just trying to illustrate that I needed nested lookup…
2019:12:18 23:01:32 alexmiller just saying that the specs themselves are not "nested" and have no naming relationship. spec references are always via fully qualified keywords (even if you use autoresolved names to specify them)
2019:12:18 23:02:57 ag yeah, it seems this isn’t as simple as I thought it would be… so basically I wanted to figure out specs for a vector of strings like:
2019:12:18 23:11:33 ag and they all may be sitting in different namespaces
2019:12:18 23:13:15 ag now without knowing the ns of city or contact but knowing where account resides and given a string “account.contact.address.city” I want to get-spec of ::address/city
2019:12:18 23:20:56 ag oh.. well, it gets a tad bit crazier when some fields are s/nilable
2019:12:18 23:30:42 alexmiller the whole point of having namespaces is being able to differentiate things. seems like you're working really hard to rebuild what that gives you.
2019:12:18 23:31:28 alexmiller the whole idea behind spec is that attributes are the main source of semantics and maps are weaker aggregations of those
2019:12:18 23:31:43 alexmiller you are working significantly at odds with that idea
2019:12:18 23:33:15 ag So if you want a spec that describes a relational data, how would you do it?
2019:12:18 23:35:21 alexmiller generically, relational data is sets of maps
2019:12:18 23:37:18 seancorfield @ag it would probably help you to remember to use explicit qualifiers (on keywords) in your spec definition and stop using :: at least until you get your head around this...
2019:12:18 23:37:29 ag so the above snippet with account -> contact -> address is okay?
2019:12:18 23:39:29 seancorfield That also gets you close to the "relational" model if you look at something like next.jdbc querying a database since it will use qualified keywords for column names, based on the table they are in...
2019:12:18 23:40:51 seancorfield (although there you have to reify the primary keys so you'd most likely have :account/id:account/contact_id and the latter would be the FK into the :contact table and its :contact/id column etc)
2019:12:18 23:43:15 seancorfield If you did the join with next.jdbc/execute! you'd end up with a flat map containing
2019:12:19 18:06:56 joefromct if i just type into a repl a fn and a fdef, is there any reason it wouldn't be enforced for a given project?
I have an older project I'm jumping into and can't understand why my fdef's aren't throwing errors... I'm sure check-asserts is true although i didn't think that would even matter.
2019:12:19 18:22:20 alexmiller fdef on macro or function?
2019:12:19 18:22:32 alexmiller if function, you have to instrument it for it to do anything
2019:12:19 18:24:10 joefromct ok, so runtime checks have to be inside the form... either with an assertion, :pre check, or conform.
I guess i missed that in the gude... I think i'll plug in an assertion so i can turn it off when i'm done with my test/dev/exploration on trying to figure out these data structures.
2019:12:19 18:24:33 joefromct ^ fdef on a function
2019:12:19 18:51:49 alexmiller note that there is a s/assert
2019:12:19 18:54:01 joefromct thanks as always for your help Alex
2019:12:27 17:18:28 Alexis Vincent Does s/def do some macro magic?
2019:12:27 17:18:42 Alexis Vincent This is on the latest spec2 btw
2019:12:27 17:22:09 seancorfield s/def converts the symbolic form to an actual Spec expression and then registers the spec name. So in (s/def ::id s-id) you do not have a symbolic form of a spec. It probably should give an error.
2019:12:27 17:23:49 Alexis Vincent thanks! so would I then rather do (s/def ::id (s/spec s-id)) or something similar?
2019:12:27 17:33:17 alexmiller Use s/register instead of s/def
2019:12:27 17:33:31 Alexis Vincent awesome! thanks!
2019:12:27 17:33:52 alexmiller s/register is a function and takes a spec object
2019:12:27 17:36:23 Alexis Vincent If I have an s/cat expression but would like to generate/validate vectors not arbitry seq’s, how would I express it?
2019:12:27 17:42:07 alexmiller If you’re on spec 2, you can use the new s/catv
2019:12:27 17:43:06 alexmiller It’s difficult to do easily on spec 1
2019:12:27 17:43:39 Alexis Vincent Perfect, thanks 🙂
2019:12:27 17:55:25 seancorfield @alexmiller Should that (s/def ::id s-id) give an error? Or is it valid but just does something unexpected?
2019:12:27 18:05:51 alexmiller Prob error but I’ll look at it next week
2019:12:27 18:20:03 Alexis Vincent This is probably the same issue as before, but, are these not equiv in spec 2?
The first one works nicely, but the second one blows up when generating maps from schemas that contain these keys, but not when generating them directly
2019:12:27 18:33:20 seancorfield @mail024 That should probably work but there are bugs in Spec 2 that affect aliasing of specs like that, I believe.
2019:12:27 20:17:04 Alexis Vincent @seancorfield ok thanks!
2019:12:27 21:25:54 Alexis Vincent If I have a schema with a qualified key I’m trying to override inline, it doesnt seem to want to use my override. It just picks the spec from the registry
Since event/type already exists in the registry and since (I assume) it is qualified. It picks the registry definition not the one i’m declaring inline
2019:12:27 21:31:33 seancorfield You can only use the inline specs for unqualified keys.
2019:12:27 21:32:11 seancorfield Qualified keys are intended to be globally unique so overriding them doesn't make sense in that context (since their meaning is supposed to be globally fixed).
2019:12:27 21:33:27 Alexis Vincent fine. Although in this case it’s the generator I want to override. Would s/with-gen work for this?
2019:12:27 21:33:48 seancorfield Yes, possibly. Depending on exactly what you are trying to do.
2019:12:27 21:34:43 Alexis Vincent I have a set of valid event types. Defined as a spec set. I want to define a particular message. Which is some schema, alongside a specific item from the set
2019:12:27 21:34:47 seancorfield If you want the keys in a hash map to "depend on" the value of a particular key, it sounds like you want multi-specs
2019:12:27 21:35:13 seancorfield If you define this as a multi-spec, it should generate correctly automatically.
2019:12:27 21:35:56 Alexis Vincent Thanks 🙂 Ill check them out.
2019:12:27 21:51:59 Alexis Vincent Perfect thanks. Working well
2020:12:29 07:17:44 ho0man Hi, everyone is using spec/assert in production a good practice ?
I want to produce detailed exceptions like spec/assert does for mismatches
but do not want to allow some erroneous code to be able to disable the asserts by calling spec/check-asserts .
Am I getting something wrong ?
Thanks a alot
2020:12:29 08:25:59 vlaaad I use spec/assert in dev and custom function which is a mostly a valid? + throw in production
2020:12:29 11:01:33 ho0man Hi @vlaaad but this way I won't have descriptive exceptions like those thrown by spec/assert. Am I right ?
2020:12:29 11:13:11 vlaaad @ho0man that's configurable by you. when I throw, I store spec/explain-data as error data and use spec/explain-str as error message — all the bits are there
2020:12:29 11:18:54 ho0man Thanks a lot @vlaaad, I didn’t know about them
Thanks
2020:12:29 14:34:42 unbalanced Curious if there's a way in spec to do the following:
or for that matter is it even possible to specify multiples instead of :count in coll-of?
2020:12:29 14:36:27 unbalanced (need 3 coordinates per vertex ID, ideally, is what I'm going for)
2020:12:29 15:38:10 codonnell @goomba Any predicate function is a spec, so you could use s/and to combine a spec like you typed above (but without the bit about counts) with a predicate function checking that the counts are valid. If you want generation to work, you'll likely want to provide a custom generator that does something like generate a map with the proper types and then truncate things to make the counts work out.
Is there a way to decompose/query other kinds of specs? For example, I would like to do something like the following to get the spec for the individual fn argument named x.
(:x (:args my-fn-spec))
;; or maybe
(first (:args my-fn-spec))
2020:01:02 03:20:20 seancorfield I think the TL;DR is not easily with Spec 1 @erp12 but Spec 2 offers more facilities for taking specs apart and programmatically building them.
2020:01:02 03:25:21 seancorfield In Spec 1, you can get the form of a Spec and break it apart, but it isn't easy to turn that back into Spec objects that you can use tho'...
2020:01:02 03:42:39 Eddie @seancorfield Good to know, thank you! Based on that, would you agree that currently the best option would be to keep the sub-specs in a map and use some utility functions to "materialize" real specs from them. Just spitballin' here but something like ...
2020:01:02 03:43:37 Eddie Or do you know of any other pattens followed by the community for stuff like this?
2020:01:02 03:59:37 seancorfield I don't really understand what problem you are trying to solve here... It doesn't look like the sort of thing I've seen anyone trying to do with Spec.
2020:01:02 04:00:20 seancorfield Have you looked at Spec 2? That's much more amenable to programmatic manipulation of specs...
2020:01:02 20:10:16 rafael Hi. I'm struggling with generating data from a simple (s/schema) use case.
2020:01:02 20:10:57 rafael The call to (gen/sample) throws a No implementation of method: :conform* of protocol: #'clojure.alpha.spec.protocols/Spec found for class: clojure.lang.Keyword exception.
2020:01:02 20:12:40 rafael While a straightforward translation to s/keys seems to work fine:
I'm probably getting something wrong in my schema definitions, but I can't figure out the problem.
2020:01:02 20:32:02 alexmiller code looks fine, prob just a bug
2020:01:02 20:32:15 alexmiller in spec
2020:01:02 20:38:02 rafael Cool, I'll stick with the (s/keys ..) version for a while.
2020:01:02 20:43:12 rafael Changing the :bar definition to
(s/register ::bar (s/resolve-spec ::baz))
appears to work around the issue.
2020:01:02 20:48:06 alexmiller that makes sense - you're basically copying the spec object rather than relying on resolving through the alias
2020:01:02 20:48:23 alexmiller I have a pretty good hunch on where that bug is
2020:01:02 20:57:18 rafael Awesome, thanks!
2020:01:03 03:42:44 Eddie @seancorfield I spent some time today with spec 2. It seems like it's design goals are exactly what I need. I attempted to migrate and it was pretty rough (not complaining, I totally understand that it is still in development and also a new major version so I expected many breaking changes). I am not sure if we will be jumping on board with spec 2 yet, but thank you for the pointer!
2020:01:03 03:46:18 Eddie There are a few things that we are doing in spec 1 that I cannot figure out how to recreate in spec 2, even though it smells like it should be possible.
For example, I would like to implement a custom generator that simply pulls a random element from a set.
I can get this to work if the set satisfies (every? constant-val? ...) but otherwise, I cannot.
2020:01:03 03:53:28 alexmiller are you using a set spec?
2020:01:03 04:00:20 alexmiller one of the caveats you might be seeing is that set specs are constrained to constant values only which is part of the new symbolic spec direction
2020:01:03 04:01:52 alexmiller there are a couple of options but you can just use the built-in generators directly if you have something else
2020:01:03 17:08:11 Eddie Thank you! gen/elements is exactly what I was looking for.
However I am running into some strange behavior on the with-gen .
user=> (s/with-gen (s/spec int?)
(fn [] (gen/elements [-1 0 1])))
Execution error (IllegalArgumentException) at clojure.alpha.spec.protocols/eval190$fn$G (protocols.clj:11).
No implementation of method: :with-gen* of protocol: #'clojure.alpha.spec.protocols/Spec found for class: clojure.lang.PersistentList
user=> (s/exercise (s/with-gen int? (fn [] (gen/elements [-1 0 1])))
10)
([-1 -1] [0 0] [1 1] [-1 -1] [-1 -1] [-1 -1] [0 0] [1 1] [-1 -1] [1 1])
Neither of those seem right to me. Am I missing a change from Spec 1?
2020:01:03 17:14:50 alexmiller yeah, seems fishy. I'd expect the first one to work
I'm unsure what parts I could be adding to babashka as a built-in. Are there any assurances what parts of the API will be unchanged, and what the future namespace of spec will be ?
2020:01:07 06:15:27 holyjak Why not spec 2? Though not finished, I guess it bears completion (just the last 10% that use 90% of the time :rolling_on_the_floor_laughing:)...
2020:01:06 15:52:56 borkdude If I have some more assurances, I could adopt parts of this lib as built-ins which would make it much faster.
2020:01:06 15:55:47 alexmiller I can assure you that the future namespace will be different :) The current spec 2 is clojure.alpha.spec, clojure.alpha.spec.gen, etc. We're expecting this to eventually be clojure.spec, clojure.spec.gen
2020:01:06 15:56:55 alexmiller most of the existing spec forms will probably be the same, with some additions and some fine details changing around a few things (and deprecation/removal of s/keys)
2020:01:06 15:57:51 alexmiller most of the spec operations will be the same although we have added arities in all of the conforming ops (valid?, conform, explain, etc) to support spec checking modes
2020:01:06 16:04:06 borkdude thanks
2020:01:06 17:51:50 seancorfield @alexmiller Any sense yet of whether there will a migration path/tooling at launch of clojure.spec?
2020:01:06 17:54:08 alexmiller as long as there are no follow-up questions, yes :)
2020:01:06 23:27:19 lgessler hi, I want to use s/fdef with protocol functions. In one namespace, I'm calling fdef and defining and implementing a protocol, but the protocol's methods appear to be unaffected by fdef. If I understand [this issue](https://clojure.atlassian.net/browse/CLJ-2109) right, this is expected behavior. Is that all right, and if so, is there a recommended workaround for speccing protocol functions?
2020:01:06 23:33:46 alexmiller That’s correct - fn specs don’t work with protocol methods. Only workaround I’d suggest is wrapping the call to the protocol function in a method (which I often do as a matter of practice anyways)
2020:01:07 18:39:28 telekid I’ve got a spec(ish) design question that I’ve been thinking about for a while. I finally decided to throw it in a gist: https://gist.github.com/telekid/0f276bfa3b3f0d395a7a71158adbb35d
I’m curious to know how other people approach this problem?
2020:01:07 20:14:49 vemv In a given (ns models.geography.continent), I tend to solve this with:
i.e. I don't use :: syntax for these cases, and I don't create additional Clojure namespaces (files).
I call it the "synthetic ns pattern"
It's a bit verbose, and losing the 1:1 mapping between cljs namespaces and spec namespaces isn't ideal either.
But it's fairly occasional so I've been happy to use these for a couple years
2020:01:07 20:15:55 vemv btw, not sure if this solution is already described in your gist. sometimes I'm impatient 🙃
2020:01:07 20:48:57 telekid yeah, I like that
2020:01:07 20:49:26 telekid I’ve experimented with that a bit in a few places (but I forgot about it when I was writing the gist - thanks for bringing it up)
2020:01:07 20:56:32 vemv ✌️!
I realised, a more accurate nickname would be "synthetic sub-ns pattern".
so, while it's not a 1:1 mapping, one is still reasonably close to that ideal
2020:01:07 18:51:19 Eddie @jake142 If EDIT: Gist has been updated. Disregard! :)
2020:01:07 18:51:19 Eddie @jake142 If EDIT: Gist has been updated. Disregard! :)
2020:01:07 18:52:17 telekid Oh, I just realized that I made a typo
2020:01:07 18:52:22 telekid facepalm
2020:01:07 18:52:24 telekid one moment
2020:01:07 18:52:52 telekid L20 how has ::animals/lions
2020:01:07 18:54:37 Eddie I see. Thanks for the clarification.
2020:01:07 19:05:23 telekid Sorry for the confusion!
2020:01:07 21:37:10 kenny Why were multi-specs implemented using multi-methods? I have never found the dynamism of multimethods useful in defining multi-specs. In fact, I think I'd prefer all of the multi-spec types to be declared statically.
2020:01:07 21:44:17 ghadi @kenny so that they're open to extension
2020:01:07 21:44:56 ghadi in fact I can't think of anything in clojure that's closed
2020:01:07 21:45:11 kenny Right. At least interally, I've never found that useful. All type->keys are declared upfront. Multimethods make it hard to determine the input data.
2020:01:07 21:45:55 ghadi > All type->keys are declared upfront.
?
2020:01:07 21:46:14 kenny Dispatch functions
2020:01:07 21:47:44 ghadi I'm sorry I still don't understand
2020:01:07 21:49:26 kenny For a particular entity, I know it has types A, B, C. It can never and should never be extended except directly in the definition.
2020:01:07 21:49:52 kenny All the variants of an entity are known upfront.
2020:01:07 21:50:09 ghadi ok that's your use case
2020:01:07 21:50:12 Joe Lane You might not be the one defining the multispecs, the users of your library may be.
Don't know if this is where it'll land but it's a start.
2020:01:07 21:54:22 ghadi wouldn't it be amazing if you can write that macro today?
2020:01:07 21:54:32 ghadi 😄
2020:01:08 00:19:40 kenny Has anyone written a version of spec2 select that works with spec1? Deciding to have keys required or optional for all use cases is so painful 😣
2020:01:08 00:45:20 seancorfield @kenny Not sure what you mean by "works with spec1"? The two libraries use different registries so you cannot combine them (at least, not easily)
2020:01:08 00:45:47 kenny I mean a macro for spec1 that imitates spec2's select.
2020:01:08 00:47:02 kenny When using nested maps, spec1 doesn't let you change which keys in the nested map are required.
2020:01:08 00:47:05 seancorfield That's exactly why Spec2 is coming -- that required/optional thing in Spec1 is painful 🙂
2020:01:08 00:47:58 seancorfield In Spec1 that is complected and baked into how s/keys works. In Spec2 the set of keys and the requiredness are decomplected.
2020:01:13 16:31:06 avi You know, it just occurred to me — this is going to make it hard to create an interop schema using spec — in other words, to fully describe a data structure that a system expects/requires, including the keys that are required in each map…
I suppose we might need to write some new plumbing to create a schema based on some spec2 specs, with additional annotation, expectations, etc. 🤔
2020:01:13 16:49:44 seancorfield I'm not sure I follow. Spec 2 still let's you specify which keys are required, including nested keys. It just teases the two concepts apart.
2020:01:13 17:20:01 avi Hmm for some reason I was thinking that the new schema function was designed solely for the case of defining function specs… but I haven’t looked super closely 😬
2020:01:13 17:20:42 avi is it schema? or maybe I’m thinking about select — I should just review the docs 🙄 sorry
2020:01:13 17:43:47 seancorfield schema defines the possible set of keys. select defines the required subset of a schema.
2020:01:13 17:44:29 seancorfield Hence, decomplecting specifying requiredness from specifying possible 🙂
2020:01:13 17:44:56 avi Sounds pretty exciting, honestly!
2020:01:13 17:45:46 avi I just did a walkthrough of a pretty involved schema using spec1 with some members of my team who aren’t familiar with it, and they asked a bunch of questions that I answered with “not right now, but that’s coming in spec2”
2020:01:13 18:10:39 seancorfield Yup, I'm looking forward to Spec2 becoming non-alpha and the "standard" way to work with Spec.
2020:01:13 18:12:22 seancorfield For a while, I had a branch at work tracking Spec2 but with the number of bugs and changes as it has evolved, it was proving a bit much trying to keep our codebase current on it... so I gave up a while back and we'll take another look once it is stable. I doubt we'll migrate, but we'll probably start using Spec2 for new code (and may convert old code over as we maintain it over time).
2020:01:08 00:48:27 kenny Ya. Was hoping for a workaround for now. It seems potentially possible to get it to work.
2020:01:08 00:49:49 seancorfield I don't know if s/merge will let you override an all-optional s/keys with a subset of keys that are now marked required. Doesn't help you much with nesting I suspect.
2020:01:08 00:50:31 seancorfield You could just start using Spec2 I suppose 🙂 But not for any production work as it's still changing a lot.
2020:01:08 00:53:25 kenny It's gotta be production worthy unfortunately
2020:01:10 01:52:12 telekid We ran into a question about how to spec multimethods while maintaining their open nature. There wasn’t much good documentation on this topic floating around the internet, so I went down the rabbit hole a bit and came back up with this:
https://gist.github.com/telekid/f2e588718dbdfe48306d64e5388bdc15
2020:01:14 11:57:42 mping is there a way to spec a map according to its key? example: if key is a number, value should be a number, if key is something else then value should be a string
2020:01:14 13:34:33 alexmiller Yes, although it’s a little complicated
2020:01:14 13:36:02 alexmiller The structure is to spec it as a s/coll-of s/tuples, where each tuple is an s/or of whatever key/value is allowed
2020:01:14 13:36:38 alexmiller The coll-of should also have an :into {}
and like that shows ::bar-ret returns ::foo.
My question is whether this is by design or perhaps a bug, and is it too hacky using something like (s/def ::bar-ret (s/spec ::foo)) to get this to work?
2020:01:14 17:41:33 alexmiller this is a known bug (there's a ticket for it) and we plan to fix in spec 2
2020:01:14 17:41:42 alexmiller your workaround is fine
2020:01:14 17:41:50 colinkahn Awesome, thanks!
2020:01:15 18:54:48 nullptr i have a dumb spec question that unfortunately is in the "difficult to google" category - say i had a sequence with the following structure:
[1 "a" 2 "b" "c" 0 3 "d" "e" "f"]
the pattern here is a number which indicates how many following items there are directly in the sequence -- i adjusted spacing above to hopefully make that clearer
this is of crouse trivial to express if the following items are in another sequence, but scanning the docs i can't see a way to spec the flat version of this -- note that this isn't motivated by any important use case, more just part of me trying to learn spec so feel free to treat this as pure curiosity
2020:01:15 19:16:19 alexmiller regex specs combine to spec one flat sequence
2020:01:15 19:20:05 alexmiller so you could do something like (s/* (s/& (s/cat :c nat-int? :s (s/* string?)) #(= (:c %) (count (:s %)))))
2020:01:15 19:23:18 nullptr thanks! that is extremely straightforward -- i feel like i had something similar but surely missed something basic and made a wrong assumption. will compare and learn -- thanks again.
2020:01:15 19:24:28 alexmiller it's very important to use s/&, not s/and there
2020:01:15 19:24:49 alexmiller s/and is not a regex spec and will expect a new nested collection boundary
2020:01:15 19:24:52 nullptr yeah i had the & but i think i had a surrounding cat there that was throwing it all off
2020:01:17 14:33:47 yuhan Is there any way of relating a neighboring spec's value into the :count option of s/every ?
2020:01:17 14:34:48 yuhan eg. I have a map spec with keys :width and :things
2020:01:17 14:35:21 yuhan and I want to ensure that things has width number of elements
2020:01:17 14:35:47 alexmiller you need to s/and with a predicate that can do that at the containing level
2020:01:17 14:36:52 yuhan okay, I have something like this
2020:01:17 14:37:45 yuhan so I'll have to write a custom generator if I want to gen examples of the spec?
2020:01:17 15:01:54 alexmiller yes
2020:01:17 15:03:40 alexmiller this is true of any data with internal constraints like this. the general strategy is to use gen the width first, then use a combination of fmap and bind to generate the slots and assemble into the map
2020:01:17 15:04:13 alexmiller you might also ask what the width is buying you in this data structure when it is the same as the count of the slots
2020:01:17 15:04:52 alexmiller (and as an aside, I find that pushing on data that is hard to spec often improves the shape of the data and the code that uses it)
2020:01:17 15:15:47 yuhan My idea was to have a degree of "memoization" or redundancy baked into the data structure - in this case slots may be a lazy sequence that is generated on demand, so I don't want to keep calling count on it unnecessarily
2020:01:17 15:16:39 yuhan of course this introduces the risk of width getting out of sync with the data, which is why I'm using spec to assert the constraints during dev time
2020:01:17 15:18:51 alexmiller fair enough
2020:01:17 15:19:28 yuhan there are a few other "derived" keys similar to this that I'm storing in the same data structure, just wondering if this isn't considered a code smell?
2020:01:17 15:22:55 alexmiller it really depends
2020:01:17 15:31:10 yuhan as with so many things 🙂
2020:01:17 15:33:12 yuhan Here's my best attempt at that generator:
2020:01:17 22:32:44 zane Let’s say I have a Datomic query and I want to find all the variables defined in that query. Would spec be a good tool to bring to bear on that problem?
2020:01:17 22:50:52 seancorfield @zane I'm not saying it couldn't be done but I certainly wouldn't expect to use Spec for such a problem.
2020:01:17 22:51:43 seancorfield Even if you wrote a complete Spec of Datomic queries, if you s/conform it so it "identifies" the ? variables, you'd still have to walk the resulting data structure to extract them all -- you might just as well walk the original Datomic query.
2020:01:17 23:15:24 zane Yeah, I had imagined doing something clever with s/conform and clojure.walk.
2020:01:17 23:15:29 zane But I see what you mean.
2020:01:17 22:57:32 alexmiller I believe there are specs of Datomic's query syntax out there btw, don't have any links handy (but I agree that I probably wouldn't pick that as the first approach)
2020:01:17 23:15:08 zane Good to know!
2020:01:17 23:20:44 zane Is it fair to say that conform is not really suited to these kinds of problems in the general case?
2020:01:17 23:22:23 seancorfield conform is not designed for parsing or general transformation, if that's what you're asking?
2020:01:17 23:23:06 seancorfield conform is intended to "tag" the resulting data with "how" the data conformed to the Spec, so downstream code can behave accordingly.
2020:01:17 23:36:07 zane Isn’t that kind of tagging exactly what I’m after in this case? I’m not sure I’m understanding the details here.
2020:01:17 23:56:12 seancorfield You'd still have to walk the result to find the tagged values tho
2020:01:18 00:03:50 zane True.
2020:01:18 00:04:29 zane I guess I’m still struggling with which kinds of “parsing” are appropriate to do with spec and which aren’t.
2020:01:18 00:05:03 zane Differentiating between different shapes of data seems like something you could achieve with it via s/or and s/conform.
2020:01:18 00:05:19 zane But you could also do that kind of thing with match, or with plain old Clojure.
2020:01:17 23:24:10 seancorfield That's not to say that some people don't (ab)use Spec to do some amount of parsing and transformation... cough ...but that's not what it was designed for: coercion is somewhat of a "side-effect" of conforming, you might say 🙂
2020:01:17 23:34:37 zane That makes sense.
2020:01:17 23:34:56 zane I find myself wanting something like Instaparse, but for EDN rather than strings, often.
2020:01:18 14:45:22 dharrigan Is there an example of how I might use spec to ensure that a field is in OffsetDateTime format?
2020:01:18 15:20:06 dharrigan Sorta gets me there, the field is a string, so I think I'll have to do some extra magic
2020:01:18 15:33:58 alexmiller probably need to use a parser then
2020:01:18 15:34:12 alexmiller which is part of the java time library
2020:01:18 15:34:21 dharrigan kk
2020:01:18 15:34:28 dharrigan thanks for the pointer 🙂
2020:01:19 13:01:00 Philipp Siegmantel Hello Everybody, what test runner/library do you use with clojure.spec.test.alpha/check? I found https://gist.github.com/jmglov/30571ede32d34208d77bebe51bb64f29 for integration with clojure.test but the errors it prints aren't verry informative. Is there something better?
2020:01:19 16:07:13 Philipp Siegmantel Do you have a link?
2020:01:19 17:58:42 respatialized has anyone written an unofficial spec for EDN? I'm looking for a programmatic way of distinguishing between Clojure forms and plain EDN forms.
2020:01:19 18:53:23 respatialized yeah, I figured I'd probably have to create one myself on the basis of the format's description that suits my purposes, which also serves as a useful exercise for improving my understanding of spec itself.
2020:01:19 21:57:54 andy.fingerhut Clojure core has an implementation of reading arbitrary Clojure forms, via clojure.core/read, and the EDN subset, via clojure.edn/read
2020:01:19 21:58:26 andy.fingerhut Those implementations in Clojure itself are written in Java. The tools.reader contrib library also has implementations written in Clojure.
2020:01:21 01:14:27 Vincent Cantin In spec2, is it possible to use s/conform on a vector parsed via a sequence matcher like vcat and then getting back a vector via s/unform?
2020:01:21 01:16:01 Vincent Cantin From the documentation, it does not look like it would work.
2020:01:21 02:55:31 alexmiller Yes, it works now
2020:01:21 02:56:13 alexmiller With s/catv
2020:01:21 02:59:45 alexmiller What documentation are you referring to?
2020:01:21 13:09:38 Vincent Cantin The example about s/vcat may potentially confuse people w.r.t. s/catv .
2020:01:21 13:24:34 alexmiller Thx, I’ll update that
2020:01:21 15:32:20 Filipe Silva heya
2020:01:21 15:32:34 Filipe Silva are there any breaking changes one should be aware of between 0.1x and 0.2.x for spec?
2020:01:21 15:33:09 Filipe Silva I'm updating a project's cljs 1.10.520 to `1.10.597` , and my production build now fails when loading something spec related
2020:01:21 15:34:13 Filipe Silva doesn't seem to affect the normal build though
2020:01:21 16:49:23 G Have folks ever used spec to define a transformation between namespaced, clojure maps to another format?
My use case is creating maps in clojure but transforming them to JSON, and I don't want for there to be coupling between my clojure key names (ie. namespaced keywords) and my JSON maps (which may even contain keys that aren't representable as clojure keywords)
2020:01:21 21:17:19 rapskalian @U3G51H7NY you can have a look at this little lib I wrote a while ago for a similar use case
https://github.com/cjsauer/disqualified
2020:01:21 21:18:05 rapskalian Might not be exactly what you’re needing but could serve as a possible reference
2020:01:21 21:18:20 rapskalian It’s less than 50 lines of code
2020:01:21 21:23:01 G it's not exactly what I'm looking for right now (for instance, my un-namespace keys have conflicting names, although the full paths are distinct), but this is definitely useful! I've had to implement something very similar for another project, so will keep this in mind in the future 🙂
2020:01:21 16:53:30 G I was hoping to have the resulting key name as part of the spec so as to keep all of the relevant information about that attribute in one place, but perhaps this is best done outside of spec?
2020:01:21 16:55:10 ghadi you can store a lookup from qualifiedkey -> key in a sibling database, doesn't have to be in the spec registry
2020:01:22 16:31:42 kenny I really like how succinct spec-tools data-specs are. It takes 17 lines worth of specs for a map that has a key that has a collection of maps down to 3 lines. Will Spec2 be able to do this as well? The syntax described here https://github.com/clojure/spec-alpha2/wiki/Schema-and-select#unqualified-keys looks quite similar. The description makes it sound like that syntax will only be available for unqualified keys though.
2020:01:22 16:32:14 alexmiller That’s correct
2020:01:22 16:32:45 kenny Why not support qualified keys with that syntax?
2020:01:22 16:46:13 kenny Well, that I guess that applies for key & value in that ::order spec. Nothing is reusable.
2020:01:22 16:46:17 alexmiller well, you are in the schema, but not in the sub levels. as unqualified names, they often are ambiguous
2020:01:22 16:47:27 alexmiller you still can do like you did in spec 1 (relate unqualified keys to qualified specs) but it's not constrained to the unqualified name match and it's optional
2020:01:22 16:51:05 kenny To be clear, the issue with something like this
but with every key qualified instead of unqualified.
2020:01:22 16:53:22 alexmiller this now occupies the "map" slot in the language of specs, which may have other uses
2020:01:22 16:55:29 alexmiller certainly, this goes against the grain of naming and registering the specifications. names have multiple utility in spec - they are used for spec reuse, but also in things like explain reporting, and I don't remember what else
2020:01:22 16:56:33 alexmiller none of this rules out the possibility of adding more support for concise definition of multiple specs in tandem
2020:01:22 16:56:54 alexmiller or even extension to support this in the future, but it's not our top concern
2020:01:22 16:57:46 alexmiller and if we were going to do so, it would be driven by a top level problem, not just "it's a thing we can do".
2020:01:22 16:58:14 kenny Understood.
I don't think it necessarily goes "against the grain of naming and registering".
2020:01:22 16:40:29 alexmiller 0.1.x is Clojure 1.9 era and 0.2.x is Clojure 1.10 era
2020:01:22 16:41:08 alexmiller I don't think there were any breaking changes between those
2020:01:22 16:42:33 Filipe Silva yes that is what I meant
2020:01:22 16:43:30 Filipe Silva trying to update from cljs 1.10.520 to `1.10.597` causes my app to have a runtime error for optimized builds on what seems to be something spec related, so I thought there might be a breaking change
2020:01:22 16:43:59 alexmiller what are 1.10.520 and 1.10.597 applicable to?
2020:01:22 16:44:07 alexmiller oh cljs, sorry that got lost
2020:01:22 16:44:16 alexmiller spec.alpha is CLJ only
2020:01:22 16:44:50 alexmiller the cljs spec impl is inside ClojureScript so the stuff above does not directly correlate
2020:01:22 16:45:33 Filipe Silva ah I see... so maybe I should ask about cljs.spec.alpha in #clojurescript instead
2020:01:22 16:47:44 alexmiller yeah
2020:01:22 16:47:50 alexmiller I don't know anything on that
2020:01:22 16:48:05 Filipe Silva coolio, thanks for the direction!
2020:01:24 17:57:07 lilactown for direction/discussion w.r.t. cljs.spec.alpha, should I lobby that here or in #cljs-dev?
2020:01:24 18:11:25 alexmiller cljs-dev
2020:01:24 17:58:12 lilactown I’m interested in exploring (and advocating for support for) the capability in CLJS to remove specs when building for deployment
2020:01:24 18:05:54 Matti Uusitalo Put them in a separate source folder which isn't part of the production build?
2020:01:24 18:09:09 lilactown they need to show up in the namespace dependency graph of the application in order to be included at dev time and be rebuilt when the implementations change
2020:01:24 18:09:44 lilactown I could manually write a namespace that includes all specs I use in my app, and add that as a preload. but that is tedious
2020:01:26 12:11:00 vemv How about a my.spec/def macro which expands to sth like when js/goog.asserts.ENABLE_ASSERTS (spec/def ...)?
2020:01:27 21:19:00 favila is there any advice for speccing containers, e.g. atoms or delays?
2020:01:28 16:38:39 vemv Say you have a check! function. I normally hook up the :validator to it, but only on *assert*
2020:01:28 17:05:22 favila I went with an fspec instead
2020:01:28 17:05:59 favila Most ref types have validators, I forgot about those. you could use s/assert in there
2020:01:28 17:06:20 favila doesn’t help with generation, but there’s manual intervention for generation all the time anyway
2020:01:28 17:06:32 favila delays however don’t have validators
2020:01:28 17:18:36 vemv > doesn’t help with generation,
yeah it's a tradeoff :)
> delays however don’t have validators
yup, as you may know though a custom IDeref impl can be quite thin
2020:01:27 21:19:25 favila I’d like to say “takes a delay that returns a thing satisfying some predicate when derefed”
2020:01:27 21:42:32 seancorfield @favila There's no way to do that. You can spec around the code that processes what's inside the container, but you can't spec the container itself.
2020:01:27 21:44:08 lilactown atoms do have validator functions that you can pass in on instantiation I think?
2020:01:27 21:49:28 alexmiller they do
2020:01:27 21:49:47 alexmiller but carefully consider the costs there
2020:01:29 02:55:36 ag I need a spec for a map like {:a :b} where :b has to be a required key, but only when value of :a is not nil. Can someone help me with that?
there’s probably better way of doing this
Basically now :foo/bar is a spec for a map that must have :a and :b, keys, but when value of :a is equal to “bobo-included”, it must have :bobo, key, otherwise, :bobo key is optional
2020:01:29 03:37:14 seancorfield s/merge is intended for two key specs -- I don't think you should use it for a key spec and a predicate.
2020:01:29 03:37:52 seancorfield And it looks like you have s/and with just a single predicate there?
2020:01:29 03:38:29 seancorfield Now that you've restated the problem, it sounds like you want to look at multi-specs.
2020:01:29 03:39:08 ag ah… yeah. let me try digging in there
2020:01:29 03:39:08 seancorfield Those are intended to select different specs based on some function of a value, in your case, the value of the :a key.
2020:01:29 03:40:25 ag well, yeah, I guess I didn’t exactly follow my own requirement, but the approach you showed me probably would work for me. I’m gonna refresh my memory on multi-specs anyway.
2020:01:29 03:40:29 ag Thank you Sean!
2020:01:29 10:18:43 lambdam Hello,
I was playing a bit with spec 2 on an Advent of Code exercise and bumped into this case:
2020:01:29 13:38:57 alexmiller Dunno, probably a bug
2020:01:29 10:37:53 lambdam Also, is there a specific reason not to instrument :ret and :fn in fdef (even optionally) in spec 2?
I use https://github.com/jeaye/orchestra with spec 1 and find it very useful.
2020:01:29 15:22:26 zclj I get an exception when using a schema referring a spec with an indirection to another spec. It works as expected if I just use the spec. Am I doing anything wrong here or is it a bug?
(s/def ::thing string?)
(s/def ::other-thing ::thing)
(gen/sample (s/gen ::other-thing)) ;; => works as expected
(s/def ::foo (s/schema [::thing]))
(s/def ::bar (s/schema [::other-thing]))
(gen/sample (s/gen ::foo)) ;; => works as expected
(gen/sample (s/gen ::bar)) ;; => Exception below
No implementation of method: :conform* of protocol:
#'clojure.alpha.spec.protocols/Spec found for class:
clojure.lang.Keyword
2020:01:29 15:34:56 alexmiller this is a bug in spec 2, so won't work yet
2020:01:29 20:13:30 kenny Does spec2 let you write a spec for a qualified keyword such that its definition changes depending on the context? For example, if I were to spec Datomic's :db/id, it would only ever be a nat-int? when part of the result is from a pull query. When transacting to Datomic, :db/id could be a lookup ref (e.g., (s/tuple keyword? some?) ), a nat-int? , or a string?. One could spec :db/id using s/or but that means everything that takes a db id as an input needs to handle all of those cases, which does not always make sense.
2020:01:29 20:52:15 alexmiller in short, no
2020:01:29 20:52:58 alexmiller in long, the general recommendation is to try to give attributes specs that truly reflect the data
2020:01:29 20:53:31 alexmiller within certain contexts, you can add on additional specs that narrow the scope if needed
2020:01:29 21:00:18 kenny By this do you mean you can add additional predicates to the :db/id spec in different places?
2020:01:29 21:05:05 alexmiller yes, you could s/valid while s/and'ing in an additional narrower predicate for example
2020:01:29 21:05:41 kenny Oh cool. Is there an example of what that looks like?
2020:01:29 21:19:58 alexmiller just what I said?
2020:01:29 21:21:01 kenny Oh - that works if the :db/id is taken as an argument itself. I was imagining a function that takes a map (perhaps nested) that has :db/ids on the maps.
2020:01:29 21:21:33 kenny Or do you mean needing to write a predicate that walks that structure validating db/id against this new predicate?
2020:01:29 21:39:01 alexmiller you can still s/and a predicate to a map that checks a value in the map
2020:01:29 21:39:13 alexmiller it's also an option to just not spec :db/id
2020:02:01 22:28:55 kenny Should spec2's s/select work for collections of collections? I would think this would return false.
2020:02:01 23:15:19 seancorfield Given that ::coll-next is a collection, not a schema, I'm not sure how that should work...?
2020:02:02 20:21:25 kenny Hi all. I wrote a library that adds support for Spec2's schema & select to Spec1. My company is working on a new product and thus a new data model. Defining the data model using Spec1's :req feels so wrong now that we see Spec2's direction. We intend to use this library to bridge the gap until Spec2 is out or official support for some sort of Spec1 schema/select is released. It's still very early but the few examples I've worked through all work as expected!
https://github.com/ComputeSoftware/spec1-select
2020:02:04 00:09:35 kenny ... now with CLJS support 🙂
2020:02:04 22:56:51 kenny Will Spec2 provide a way to select the required keys of a nested Spec based on a dispatch value? For example, say you have a map that has a key ::items . The required keys for each map within the items list will vary based on the :type key on each item map.
I could write add a top-level predicate to the ::items spec via s/and that would check the validity of each map in the items list against another defined spec. That would result in poor error messages though.
2020:02:04 23:06:40 alexmiller Tbd
2020:02:04 23:11:53 kenny Got it. You probably have a good idea of what it may look like already. In case it may be of any use, this is what I'm thinking we'll end up implementing internally for this problem.
(s/select
::report-input-data-schema
[::instances
{;; map of dispatch value to required keys
::instances {:default []
:instance.type/a [:instance.a/prop1]
:instance.type/b [:instance.b/prop1]}
;; required keys for all maps
:common [:instance/status
:instance/name]
;; dispatch fn
:dispatch :type}])
2020:02:05 06:45:09 colinkahn @kenny couldn't you do this with a multi-spec defmethods returning select specs?
2020:02:05 15:55:51 kenny Not when it's a nested collection.
2020:02:05 16:02:42 colinkahn Ah right, agreed this would be great to have built in
2020:02:05 16:08:04 kenny Definitely agree. Fortunately it's not that hard to build in yourself if it doesn't include it. It really does seem like something it should support by default though.
2020:02:05 17:13:50 colinkahn Right, I built a spec that did something like this for spec-1, basically “clamped” nested multi-specs. It was a bit unwieldy because the definitions had to be so long.
2020:02:05 14:42:05 Wilson Velez hi, when to use :req or :req-un? I’ve used :req-un always
2020:02:05 15:44:32 orestis If you have maps with namespaced keywords, use :req
2020:02:06 01:15:42 kenny I guess it's more of a fundamental thing -- a schema is always a set of optional attributes. If attributes become required, it cannot be a schema.
2020:02:06 03:50:06 alexmiller yes, that
2020:02:06 16:57:01 Ben Hammond I'm working on generators for a spec like
but I want to create linkage between the start/end dates.
I've written a generator `
(defn sqlinterval-generator
to give me the interval boundaries that I want,
but what is the best way to poke them in to the overal keys structure ?
In the past I've used a bind and fmap to overwrite the interval boundaries
AFTER the default generator has given me random ones;
seems a bit non-intentional though; is there a better way?
2020:02:06 17:04:34 alexmiller go the other way
2020:02:06 17:05:08 alexmiller I guess you kind of are
2020:02:06 17:05:28 alexmiller I don't think what you're doing is bad necessarily
2020:02:06 17:05:51 alexmiller and it might be the easiest way in this situation
2020:02:06 17:06:16 Ben Hammond so I have this to daisy-chain the generators
(defn generator-decorate-sqlinterval
"overwrites the gen-in values with values from the interval generator"
[gen-in start-key end-key interval-generator]
(test.gen/bind gen-in
(fn [o] (test.gen/fmap
(fn [[start end]] (assoc o start-key start
end-key end))
interval-generator))))
2020:02:06 17:07:39 Ben Hammond I guess I'm asking if there is something like (test.gen/tuple but that will give me a map instead of a vector
2020:02:06 17:08:24 Ben Hammond nah that wouldn't be any easier would it?
2020:02:06 17:08:33 Ben Hammond I'll keep hacking away at it then
2020:02:06 17:08:33 Ben Hammond thanks
2020:02:06 19:09:46 waffletower Does clojure.spec.alpha support defining a spec for a map where keys are not known in advance, but a spec exists for their values?
2020:02:06 19:14:22 seancorfield @waffletower For qualified keys, yes, if I'm understanding you correctly. Not for unqualified keys.
2020:02:06 19:16:41 seancorfield Spec checks any (qualified) keys' values against existing specs, even if s/keys doesn't list them. But if you generate/exercise ::bag, you will only get empty maps.
2020:02:06 19:27:07 waffletower Thanks Sean, I need something like a wildcard key, such that every key in a map would validate against a specific spec.
2020:02:06 19:33:55 waffletower
{:unknown 1
:mystery 4
:unspecified 7} etc.
2020:02:06 19:34:48 waffletower And somehow link any key to be validated against (s/def ::wildcard-key int?)
2020:02:10 06:29:03 jrwdunham I want a spec that accepts nilable values but does not generate them. So far I'm going with a macro like the following. Is there a better way to do this? Is wanting to do this an indicator that I'm making a bad design decision? The motivation is that I'm working with a db schema that allows NULL but my validation on data coming from users will prevent incoming nil values. I want my generators to simulate user-supplied data going into the DB and not data coming out of the existing DB.
2020:02:10 12:54:37 thom Does anybody know of anything resembling linear temporal logic (or something weaker but still useful for describing sequences) for spec?
2020:02:10 21:20:11 andy.fingerhut I do not know if this is anywhere near what you are looking for, but a Google search for the terms "clojure linear temporal logic" turned up this project: https://github.com/esb-lwb/lwb
2020:02:10 21:21:04 andy.fingerhut Spec I am sure has nothing built in for this, except the general purpose escape hatch of letting you use arbitrary functions you write that return true/false as specs -- which is a huge enabler for anything you can write code for.
2020:02:10 21:21:39 andy.fingerhut It won't necessarily help you generate random instances satisfying that function -- that is separate work you would need to do if you wanted that capability.
2020:02:12 23:25:55 telekid Instrumentation / generation question. In the following gist, an outer function calls an inner function twice with the same value (`i`). I want to write an fdef for the outer function, using a :fn predicate to assert that the return from the two calls to the inner fn are identical. However, when I try to stub the call to inner with instrumentation, the generated values returned by inner’s stub are (sensibly) different.
Is there any way to express on the fdef of inner that the generated values should be some known function of the input?
gist here: https://gist.github.com/telekid/01b0aed73511167c337794bc286908cf
2020:02:13 04:50:44 colinkahn Would using :replace instead of :stub work in this case?
2020:02:13 20:45:53 telekid Oh, interesting! I’ll look into that, cool idea
2020:02:13 13:37:11 cddr Hey folks,
Looking for a bit of advice about adding specs to an open source project. We have a "command syntax" where commands are basically vectors. A few examples are provided below but essentially each command's first item is an op-code and subsequent items configure the operation in an op-specific way....
Seems like there's a few ways this could be spec'd and I was just wondering if anyone had any thoughts about how I should structure it. There are some additional commands not shown here but not more than 10 overall and I don't anticipate too many more being added in the future.
2020:02:13 17:41:56 colinkahn If you look in the spec 2 repo currently there is still the keys spec, will that stay? If so, with schemas/select is there a suggested limited set of usecases for keys now?
2020:02:13 17:59:06 seancorfield @colinkahn I have seen a couple of mentions from @alexmiller indicating that keys may not stay in the final Spec 2 version -- but Spec 2 is still very much alpha and in flux. Also the fdef stuff is likely getting a substantial overhaul.
2020:02:13 18:01:27 seancorfield I spent quite a bit of time tracking Spec 2 last year and had a branch of our system at work that passed all tests using Spec 2 instead of Spec 1 -- but still using keys and there were quite a lot of changes just to get to that point (and a number of bugs and workarounds I had to deal with). I gave up in early October and decided that we wouldn't attempt a direct migration from Spec 1 to Spec 2: instead we'll adopt Spec 2 for new code and perhaps incrementally replace our Spec 1 usage over time.
2020:02:13 18:02:30 seancorfield One thing that needs to be figured out for a mixed Spec 1/2 codebase is how the spec systems interact, because Clojure itself uses Spec 1 (1.9, 1.10) and so you can't (easily) mix core specs with your own Spec 2 stuff right now.
2020:02:13 18:08:31 colinkahn Thanks for the insight @seancorfield. In your experience do you think your refactor would have been easier if you were already using https://github.com/ComputeSoftware/spec1-select for you spec 1 code? I’m looking to port some of my code over to it.
Regarding keys I was curious if the (and) and (or) syntax used within :req/opt(-un) would exist in some form in spec 2.
2020:02:13 19:28:44 alexmiller Re and/or - no, probably not
2020:02:13 19:29:06 alexmiller Schemas support unqualified keys in spec 2
2020:02:14 06:41:00 colinkahn Thanks!
2020:02:13 18:09:15 seancorfield Our use of Spec dates back to early alphas of Clojure 1.9 so that library is moot for us.
2020:02:13 18:10:09 colinkahn Right, not saying for your specific case but I’m evaluating using it to make our transition to spec 2 eventually smoother
2020:02:13 18:11:07 seancorfield Given how much Spec 2 is in flux, I don't know that such a library would necessarily be a useful starting point. Most of the work for us in attempting to get our Spec 1 codebase running on Spec 2 was around the much stricter delineation of symbolic specs vs spec objects etc.
2020:02:13 18:11:40 seancorfield We had a lot of Spec 1 specs that simply weren't legal in Spec 2 and we had to rewrite them to use defop and various other constructs.
2020:02:13 18:13:06 seancorfield Remember: we stayed with keys when we attempted that migration -- it wasn't about switching to schema/`select`... that was considered "future work".
2020:02:13 18:18:28 colinkahn Good to know, I haven’t dived into the defop stuff in spec 2 yet so didn’t consider that. Besides future compatability I definitely feel the pain points that schema/select are meant to solve, which is why I’m seeing that as a valuable first step.
2020:02:13 19:47:55 seancorfield Sure, they are a big improvement on Spec 1's keys -- but I would be wary of relying on a 3rd party lib that tries to backport Spec 2 features onto Spec 1 when a) Spec 2 is still changing so much and b) Spec 2 already has stricter rules about "what is a spec" than Spec 1.
2020:02:13 19:48:56 seancorfield I don't consider a lib like that to be a useful "future proofing" option...
2020:02:13 19:50:07 seancorfield (it's an interesting exercise, to be sure, and if folks want some of what Spec 2 offers while staying on Spec 1 for an unspecified future time period then maybe it's of some use -- but I really doubt it will make the "migration" from Spec 1 to Spec 2 any easier)
2020:02:14 23:21:55 jasonjckn A way to attach meta data to a spec or something similar enough ?
e.g.
(s/def ^:property1 ::myspec ...)
2020:02:14 23:30:41 seancorfield @jasonjckn You can't add metadata to a keyword.
2020:02:14 23:34:45 jasonjckn yah I'm aware; thanks for the reply
2020:02:14 23:35:13 jasonjckn https://github.com/mpenet/spex is a 3rd party lib that offers meta data ; but unlikely to be maintained i'd guess
2020:02:15 11:21:52 mpenet You'd be surprised ;)
2020:02:15 11:22:41 mpenet In any case, it's just an atom and a couple of functions, easy to make a metadata registry of your own until core implements one
2020:02:15 20:08:04 Eddie Does anybody know of a java-time LocalDate generator? My search came up empty.. I am fine with implementing my own, but it seems like something that probably exists.
2020:02:15 21:18:13 dominicm I used a few number generators.
2020:02:15 21:18:23 dominicm Using gen/let
2020:02:17 18:24:28 Eddie Thanks, thats what I ended up doing. 👍
2020:02:16 08:46:55 abdullahibra Hi everyone,
2020:02:16 08:47:32 abdullahibra is there any simple example for howto modify the error message produced by clojure.spec to be more readable ?
2020:02:16 18:22:09 seancorfield @abdullahibra You could look at Expound and Phrase, and there was a new one announced recently (which I think extends Phrase).
2020:02:17 17:03:23 ikitommi oh, the closed spec don’t even work there. Can one close selects (or subslects)?
2020:02:17 17:04:07 ikitommi closing the schema seems to work
2020:02:17 17:04:26 ikitommi … but can’t make the one field optional just with schema.
2020:02:17 17:12:10 alexmiller you can only close schemas, but I think there are some impl gaps in properly validating use of a select over a closed schema
2020:02:17 17:12:53 alexmiller but otherwise, sure? I don't consider any of the schema/select stuff done, so hard to suggest "best" practices :)
2020:02:17 17:14:51 ikitommi thanks. would be great to able to say, “close all schemas/selects” in the future.
2020:02:18 05:19:08 bmaddy Is it possible to parse streams or lazy seqs with spec? Specifically, I'm interested in a stream where trying to consume too much data would cause it to hang forever.
2020:02:18 05:21:07 bmaddy To clarify that a little, there would be a number at the beginning that tells me how many more numbers to consume.
2020:02:18 05:21:24 seancorfield Spec is not designed to be used for parsing. I suspect you already know that but it's worth repeating.
2020:02:18 05:24:09 bmaddy Huh, I guess I'd forgotten that. So definitely worth repeating! Thanks!
2020:02:18 05:26:00 seancorfield (you can abuse Spec for parsing but it will often be suboptimal... at best)
2020:02:18 08:44:35 jeroenvandijk @seancorfield I understand that Spec is not meant for high performance parsing. I do find it useful and use it sometimes to parse datastructures. Do you know of a nice alternative that is performant?
2020:02:18 08:58:33 delaguardo https://github.com/youngnh/parsatron
this library could be a good starting point
but it was designed to parse strings instead of data so might need some tweaks
2020:02:18 09:02:29 jeroenvandijk That's useful as well as alternative to instaparse. Thank you
2020:02:18 09:18:33 delaguardo instaparse can not be turned into the parser of data structures unfortunately. Only strings are allowed and this is deeply hidden inside of implementation details
2020:02:18 09:19:40 jeroenvandijk Yeah I might have combined instaparse with clojure.spec. Not sure actually
2020:02:18 10:59:36 mmeix I just read this article: https://juxt.pro/blog/posts/parsing-with-clojure-spec.html (a small parser for ical-entries) and watched this video: https://www.youtube.com/watch?v=xLB8Xgt8lpA (a simple hiccup-parser). These seemed (to me, as a spec beginner) reasonable applications of spec - maybe, because, they are quite simple? Would there be a problem for using spec at this scale?
2020:02:18 11:54:24 mmeix Additional question: is it considered idiomatic to dispatch a multimethod on the result of a s/conform? Are there drawbacks?
2020:02:18 12:20:16 jeroenvandijk I have done that. Cannot think of real drawbacks expect for that if your spec changes you have to update the multimethods
2020:02:18 12:20:18 jeroenvandijk (Changing) dispatch functions of multimethods are a bit of a hassle in a repl environment as it is hard to undefine (for good reasons). But to make this work in the repl you need to use some tricks
2020:02:18 13:39:47 mmeix ok, thanks!
2020:02:18 14:01:29 dergutemoritz Temporarily placing (def my-multimethod nil) in front of the defmulti and then recompiling the namespace is an easy way to do it!
2020:02:18 15:41:04 jeroenvandijk @U06GVE6NR This is true, but you have to reload all the namespace or the method is not extended the way you expect. My trick is to define a dispatch function and define the method as (defmulti my-method #'dispatch-my-method). This has been the most reliable way for me when working in the repl for a long time
2020:02:18 15:41:39 dergutemoritz Ah right, if you have method implementations in other namespaces, then this trick won't suffice, good point 🙂
2020:02:18 15:41:47 dergutemoritz Good idea with the var
2020:02:20 18:21:29 rickmoynihan hmm…
I’m wanting to spec some data that is coming from a 3rd party library, the Ataraxy router.
It always uses the namespaced keyword :ataraxy/result to contain the parsed parameters for the route; however because ataraxy is configured on a route by route basis; the exact data structure differs, yet the ring request map always contains the same namespaced keyword.
It strikes me that this is an unfortunate mistake; in that the same global keyword is being used to name different data.
What is the best way to spec such a thing?
I’m thinking a multispec is really the only option, that could dispatch on the first argument. :ataraxy/result is a vector of the form [:route/id ,,,route-params,,,]
2020:02:20 18:22:58 rickmoynihan or I suppose I could coerce it into something else first 😞
2020:02:20 18:26:05 rickmoynihan actually thinking it’s better to just spec the functions after it, that don’t depend on the :ataraxy/result key.
2020:02:20 18:45:04 alexmiller you could just do something like (s/cat :route-id ::route-id :params (s/* any))
2020:02:20 18:45:26 alexmiller but it depends a lot what route-params is
2020:02:20 18:45:39 alexmiller if it's kwargs, then keys* would work great
2020:02:20 21:47:29 jasonjckn my data looks like {::type "foobar" ::a ... ::b ... ::c ...} I want different s/keys validation based on the value of ::type
whats the feature i'm looking for
2020:02:20 21:47:42 jasonjckn it's like ~ conditional spec validation polymorphic to field value
2020:02:20 21:47:58 jasonjckn i guess some sort of mulitmethod on the spec
2020:02:20 21:48:16 jasonjckn reading about multi-spec now, this looks like it
2020:02:20 21:57:23 jasonjckn for the :default case
2020:02:20 21:57:29 jasonjckn what would be a canonical way to fail spec validation
2020:02:20 22:00:13 jasonjckn can I do :default ::s/invalid ?
2020:02:20 22:00:20 alexmiller oh, you mean multi-method default
2020:02:20 22:00:30 jasonjckn
(defmethod tagmm :default [_] ::s/invalid)
will test that out
2020:02:20 22:00:42 alexmiller the multimethod returns specs so you need to return a spec that always fails
2020:02:20 22:01:19 alexmiller so probably more (constantly false) would work
2020:02:20 22:01:32 jasonjckn ok great
2020:02:20 22:02:00 alexmiller or #{}
2020:02:20 22:02:08 jasonjckn that seems more canonical
2020:02:20 22:02:11 alexmiller a set of no values :)
2020:02:20 22:02:14 jasonjckn particular in the spec explanations
2020:02:20 22:02:19 jasonjckn it'd be nice to get something that made sense
2020:02:20 23:45:32 vemv a q out of curiosity more than anything else:
(spec/valid? (spec/coll-of char? :min-count 1 :max-count 20) "abc") doesn't work (and a straightforward impl would underperform anyway?)
are there spec-related libraries offering a coll-of-like API that efficiently works on strings?
I just like coll-of's API, in that it doesn't resemble regex
2020:02:21 00:08:49 seancorfield @vemv The standard answer is that Spec isn't designed for parsing strings 🙂
I would like to use conform to know if the second element of the sequence is :b or :c (labels given in cat). conform returns a clojure.lang.PersistentHashMap, which does not key the keys in the same order as in the sequence.
2020:02:24 22:49:13 alexmiller s/tuple is a better match
2020:02:24 21:57:30 seancorfield s/conform just tells you how it matched. If you call s/valid? and get true then the input sequence is valid as is.
2020:02:24 21:59:19 gariepyalex I would have liked to use conform to parse the sequence and end up with {:a -2, :c 10}
2020:02:24 22:02:31 seancorfield Spec isn't really a "parser" but if you know your elements should be in a given order, you can easily manipulate the data you get from s/conform to produce a more appropriate structure.
2020:02:24 22:03:28 seancorfield (you know you're going to get :a first and :c last so the only question is (contains? conformed-data :b) to see if a zero is present...
2020:02:24 22:04:16 gariepyalex ok thanks for your help!
2020:02:25 14:12:28 Filipe Silva heya
2020:02:25 14:12:45 Filipe Silva is it possible to rename a key with s/keys ?
2020:02:25 14:13:11 Filipe Silva I'd like to do something like this
(s/def ::msg (s/and string? #(< 20 (count %))))
;; TODO: declaring a spec for a limited :db/id doesn't sound right. Figure out a better way.
(s/def :db/id (into #{} (range 1 10)))
(s/def ::map-datom (s/keys :req [:db/id ::msg]))
(s/def ::simple-tx (s/coll-of ::map-datom :kind vector? :min-count 0 :max-count 3))
2020:02:25 14:13:43 Filipe Silva but instead of (s/def :db/id, I'd like to do (s/def ::id
2020:02:25 14:14:18 Filipe Silva and then on s/keys say that ::id is required but should show up as :db/id
2020:02:25 14:15:15 Filipe Silva I'd like to do this because I don't want to be declaring the spec for the real :db/id, but would like to have an alternative version of it for testing
2020:02:25 14:27:56 alexmiller in short, no - the whole idea with spec is that attributes are primary and have meaning and maps are just aggregations of attributes
2020:02:25 14:28:57 alexmiller if you want a broader definition of ::db/id, then your spec should reflect that
2020:02:25 14:32:06 Filipe Silva ok, thank you for explaining it to me
2020:02:25 14:32:41 Filipe Silva if I want to just generate test data that is a subset of all possible data, should I be using something else?
2020:02:25 14:34:35 rickmoynihan @filipematossilva: It sounds like you might be complecting generating data with specing. If in your tests you want to generate known id’s that are a subset of valid id’s, you can probably just pass a generator for that set where you need it.
2020:02:25 14:35:19 Filipe Silva yes, I think I am
2020:02:25 14:35:55 Filipe Silva but s/keys does not support inline value specs (by design, according to the docs), so I don't think I can do that
2020:02:25 14:36:20 Filipe Silva (unless, of course, I am misunderstanding what you propose)
2020:02:25 14:36:22 Filipe Silva an alternative is to generate the test data with exercise, and then map over the results to change the key
2020:02:25 14:36:30 Filipe Silva which sounds fair enough, but I wanted to inquire first
2020:02:25 14:36:41 alexmiller you can pass generator overrides when doing most of the functions that generate in spec, but you still have to generate data that conforms to the spec
2020:02:25 14:39:04 Filipe Silva ah, then this was just something I did not know existed
2020:02:25 14:40:31 rickmoynihan Totally! :-)
When I first started with spec I made the mistake of over speccing to get spec to try and generate test data of the right shape… but now prefer the alternative of specing a little more broadly but using :gen ’s to generate the more precise data I want.
Obviously it’s all trade offs.
Anyway I was just suspecting Filipe might be falling into this same trap.
2020:02:25 14:42:44 Filipe Silva yes, I was 🙂
2020:02:25 14:45:04 Filipe Silva thank you
2020:02:25 14:45:18 Filipe Silva I'll try to fiddle with the :gen option on s/keys and see where that takes me
2020:02:25 15:00:52 rickmoynihan 👍 hope it helps
2020:02:25 15:11:11 Filipe Silva @rickmoynihan I'm trying to use the :gen key on s/keys, but I don't quite follow how I can use it to replace a single def
2020:02:25 15:11:30 Filipe Silva is that possible? or does :gen need to override how everything is generated
2020:02:25 15:11:53 Filipe Silva if you have an example of using it with s/keys it would be super useful
2020:02:25 15:23:15 Filipe Silva oh, I think this is the idea?
2020:02:25 15:23:55 Filipe Silva making the real spec, then making another spec that has a generator that takes the real spec generator and adds an override
2020:02:25 15:26:20 rickmoynihan That’s not quite what I was suggesting…
2020:02:25 15:33:33 rickmoynihan @filipematossilva: You can override on s/exercise, or st/check as well, so you can tailor your generation there rather than in the specs if you want.
2020:02:25 15:33:36 Filipe Silva ah on exercise itself
2020:02:25 15:34:08 Filipe Silva thank you for taking the time to make the examples for me
2020:02:25 15:34:13 rickmoynihan 👍
2020:02:25 15:34:16 Filipe Silva it was very helpful
2020:02:25 15:36:50 rickmoynihan essentially s/exercise / st/check are the testing entry points to spec; so you can split the concerns of specing and overriding generation/testing at those entry points… whilst s/conforms/valid? and s/explain-xxx are the checking side of spec — and consequently don’t support generators at all (because they’re a different concern)
2020:02:25 15:36:57 rickmoynihan at least that’s how I think of it
2020:02:25 15:37:12 rickmoynihan There are probably more nuanced explanations 🙂
2020:02:25 15:39:08 Filipe Silva that sounds much more sensible than I what was attempting
2020:02:25 15:39:53 rickmoynihan its ok been there, done it, got burned! 🙂
2020:02:26 04:48:03 yuhan I was playing around with spec2 and noticed that it doesn't allow for using local bindings in definitions
2020:02:26 04:48:21 yuhan ie. this worked in spec.alpha:
2020:02:26 04:48:56 yuhan but not in spec2, where it would throw an error "Unable to resolve symbol max-v"
2020:02:26 04:50:08 yuhan Is this a deliberate design decision or was it ever officially supported in spec.alpha in the first place?
2020:02:26 05:41:53 seancorfield @qythium Spec 2 draws a much sharper line between symbolic specs and spec objects. Spec 2 has new facilities for creating specs programmatically that allow you to create such a spec, but in a different way.
2020:02:26 06:12:27 seancorfield I don't have any code handy but defop is also handy for building new spec predicates.
2020:02:26 06:13:23 yuhan That actually makes a lot more sense than the nested macros in the original Spec 🙂
2020:02:26 06:14:13 yuhan which I recall macroexpanding to trace the logic and eventually just treated as magic
2020:02:26 06:14:34 yuhan thanks!
2020:02:26 18:13:57 ben If I have written a s/fdef for a function, is there an easy way to generate property tests for that function (as part of a test suite), without having to manually (re)write all the generators?
2020:02:26 18:28:20 ben I saw this, but my understanding is that is runs the tests (like test.check/quick-check) rather than creating a test (`deftest`). Have I misunderstood?
2020:02:27 09:59:18 ben Thanks @U6GFE9HS7 - I am aware of test.check, but as I understand it, I will have to write my own generators again, rather than using fdef’s :args to check the function
2020:02:27 10:00:03 ben Thanks @U083D6HK9 - this looks like what I was imagining. Why do some not recommend it?
2020:02:27 16:01:28 kenny I think the argument is that they should run at a different time than regular unit tests because they are different. I don't have a problem with that -- run them in whatever way fits your company's workflow. These sort of tests are invaluable though. You should certainly run them before deployment. Having one test runner (https://github.com/lambdaisland/kaocha for us) is quite nice.
We have a large codebase with thousands of tests. Running gen tests with our unit tests has not caused us any substantial pain. They typically take 1s or less to run. The more complex functions with custom gens can take longer. The results of a gen test do not fit well into the expected/actual framework clojure.test uses though. That is certainly another downside.
2020:02:27 18:57:15 robertfw @UFUDSN6BE re: generating args. I use the following code to generate args from an fdef:
(defn arg-gen
"Creates an argument generator for a given function symbol"
[sym]
(-> sym s/get-spec :args s/gen))
2020:02:26 18:14:33 ben Like stest/check but as part of my tests
2020:02:27 11:40:22 rickmoynihan @ben Kaocha has support for suites of fdefs
2020:02:28 00:41:41 robertfw I'm trying to figure out how the overrides map for (s/gen) works, namely for overriding a generator that's down the tree a bit. That is, I have a top level spec ::data which has a key ::things which is a collection of ::thing, which has a key ::properties . I'm generating the top level - ::data - and wanting to override both ::things so I get only 1 thing, and ::properties so I get an empty list. using (s/gen ::data {::things #(s/gen (s/coll-of ::thing :min-count 1 :max-count 1))}), I'm able to force ::things to only have 1 ::thing, but I haven't figured out the incantation needed to have ::properties be empty.
2020:02:28 00:50:35 robertfw The docstring talks about providing a vector of keyword paths, but I haven't been able to find any more detail on what those paths should look like. I've been trying various combinations
2020:02:28 01:29:13 robertfw I suspect the issue is in using (s/coll-of) - for now I'm using an spec.gen/fmap call to remove any extra data I don't want, but would prefer to be able to just generate only the data needed
2020:02:28 01:32:23 seancorfield Your s/gen override for ::things could have a third argument which is the override for ::properties
2020:02:28 01:33:01 robertfw gotcha. that makes sense
2020:02:28 01:33:37 seancorfield I would have expected to be able to use [::properties] as a key in the top-level overrides but that doesn't seem to work.
2020:02:28 01:33:51 seancorfield I can't figure it out from the code 😕
2020:02:28 01:34:16 robertfw Yeah I spent some time walking through some spec guts but couldn't unravel it
2020:02:28 01:35:09 seancorfield s/keys and several others build a vector of keys to index into the overrides but I got lost trying to trace through multiple layers
2020:02:28 02:54:14 colinkahn I think it's because the nested s/gen creates a different context that doesn't know about the overrides in the top level s/exercise
2020:02:28 03:01:03 alexmiller ^^ this, there's actually a bug around this iirc
2020:03:01 19:42:23 bbrinck In spec2, how would I use a set spec to match against a symbol that happens to be the name of a macro?
2020:03:01 20:08:46 alexmiller there is actually a known issue around sets of symbols (kind of a collision with symbol as function reference, which need qualification)
2020:03:01 20:13:12 alexmiller a workaround for the moment is (s/register ::is-or (s/resolve-spec #{'or}))
2020:03:01 21:06:05 bbrinck Perhaps just a different view on the same underlying issue, but it looks like spec2 is using clojure.core/and instead of the and symbol for combining specs in :req?
2020:03:01 21:09:25 bbrinck I would have expected :req [(and :keys-spec/name :keys-spec/age)]
2020:03:01 22:35:41 favila I’m pretty sure spec1 does the same? I think the form (in spec1) gets rewritten to a predicate fn in a trivial way. Iirc you can even use your own predicates instead of only and/or
2020:03:02 02:36:32 alexmiller technically, it is the clojure.core/and function in both spec 1 and 2
2020:03:02 02:41:54 alexmiller not really going to do anything with it as the plan is for s/keys and and/or to go away in spec 2 anyways
2020:03:02 15:30:29 bbrinck Ah, that’s good to know. Thanks!
2020:03:02 15:14:35 norton @alexmiller I tried the following example from spec 2 wiki page but it fails with “Couldn’t satisfy such-that predicate after 100 tries”. Is this a known issue?
2020:03:02 15:14:35 norton @alexmiller I tried the following example from spec 2 wiki page but it fails with “Couldn’t satisfy such-that predicate after 100 tries”. Is this a known issue?
2020:03:02 17:07:07 alexmiller sorry, not sure off the top of my head
2020:03:03 00:25:11 norton @alexmiller I was able to find the reported error (outside of the generator).
The last s/def below fails with the following error:
The result of the last 2 expressions is not expected (to me).
2020:03:04 03:53:35 jrwdunham Are there best practices for how specs are intended to be identified with one another? I have the situation where I want to define spec ::data via (s/def ::data ::specs/jsonb) (where (s/def ::jsonb map?)). I want to define a map (`spec-casters`) from specs to the functions that can be executed to transform their values (for db insertion). The spec-casters-YES-PLEASE function (which I want to use) seems to do what I want, but I have already run into situations in development where (get spec-casters-YES-PLEASE (s/spec ::other-ns/data)) ;; => nil. I could use spec-casters-NO-THANKS but I would rather avoid that verbosity. This makes me wonder consider that there might be something inadvisable about this approach.
2020:03:04 04:01:18 alexmiller I only understand about 30% of what you're trying to do, but (s/spec ::specs/jsonb) is a spec object without well-defined equality semantics so it seems like a bad choice for a key
2020:03:04 04:09:49 jrwdunham Ok, the fact that specs lack well-defined equality semantics suggests that I should just be more explicit and use the NO-THANKS approach where the keys are just the namespaced keywords.
2020:03:04 04:10:57 jrwdunham However, that seems to violate the intention of a key spec though, namely that its value should conform to it ...
2020:03:04 04:11:49 alexmiller but you're not conforming
2020:03:04 04:14:46 alexmiller you're just matching spec objects
2020:03:04 04:15:04 alexmiller not using them for anything
2020:03:04 04:17:57 jrwdunham right, that's true.
2020:03:05 22:03:58 jrwdunham (btw, thanks for taking the time to discuss my question Alex. I do appreciate it.)
2020:03:08 00:12:31 ag Do we have a repo with a collection of arbitrary specs for all sorts different cases? I often need to create a spec for something so trivial but end up spending more time than intended, only to remember later that I did it already in some other Clojure project. Sometimes I just need to perform Github search and find some gems. Would be nice to have a community driven repo of good examples.
2020:03:08 00:16:40 seancorfield Hmm, I see Specs as being tied to domains rather than being that sort of reusable -- what sort of things do you have in mind @ag?
2020:03:08 00:28:13 ag things like various datetime, timestamps, comma separated lists of currencies or/and US states and US-state abbrevs, etc.
2020:03:08 00:29:36 ag You right though, maybe not a collection, but perhaps something like a cookbook
2020:03:08 00:53:39 seancorfield You're talking about strings? Parsing strings? That's not a good use for Spec tho'...
2020:03:13 07:59:39 ro6 I went down that road a bit once too, mostly for want of the automatic error descriptions Spec gives, but extending down into a String (eg "That's not a valid email because X, Y, Z"). Spec definitely does a good job of that for aggregates/shapes. I don't recall finding a satisfactory solution at the time.
2020:03:08 00:57:10 seancorfield Specs for datetime/timestamps -- inst?, s/inst-in.
2020:03:08 01:01:04 seancorfield Locales, country codes, currency codes -- all easily available from Java classes but I guess ready made Specs for sets of those things would save you writing a single line of Clojure?
2020:03:08 01:04:00 seancorfield US states -- that's domain-specific: do you just want the 50 states, plus DC? Plus the various US territories/islands? But stuff like that belongs in databases and you can construct a set spec from that.
2020:03:08 16:06:24 derpocious hi, I am wondering if it is possible to instrument the return value of a function?
2020:03:08 16:59:33 Eddie I like to use post-assertions combined with spec. They look like this.
2020:03:09 07:10:24 andy.fingerhut I have not used it before, but the orchestra library aims to do this: https://github.com/jeaye/orchestra. It is not included in spec because it is instead recommended to only do checking of return values during testing, e.g. property-based testing using randomly generated arguments.
2020:03:09 18:55:15 Gustavo Goretkin Hi. Total beginner here. I've seen mention of this global registry of specs. Where can I see which specs are defined there? Also, are there any writings about spec and other format-specific Schema definitions (like XML Schema (.xsd) or JSON schema things)?
2020:03:09 18:59:09 favila it’s global--to your process, not the world
2020:03:09 19:00:52 Gustavo Goretkin I see. Thanks! I think I misinterpreted. Are there clojure libraries/packages that primarily define specs?
2020:03:09 19:05:54 favila I only know of https://github.com/cognitect-labs/anomalies Sometimes a library will include specs for its api or extension points. I’m not aware of any library that just shares spec objects the way one would share utility functions
2020:03:10 20:35:51 Gustavo Goretkin Thanks for the pointers.
I was expecting more packages like this, I think, to capture "ontologies", the same way there are schemas for e.g. geometry data, like https://github.com/mapbox/geobuf , http://wiki.ros.org/geometry_msgs
Is that kind of the spirit of spec at all?
2020:03:10 12:09:11 ataggart Our tests are occationally failing with "Couldn't satisfy such-that predicate after 10 tries.", and the associated pred is :pred #object[clojure.core$ratio_QMARK_ 0x62fb2f00 ".
2020:03:10 12:25:54 gfredericks ratios are conceptually goofy because you may or may not want to consider integers to be ratios in a given context
it's useful to think of integers as special kinds of ratios, because almost any use case for ratios should almost certainly apply to integers as well. It's hard to do arithmetic with ratios without bumping into integers inadvertently at least (e.g., (* 3/4 4/3))
but arguably it would surprise people if (ratio? 42) returned true, so it doesn't; maybe there are other good reasons for this too beyond just surprise
in any case, there's a mismatch because test.check takes the first approach, but ratio? wearing its spec hat obviously has the same semantics as ratio? wearing its predicate hat, and so when you generate from it you don't want to generate integers
I think (such-that ratio? (ratio)) could definitely be improved; setting a higher retry threshold would stop the exceptions, but a custom generator would be the best; something like (gen/let [[n d] (gen/tuple gen/large-integer gen/large-integer)] (let [r (/ n d)] (if (ratio? r) r (/ (inc n) d))))
there might be some arithmetical edge case there I'm not thinking of, but I think that works
2020:03:10 13:25:11 ataggart Thanks for that! I thought I was losing my mind.
2020:03:10 13:28:09 ataggart Perhaps spec'ing on rational? is the way I should go.
2020:03:10 12:27:07 gfredericks also I forgot to handle (zero? d)
2020:03:11 23:06:36 ag how can I “unalias” an fdef spec?
2020:03:24 01:44:01 seancorfield @quest You can certainly specify that with clojure.spec. No idea about spec-tools.
2020:03:24 01:46:00 Quest Yeah, the :min-count option works fine in normal specs... I'm trying to cast to clojure specs from an EDN business schema I have, proving difficult
2020:03:24 01:46:33 Quest I know spec2 has much improved support for runtime specs, but I need the CLJS side unfortunately
2020:03:24 01:47:00 seancorfield Ah, so you can't eval either...
2020:03:25 02:35:11 Quest Just to follow up on what I ended up doing:
Creating a separate "pre-runtime" step which generates a file of clojure.spec definitions from the data model.
I expect this can be eliminated once spec2 is usable everywhere.
2020:03:25 02:46:12 seancorfield Sounds like a good compromise for now!
2020:03:24 01:48:04 Quest Yeah. I had some success by doing funky things using the spec private "impl" functions, but it completely mangles the error messages when data fails validation
2020:03:24 18:03:39 yonatanel Is it possible to spec a map where either a namespaced key or another unnamespaced key is required? e.g {:my/a 1} ok, {:my-a 1} ok, {:my/a 1, :my-a 2} should also be fine, {:b 2} invalid.
2020:03:24 20:00:31 colinkahn Spec V1 doesn’t really have a way to say “this map can’t contain :b”. For aliasing :my-a to :my/a you could choose a single “canonical” version of your map and use a conformer when you expect there to be variations of it.
2020:03:24 20:58:55 yonatanel @U0CLLU3QT Thanks. I wasn’t aware of nonconforming. Gave me some options anyway.
2020:03:24 18:40:29 kszabo not directly
2020:03:24 18:40:55 kszabo you have to spec that with additional predicates. (s/keys) doesn’t have support for that AFAIK across req/req-un groups
2020:03:24 21:08:46 ag how do I make a (s/coll-off ::foo), making sure that the elements are distinct by one given field?
e.g.: if (s/def ::foo (s/keys :req-un [:name])), I want that all the names in the generated collection are unique.
2020:03:24 21:11:59 ag nevermind… figure it out… used: (s/and (s/coll-of ,,,, ) #(apply distinct? (mapv :name %)),,,
2020:03:25 11:05:31 Janne Sauvala What is your recommendation using spec2 in a new project? My observation is that many are using spec1, Plumatic/Prismatic Schema or metosin malli and not migrating to spec2 yet.
2020:03:25 12:17:31 alexmiller Spec 2 is not ready for use yet
2020:03:25 13:06:46 Janne Sauvala Thanks, good to know :+1::skin-tone-2:
2020:03:26 06:14:15 Ben Sless I have a bit of a philosophical question: I watched several talks about spec and it's pretty clear how you'd use it for domain modelling when starting from scratch, but several mentioned its benefits in speccing an existing codebase. Any tips on where and how to start with speccing an existing code base?
2020:03:26 06:34:51 robertfw I would start by adding fdefs for key functions
2020:03:26 14:32:28 Timur Latypoff Maybe it's also a good idea to spec places where pieces of code, written by different people in the team, meet. To set expectations and settle future disputes.
2020:03:26 22:02:07 bertofer Hi, does clojure-spec provide or will provide a way to convert schema maps (unqualified) without explicitly specifying spec on every val? e.g.
Seems like a map traversal, is it something that can already be done with normal clojure on the context of spec and schemas?
2020:03:26 23:39:16 alexmiller I dont think you should even need that?
2020:03:27 00:00:56 bertofer yeah on a second thought it’s not very useful
2020:03:27 00:26:32 bertofer Also because it would only apply to spec from simple predicates, not other types of specs
2020:03:27 00:34:53 alexmiller it already does what you're asking, unless I misunderstand what you're asking
2020:03:27 10:41:31 bertofer True, I was confused by this in the schema and select wiki [::a ::b {:c (s/spec int?)}] , but in the unqualified keys section is shown as I wanted
2020:03:27 21:59:41 hiredman https://gist.github.com/hiredman/60e5e6f0025cb38c8a7739d21f5a8ce6 is a little thing I've been toying with for using spec to define and check stateful protocols, meaning it defines how a client and server communicate, and certain client requests are only valid when the server is in certain states. given such a definition you stick it between your client and server and it checks the communication back and forth and gives errors if either side violates the protocol.
2020:03:30 20:21:35 robertfw I'm experimenting augmenting some of our fdefs with some additional keys, to make it easier to write fdefs that test async functions, or fns that have some stateful dependencies injected in the args. I thought I would ask in here to see if there is any strong reason not to do this, or any related changes coming in spec2 that I should be aware of
2020:03:30 20:22:42 alexmiller none that you can prepare for :)
2020:03:30 20:22:59 alexmiller fdefs will probably be changing significantly in spec 2
2020:03:30 20:23:24 robertfw ah. any dev logs or documents that discuss possible directions they will be taking?
2020:03:30 20:24:06 alexmiller no, still in ideation
2020:03:30 20:36:54 robertfw gotcha. thanks 🙂
2020:03:31 18:10:43 arohner Is there a way to get there-exists behavior to compliment for-all? Context: I have a protocol, and a bunch of implementations (30+), and a test suite that is a series of test.check properties. A broken implementation slipped through, because it returned a pathological response: i.e. (s/nilable ::foo) always returned nil. I’d like to assert during the test, “this can return nil or ::foo, but the test must return ::foo at least once”
2020:03:31 18:11:45 gfredericks You could generate 10 or 50 calls or something
2020:03:31 18:12:01 gfredericks Something large enough that it should never fail
2020:03:31 18:12:04 arohner true
2020:03:31 18:12:47 gfredericks It does seem like that would be a useful feature though
2020:03:31 18:13:08 gfredericks Because then you could test things of that sort without paying for 50 trials each time
2020:03:31 18:13:45 gfredericks A similar structure is using such-that and just generating a single thing
2020:03:31 18:18:42 arohner I think I’ve got it using such-that, thanks
2020:03:31 18:18:52 gfredericks There's a bunch of dumb tests in t.c itself that use the 50 things approach; it's slow
2020:04:01 15:34:21 abdullahibra Hi everyone,
2020:04:01 15:34:38 abdullahibra is there a way to convert json schema into clojure spec in dynamic way ?
2020:04:01 15:35:48 abdullahibra what i mean by dynamic here is: i read json schema files and convert them into equivalent clojure spec which could be used
2020:04:01 20:11:21 dominicm I would make a single spec per schema which was a predicate which validated the data using the schema?
2020:04:03 16:22:16 Ben Grabow So far I've been using spec mostly for validating data structures "in-place", meaning I have an x, at the front door of my API I check if it's a valid format, and if it is I continue using x inside my API. There's another way of using spec that I don't really understand yet so I'm looking for good resources on it: I have an x, and at the front door of my API I check if it's a valid format and destructure/reformat it to the format expected internally in my API, y. y might have all the same data as x but perhaps arranged in a different nested structure or with different key names. Are there any good guides to using spec for this kind of thing? I guess this is using spec as a parser, rather than just a validator. This use case is hinted at (https://clojure.org/about/spec#_conform) but I haven't found a good set of examples yet.
2020:04:03 16:25:30 alexmiller That’s what conform is for
2020:04:03 16:25:48 alexmiller “Is this valid, and if so, why?”
2020:04:03 16:26:39 alexmiller With the caveat that it is NOT for “perform arbitrary transformation to other structure”
2020:04:03 16:27:28 alexmiller If you want that, use conform + standard Clojure data transformation
2020:04:03 16:27:29 Ben Grabow In that example, a sequence of characters is being parsed into a map that describes the components.
2020:04:03 16:27:48 Ben Grabow I have one map with a nested structure that I want to transform to another map with a flat structure.
2020:04:03 16:27:51 alexmiller Yeah, dont do that
2020:04:03 16:28:11 alexmiller (That applies to the string case)
2020:04:03 16:28:35 alexmiller If you want to transform maps, use Clojure stuff to do that
2020:04:03 16:28:59 alexmiller If you want to validate, use valid?
2020:04:03 16:29:31 alexmiller If you want to know why something is valid wrt options and alternatives, use conform
2020:04:03 16:29:55 Ben Grabow It's that last part I'm looking for more content on.
2020:04:03 16:30:25 Ben Grabow I don't quite understand when I'm dealing with a situation that's good for conform and when I'm not.
2020:04:03 16:33:57 Ben Grabow I remember in one of Rich's talks he mentioned David Nolen rewriting part of the cljs parser in spec. Is that code publicly available somewhere? It sounds like a powerful use case.
2020:04:03 16:42:09 alexmiller Well that’s mine :)
2020:04:03 16:42:35 alexmiller Using it to parse the input to a complex macro is a good fit
2020:04:03 16:42:57 alexmiller It’s particularly good for “syntax” stuff
2020:04:03 16:43:26 alexmiller If your input is mostly maps and stuff, probably not as useful
2020:04:03 16:45:12 Ben Grabow The difference in the way my input map is structured vs how my output map is structured feels like syntax to me so I wonder where the distinction lies
2020:04:03 16:47:32 Ben Grabow There's the idea that a map is just a seq of kv pairs. What if the spec is an s/cat of kv-pairs instead of an s/keys?
2020:04:03 16:49:37 alexmiller Syntax is positional (this is kind of present in the Greek roots of the word even)
2020:04:03 16:49:57 alexmiller Maps are not positional
2020:04:03 16:50:28 alexmiller Position in syntax is implicit
2020:04:03 16:50:52 alexmiller Spec regex ops describe that, and the conformed values tell you how it was interpreted
2020:04:03 16:51:26 alexmiller Must of the map specs essentially return the map if it’s valid - there was no thing to figure out and tell you about
2020:04:03 16:53:27 alexmiller Specing a map as a seq of kv pairs is not the same thing as maps are inherently unordered
2020:04:03 16:53:45 alexmiller You can’t rely on position to describe the structure of the map
2020:04:03 16:54:04 Ben Grabow hmm, that's a good point
2020:04:03 16:54:09 alexmiller The regex specs like cat will even error if you try to do this, for this reason
2020:04:03 16:56:27 Ben Grabow I had in mind using a big s/or to describe all the possible keys. Which brings up an interesting question, how does the positionality of syntax impact s/or? I usually reach for s/or when I have a couple different possibilities for my input, like a union type, and I'm usually frustrated that I'm pushed towards destructuring and naming the options instead of conforming to just the value.
2020:04:03 16:56:38 Ben Grabow (s/or :even even? :small #(< % 42)) from the docstring as an example.
2020:04:03 16:57:19 Ben Grabow I wish the API were (s/or even? #(< % 42)) for the use I have in mind
2020:04:03 16:57:49 Ben Grabow I feel like there's something I don't "get" in the reasons why s/or asks for keywords to name the options.
2020:04:03 16:59:09 alexmiller or gives you back a map entry of tag (key) and value
2020:04:03 17:02:17 Ben Grabow Sure, I get that. I'm wondering why it does that when it doesn't seem to be about processing the position of a thing. Instead it seems to be about processing the alternatives of a single thing.
2020:04:03 17:27:59 alexmiller well the idea is the same - tell me what choice you made when there was an alternative
2020:04:03 17:28:39 alexmiller in practice, it is often a source of issues in deeply nested data where that's the only conformed thing so I kind of would like to have some more options in spec 2
2020:04:03 17:28:57 alexmiller whether that's an or variant, or a s/nonconforming, or something else
2020:04:03 17:29:07 alexmiller but that's down the list a bit and we haven't talked about it
2020:04:03 17:33:57 Ben Grabow Glad to hear the sentiment is shared.
2020:04:03 17:35:38 Ben Grabow I often run into situations where a map is mostly just a set of specific things so s/keys works, until we add uuids into the mix and they could be strings or uuid objects. I'm using a custom conformer to get around the s/or destructuring for that case but it would be nice to have a more general solution.
2020:04:03 17:39:00 alexmiller one workaround right now is to wrap those s/or's in s/nonconforming
2020:04:03 17:39:07 alexmiller which is not documented, but exists
All these work as expected. The first two produce just the value, without the keyword tag. The third generates both strings and uuids.
2020:04:03 17:51:26 Ben Grabow My ::uuid-string spec is still a little complicated to make the generator work but I think it could be simplified.
2020:04:08 19:06:46 dspiteself I see spec alpha 2 adds select. Will select have a special knowledge of coll-of so that you could select a map of a vector of maps?
2020:04:08 19:07:22 dspiteself Like pull does across datomic cardinality many attributes.
2020:04:08 19:07:40 alexmiller Currently, no, but that’s under discussion
2020:04:08 20:21:57 dspiteself Thanks I really like what is going on over there. That is the only problem left I see that limits our uses.
2020:04:08 20:29:33 alexmiller certainly there are a lot of similarities to pull
2020:04:11 13:46:10 credulous Hi! This is definitely a novice question, but I’ve struggled for a day or so trying to figure out the best way to use spec in my re-frame application. My problems stem from not grokking spec I think.
2020:04:11 13:46:48 credulous I’m using my app db to store form information under a :form key. Forms are stored by form-id.
2020:04:11 13:48:13 credulous
2020:04:11 13:52:21 credulous In each field the “error” is set based on a spec that gets checked on each edit:
2020:04:11 13:52:35 credulous (Lengths are set to zero but will change)
2020:04:11 13:54:28 credulous But I also want to define a spec for the form as a whole, so I can pass the “signup” form above to (s/valid?). But I can’t figure out how to define the structure above, where signup has four keys, each of which have the same structure
2020:04:11 13:55:30 credulous in :signup :team for example, I want to say signup has a map that requires a :team key, and the team key is a map that has a :value key that is defined by ::team in the snippet pasted above
2020:04:11 16:22:44 shooit I’m not sure that I understand 100% but you might need to put each :value key into a different namespace so that they can be a distinct spec e.g :foo/value, :bar/value. You could also check out multi-spec for multimethod like dispatch on the submaps
2020:04:11 13:56:37 credulous I can’t define a ::value spec to include in (s/keys) for :team - because what :value is is different for each sibling key in :signup
2020:04:11 16:26:39 alexmiller spec is primarily designed to associate semantic specs with namespaced keys, so by using :value in each of these places (instead of :id/value or whatever), you are outside the sweet spot
2020:04:11 16:27:21 alexmiller you can do this with s/keys and :req-un for unqualified keys though, but you'll need to match up the name parts of the specs
2020:04:11 16:32:19 alexmiller
(s/def :team/value ::team) ;; same for the others
(s/def :team/team (s/keys :req-un [::team/value])) ;; same for the others
(s/def ::signup (s/keys :req-un [:team/team ..etc..]))
2020:04:11 16:33:23 alexmiller as I said, this is outside the expected pattern so it's more work for sure
2020:04:11 18:08:18 credulous Thanks Alex. What is the preferred way to satisfy the goal I have here, which is to independently validate the form contents and the form data as a whole? Is the correct answer to my confusion “read more”? 🙂
2020:04:11 19:17:40 alexmiller the preferred way is to have namespaced attributes with one meaning
2020:04:13 08:21:23 ashnur Hi, I would like to do some "spec driven development" in cljs, where should I start? Is it the same for clojure and clojurescript?
2020:04:14 13:56:53 colinkahn I think in general it’s similar. For UI applications the boundries that you want to focus your specs on are different of course. I consider anything coming over http and websockets a good candidate for checking with spec and all of my component props as well. If you’re using Reagent the later is easy because your components are already functions and you can just us s/fdef . Another consideration is I like to keep my specs separate from my other code on the F/E. Instead I write separate spec files that require the namespaces that I’m specing. Then you can make a preloads file that imports those spec files and runs st/instrument to get development-time feedback.
2020:04:14 14:07:43 ashnur I am not using reagent because it has no support of modern react, I am using helix.
2020:04:14 14:09:24 ashnur Thanks for the note on 'separate', I was just thinking about that, I saw a video with @U064X3EF3 where it was noted to not use these in production and I am not quite sure how to do that. I thought that the compiler would remove these when I build for production and I will not have to maintain two versions of my code
2020:04:14 14:19:57 ashnur How many years before I can do what you just did @U0CLLU3QT ? : D
2020:04:14 14:20:32 ashnur I can read the code, but all these ^s and ~ confuse the hell out of me.
2020:04:14 14:21:53 ashnur and yeah, I am fairly sure I can spec it, it's more about the context/reducer situation that I am somewhat unsure. I think I can do state transition functions and spec those and then I can create spec for the state as input and output before and after the transition.
2020:04:14 14:24:01 colinkahn The “separate” approach for me works because I agree these are more development-time tools and don’t really belong in my production source code. The compiler I believe will remove most of it (I’ve seen tricks where frameworks using Google Closure features to help with this as well) but I think certain parts of the spec library itself cannot be dead code eliminated (though don’t quote me on that, and probably better to try it yourself).
For having two versions, this isn’t necessary. For example:
app.components.cljs
2020:04:15 15:14:13 colinkahn Actually yeah, it’s under a different key
2020:04:14 10:00:06 Jim Newton I've never used spec before. Can someone help me with a recipe ?
for a given integer `n` and a given sequence, possibly heterogeneous, whether the sequence is of the form `n` many optional integers followed by exactly `n` integers? some analogous expression to "a?^8a^8" which I can substitute any fixed integer for 8 ?
2020:04:14 11:29:16 gfredericks is that the same thing as "between 8 and 16 integers"?
2020:04:14 18:15:36 zane How common is it to use spec (specifically s/or + conform + (`case` / multimethod / …)) to define polymorphic functions, and when would / wouldn't doing so be appropriate?
2020:04:14 18:31:34 alexmiller I have no way to judge frequency, but seems ok to me. It’s probably not the fastest option.
2020:04:15 09:28:40 flowthing I often test my specs in the REPL by doing something like (clojure.test.check.generators/generate (clojure.spec.alpha/gen ::foo)). Sometimes I mess up the spec such that the generator can't generate a value from my spec, and I get Couldn't satisfy such-that predicate after 100 tries.. Is there an easier way to debug cases like this than process of elimination? If the spec is fairly large, it's can be pretty time-consuming and difficult to find the offending spec. The exception and the stack trace aren't of much help.
2020:04:15 12:38:44 ashnur 'test in the REPL' means you type it in?
2020:04:15 12:41:35 flowthing Well, I evaluate the forms in my editor, but yeah.
2020:04:15 12:50:27 ashnur are you by chance using neovim and shadowcljs?
2020:04:15 12:50:49 flowthing Cursive, but I have dabbled with Conjure a bit.
2020:04:15 12:52:30 ashnur should've paid attention, it's been MONTHS
2020:04:15 12:52:46 ashnur thanks flowthing! : )
2020:04:15 15:12:21 ashnur well, nope.
2020:04:15 12:56:55 gfredericks somebody needs to write up a summary of what the such-that error means for spec users
2020:04:15 13:07:32 potetm @gfredericks It looks like @flowthing knows what it means. The question is: is there an easier way to find it than manual searching?
2020:04:15 13:07:51 potetm (I’m curious your answer to that as well.)
2020:04:15 13:08:41 gfredericks I don't think so. There's a feature added to such-that that enables spec to give better errors, but that was never implemented in spec
2020:04:15 13:23:19 alexmiller s/was never/has not yet been/
2020:04:15 13:25:42 gfredericks I was uncertain whether these situation with spec2 was similar or not
2020:04:15 13:48:12 alexmiller no changes for this yet
2020:04:15 13:48:29 alexmiller I did a little spike of it a while back but it was less obvious than I expected
2020:04:15 13:48:50 mpenet sorry to ask again: any eta for spec2 ?
2020:04:15 13:59:29 alexmiller no, kind of been dormant recently but we'll be back to it soon I hope
2020:04:15 17:02:51 seancorfield Re: such-that -- about the only thing I've found that helps avoid that is relentlessly running s/exercise on each spec as I write it, so that I trip over a "Couldn't satisfy" error at the earliest possible point and therefore have a really small surface of possibilities to investigate/debug. It's not perfect but it helps me, and it fits naturally in with a very tight RDD workflow that has you evaluating every piece of code as you write it.
2020:04:15 18:36:46 alexmiller they both ultimately call the same function so I would expect perf to be pretty similar right now. we have considered fast-pathing a separate valid? path in spec 2 though and I would expect that to be the faster option.
2020:04:15 19:08:05 ballpark Got it, thanks!
2020:04:15 19:11:57 alexmiller the big differences are that a) with valid?, you can fail fast and b) with explain, you have to track a lot of additional info and accumulate it for reporting
2020:04:15 17:43:14 vlaaad Isn't it sort of obvious what specs might throw such-that errors? The ones that use s/and with second condition rejecting most values from the set specified by first condition?
2020:04:16 06:49:55 flowthing Yes, it's often obvious, but not always, if the spec is large. An error message that points to the location where it originates would certainly be helpful. Anyway, I think the approach @U04V70XH6 proposed is the way to go for the time being.
2020:04:15 18:18:45 zane I think it's a good idea to avoid using the word "obvious" in this kind of context. Spec is pretty complicated, and people come to it from a variety of backgrounds.
2020:04:15 18:42:22 bbuccianti Could be possible to use spec to validate lambda expressions?
2020:04:15 18:42:41 bbuccianti I mean, I'm building a lambda calculus reducer
2020:04:15 18:43:29 bbuccianti And I'm thinking if would be possible to use property based testing in order to see if it's reducing properly
2020:04:15 18:43:56 bbuccianti But I never used spec and I don't know (yet) if that would be possible
2020:04:15 18:48:47 alexmiller s/fspec is the tool provided, but it's often more trouble than it's worth when using gen
2020:04:15 18:51:40 bbuccianti Do you mean than it doesn't ve of any value building such a thing?
2020:04:15 18:51:46 bbuccianti *be
2020:04:15 18:59:34 alexmiller spec is not particularly useful for checking functions as args in a deep way
2020:04:15 19:01:57 ashnur out of curiosity, is there anything that can do that? Sounds like something that I call the halting problem?
2020:04:15 19:14:46 nwjsmith @U0VQ4N5EE there are several property-based testing tools that can do this, they generate functions that satisfy the function's type
2020:04:15 19:20:59 ashnur Then I misunderstood what functions as args in a deep way means, sorry.
2020:04:15 19:18:35 ikitommi fast-path to valid? is a good idea. Any other new ideas for Spec2? New syntax for fn specs still wip?
2020:04:15 21:17:18 alexmiller too many ideas :) fn specs dormant atm, working on other stuff, but we'll get back to it
2020:04:15 21:54:50 seancorfield @kenny Why use #(string? %) instead of just string??
2020:04:15 21:55:18 kenny For demonstrating the bug, nothing else 🙂
2020:04:15 22:37:50 seancorfield Ah, so it works just fine with string??
2020:04:15 22:38:00 kenny Yep
2020:04:15 22:38:27 kenny I encountered this with a spec defined with a #(re-find ...)
2020:04:18 12:23:57 Joe Is this the correct way to spec a non-empty collection?
(s/def ::things (s/coll-of ::thing :min-count 1))
2020:04:18 13:34:19 shooit That would do it. The other thing that some to mind is from the sequential specs: s/+. It depends on context which is preferred. Just a non-empty collection of things? Your s/coll-of solution is it. A non-empty sequence of things as part of a larger pattern? Look to the sequential specs for their regex like functionality.
2020:04:18 17:32:15 prnc Spec rationale quote I’m pondering
> Invariably, people will try to use a specification system to detail implementation decisions, but they do so to their detriment. The best and most useful specs (and interfaces) are related to purely information aspects.
https://clojure.org/about/spec#_informational_vs_implementational
Trying to get clear on the distinction — what would be the canonical example of the offending type of use and how far you can push the boundary so to speak? Is this mostly about trying to enforce implementation a la types (an thus cussedness/rigidity etc.) ?
2020:04:18 18:00:21 alexmiller Clojure core functions are a good example
2020:04:18 18:01:58 alexmiller You can spec a function as talking PersistentVector (bad, concrete, may change) or as satisfying vector? or seq? or coll? or sequential? (depends on the fn of course)
2020:04:18 18:02:54 alexmiller And it’s common to try to spec what exactly it does now vs leaving open for future
2020:04:19 01:47:54 salam @alexmiller It seems like the Clojure Spec 1 alpha JAR is compiled against Java 1.5 (class file version 49). Is that intentional? It seemed a bit odd now that Clojure itself targets Java 1.8 (class file version 52).
2020:04:19 01:48:39 alexmiller Which version in particular?
2020:04:19 01:50:04 salam this is what's on my class path: spec.alpha-0.2.176.jar
2020:04:19 01:51:57 salam i took a look at the pom.xml (https://github.com/clojure/spec.alpha/blob/master/pom.xml) file for spec.alpha and the absence of <target.version>1.8</target.version> in the <properties> section seems to confirm that.
2020:04:19 01:52:50 alexmiller I’m not at a computer but certainly older versions of the parent pom could have produced that
2020:04:19 01:55:58 salam I see. I was experimenting with https://openjdk.java.net/jeps/310 and noticed the following warning and figured I should at least let you know:
OpenJDK 64-Bit Server VM warning: Pre JDK 6 class not supported by CDS: 49.0 clojure/spec/alpha/Spec
Preload Warning: Cannot find clojure/spec/alpha/Spec
2020:04:19 01:58:14 alexmiller There is a newer version that I suspect is newer bytecode. - 0.2.187
2020:04:19 02:00:14 alexmiller Spec was introduced in the same release that bumped to java 1.8 bytecode and it took a while for the contrib libs to catch up
2020:04:19 02:33:51 alexmiller You can add an explicit dependency on the newer spec.alpha version to override
2020:04:19 02:35:03 salam yep, i'll go with that. thank you.
2020:04:19 03:17:50 salam ok, reporting back here. it turns out [org.clojure/spec.alpha "0.2.187"] is compiled against java 1.5 as well.
2020:04:19 04:27:34 seancorfield Confirmed, if I'm reading this right. class version 49 for spec.alpha 0.2.187, but class version 52 for clojure itself. The latter is Java 8.
2020:04:19 08:13:06 practicalli spec version 1 or version 2, that is the question 🙂 My default assuming is spec 1
I am starting a new project, which I will mainly use spec for data structures I design and pass to functions as arguments. Its a data science visualisation project so there is a lot of data. I will also write specs for the functions that for the API for each namespace.
Just checking that I am not missing an obvious benefit of using spec 2. Thank you.
2020:04:19 08:38:13 mpenet 1 for sure. v2 seems still wip, especially fn annotations which apparently will be improved/modified
2020:04:19 08:38:17 didibus Spec 2 is experimental and subject to breaking change
2020:04:19 08:38:30 didibus So unless you are doing this for fun, go with Spec 1
2020:04:20 21:42:18 ashnur so I am reading docs and I am sure I will figure this out too, but someone could help me here a lot with a shortcut to speccing a hashmap that has 3 levels, the first is a fixed set of keys, second also a fixed set of keys, and the third is depending on the key either a string or a vector of keywors. so a valid example would be {:firstlevelkey {:label "string" :fields [:fieldone :fieldtwo]}} and I just want to say that on the first level only certain keys are allowed( one predefined set of keywords), second level always have to be both a :label and :fields and label always have to be string and the :fields always needs to be a vector of keywords from another set of predefined keys
2020:04:20 21:42:49 ashnur i don't need a full solution, just tell me please what to use so I can read up on those specifics because it's taking forever to sieve through 🙂
2020:04:20 21:43:27 ashnur I am trying to google
> clojure spec "-un"
and it's not going well
2020:04:20 21:53:03 ashnur the more I read the less I understand.
2020:04:20 22:06:41 seancorfield > I just want to say that on the first level only certain keys are allowed
This isn't something Spec favors -- it follows the concept of "open for extension". So you will need additional predicates with s/and to restrict keys (and remember that it's not really "idiomatic" to do that).
2020:04:20 22:07:40 ashnur Ok, good to know, so I won't force it if it's not expected
2020:04:20 22:11:41 seancorfield The whole spec is going to end up looking something like this:
(s/def :key/label string?)
(s/def :field/keys #{:fieldone :fieldtwo ,,,}) ; your set of predefined keys allowed
(s/def :key/fields (s/coll-of :field/keys :kind vector?)) ; maybe you want min and/or max counts too?
(s/def :second/level (s/keys :req-un [:key/label :key/fields]))
(s/def :top/keys #{:firstlevelkey ,,,}) ; available top-level keys
(s/def :top/level (s/map-of :top/keys :second/level))
2020:04:20 22:12:30 seancorfield If your set of keys isn't known at spec-time, it's a bit harder but that's probably a good first cut outline to start with.
2020:04:20 22:12:43 ashnur it's known 🙂 luckily
2020:04:20 22:12:44 seancorfield (with better names, of course)
2020:04:20 22:13:23 seancorfield That actually solved the restricted set of top-level keys, but at the expense of not requiring a minimum set either.
2020:04:20 22:13:30 ashnur the naming of keywords is arbitrary, right? so I could do just :label instead of :key/label, it just pays off to have very specific names of symbols so there is no collision?
2020:04:20 22:14:00 seancorfield Right. Qualified names help avoid collisions.
2020:04:20 22:14:46 seancorfield Only s/keys cares -- :req-un treats the qualified names as just their unqualified part (`:req` would pay attention to the qualifier as well).
2020:04:20 22:15:20 ashnur it's quite hard to grasp the pattern
2020:04:20 22:15:34 ashnur I need to experiment with this, thank you very much
2020:04:20 22:17:40 ashnur one more question that I think I know the answer for but still, if I want to spec anything, I always have to give a name to both the thing I want to spec and either use the definitions in place or create a named spec, right? So there is no way to write specs that would fit existing names automatically.
2020:04:20 22:21:43 seancorfield I'm not quite sure I'm following your question but I think the answer is "no, no way to do that in Spec 1".
2020:04:20 22:22:09 seancorfield Spec 2 does allow unqualified keys in a schema to have inline specs (predicates).
2020:04:20 22:22:52 ashnur in app namespace I define something like (def whatever "string") and then in app.specs namespace I say (s/def whatever string?) and there is some magic to match them
2020:04:20 22:23:03 ashnur probably a bad idea
2020:04:20 22:25:43 seancorfield Specs are in a completely separate "space" from regular Var names so you need to associate them yourself explicitly.
2020:04:20 22:26:03 seancorfield With functions, that's s/fdef but there's nothing for other Vars.
2020:04:20 22:26:22 seancorfield You need to use s/valid? or s/conform to "apply" a spec to a value at runtime.
2020:04:20 22:26:36 seancorfield (does that answer your question?)
2020:04:20 22:27:28 ashnur yes : )
2020:04:21 01:12:03 Alex I'm having some trouble with adding a spec for a keyword which is supposed to represent a function. I tried this
2020:04:21 01:12:20 Alex > Var clojure.test.check.properties/for-all* does not exist, clojure.test.check.properties never required
2020:04:21 01:12:54 Alex from trying to pass ::on-change to fdef
2020:04:21 01:14:43 Alex After reading the API docs, it seems fdef can only take a symbol naming a function.
My question is... how can I create a spec for the function that is passed as :on-change?
2020:04:21 01:17:52 robertfw It's funny you ask, I was just arriving to ask something similar. I have a spec describing a map, of which some keys are functions. I'd love to be able to specify that those functions should conform to a given fdef, but not sure how to accomplish that. As is, I just have the keys referring to the functions specced using fn?
2020:04:21 01:19:35 Alex Now I don't feel too silly asking 🙂
2020:04:21 01:26:02 alexmiller use ifn? not fn?
2020:04:21 01:27:20 alexmiller you can use fspec to do this as well, but there are some big caveats for generation
2020:04:21 01:27:43 alexmiller personally, I have not found it to be worth doing fspec over ifn?
2020:04:21 01:32:36 Alex @alexmiller Does ifn? allow you to specify the shape of the arguments that are passed in? I can't seem to find any examples doing so
2020:04:21 01:37:39 alexmiller no
2020:04:21 01:38:17 alexmiller you can do that with fspec (same args as fdef)
2020:04:21 01:45:23 Alex Tried a simple fspec but that doesn't seem to compile. Hmm
2020:04:21 14:09:06 Alex I get this error when I try that. Is this expected?
> Uncaught Error: Var clojure.test.check.properties/for-all* does not exist, clojure.test.check.properties never required
2020:04:21 14:09:43 kenny fspec uses test.check to ensure the correctness of the spec'ed input.
2020:04:21 23:10:08 Alex Thank you!
2020:04:21 07:23:13 ashnur how can I spec transient things like values of let bindings?
2020:04:21 09:06:37 Ben Sless It might be terrible but I recently had a similar issue, went for this solution:
2020:04:21 09:07:00 ashnur i am not sure about terrible but it's scary
2020:04:21 09:07:58 Ben Sless it is. If you can convince yourself the expr doesn't do any IO then you can sleep well imo
2020:04:21 09:09:57 Ben Sless In my case I had to create a qualified keyword who's name starts with a number. seems like :1foo is a valid kw but :user/1foo can't be read by the reader. (keyword "user" "1foo") works
2020:04:21 09:12:47 ashnur this is supposed to run in a browser 🙂
2020:04:21 09:13:45 Ben Sless ah
2020:04:21 07:33:05 ashnur I see the suggestion to use s/assert in docs, but isn't that also something that I should only do in development and not make it part of the code that might be shipped eventually?
2020:04:21 12:33:34 alexmiller Spec asserts can be turned off and even compiled out in prod code
2020:04:21 12:47:24 ashnur thanks, I realized there has to be some solution because there were blogposts about how to use it, but now I know where to look for it.
2020:04:21 09:02:03 Ben Sless Is it possible to spec/validate java collections? this naive example doesn't work
2020:04:21 12:32:48 alexmiller Spec does not cover Java collections so you’d have to pour one into a Clojure collection first
2020:04:21 17:35:32 Ben Sless This is probably a terrible idea, but I guess everything is possible if you try hard enough
2020:04:21 17:52:24 Ben Sless It would be nice, maybe in spec2, if the meanings of maps and sequences were relaxed a bit. But I'm coming at it from an esoteric use case
2020:04:21 17:52:44 alexmiller that seems way harder than converting the HashMap into a Clojure map
2020:04:21 17:53:05 alexmiller we do not have plans to support Java colls in spec 2
2020:04:21 17:56:51 Ben Sless That's the trivial case, in reality I might be dealing with an arbitrarily nested java collection 😞
2020:04:21 18:05:26 Ben Sless It's also unfortunate because it creates a bit of a mismatch between the validation mechanism and the applied predicates. In this example, get works on java.util.Map, so does seq, so conform* can do its work. The only "hurdle" is map?, which is a pretty stringent requirement. Why not expand spec a bit more towards reflecting intent and less implementation?
2020:04:21 18:15:55 alexmiller the intent is to validate clojure data
2020:04:21 18:16:13 alexmiller validating java colls was out of scope for us
2020:04:21 16:23:39 eval2020 What spec could validate that a map contains at least one true value?
2020:04:21 16:25:23 alexmiller any function that takes a value can be a spec
2020:04:21 16:25:46 alexmiller so rephrasing - what function could validate that a map contains at least one true value?
2020:04:21 16:25:55 alexmiller write that function and you're done
2020:04:21 16:28:28 eval2020 Ah, I was overthinking this - thanks
2020:04:22 01:08:36 ashnur should spec have their own ns an import app ns? or should the app import the spec ns? I am having this issue that circular dependencies seem unavoidable
2020:04:22 01:12:19 seancorfield @ashnur The answer is "it depends". You should be able to find a partial ordering of dependencies that would allow you to split things into different namespaces -- probably three, so you have an "app" ns, a "spec" ns, and a "predicate" or "utility" ns that contains things the specs depend on, and the app can also depend on. But it may not be worth doing that analysis so having the specs in the same ns as your app code may just be easier.
2020:04:22 01:13:40 seancorfield If you're writing code that you want to be able to run on older versions of Clojure(Script) that don't include spec, you have to break them out into optional namespaces so that users can opt in if they want the specs -- but that's really only for library writers.
2020:04:22 01:14:28 ashnur I am running shadow-cljs
2020:04:22 01:14:40 seancorfield At work, I try to have the data specs in a separate namespace (and do the work to untangle any problematic dependencies) but the function specs will generally go in the same namespace as the functions they are for.
2020:04:22 01:14:57 ashnur I initially imagined having files for data/specs/components/react-app all separated
2020:04:22 01:15:46 ashnur can I have multiple files all in the same namespace?
2020:04:22 01:21:30 seancorfield The simple answer to that question is no. The complex answer is there are ways to spread a namespace across multiple files but I don't think it will solve the problem you have here.
2020:04:22 01:22:05 ashnur ok, no it is 🙂 I want to keep it simple
2020:04:22 01:22:47 ashnur I must be doing something silly again because I am unable to refer to my s/defs from the spec file/ns
2020:04:22 01:24:29 seancorfield Can you share some code to illustrate the problem? Is this in a project up on GitHub?
2020:04:22 01:29:51 ashnur in spec file I wanted to do something like
(s/def :my.specs/one-spec identity)
then in app file ns require
[my.specs :as myspecs]
2020:04:22 01:31:55 ashnur it's closed source but I can make a new repo with completely new code, it might just take a couple of weeks before i have the time
2020:04:22 01:31:59 ashnur 😄 only half kidding though
2020:04:22 02:00:38 seancorfield OK. And what can't you do in the app file ns?
2020:04:22 02:01:08 seancorfield (sorry for the delay in responding -- was dealing with an issue someone just opened on clj-new that I needed to repro!)
2020:04:22 02:04:06 seancorfield In you app file ns, after that require, you should be able to reference that spec either as :my.specs/one-spec (i.e., its fully-qualified name) or ::myspecs/one-spec using the auto-resolve syntax which will expand ::myspecs to :my.specs using the alias of the namespace... the latter requires that the actual spec name really matches the spec ns in the qualifier. (and you could just (s/def ::one-spec identity) in the spec ns to have the spec automatically match the ns of the namespace it is defined in).
2020:04:22 02:04:23 ashnur I feel bad if you apologize for such stuff, I realize people have lives beyond what is visible here : )
2020:04:22 02:04:41 ashnur I had to do (def (s/def ... and it worked
2020:04:22 02:04:51 ashnur for some reason I thought just (s/def would be enough
2020:04:22 02:05:34 ashnur I am also in utter confusion about how keywords work, not just in spec but other places. I know it's a symbol that retuns itself, but apparently it also can hold specs
2020:04:22 02:06:47 seancorfield s/def updates a registry behind the scenes which is essentially a map from keywords to spec objects. The keywords don't "hold" specs -- the association is done separately.
2020:04:22 02:07:00 ashnur gotcha
2020:04:22 02:08:37 seancorfield I have no idea what (def (s/def ...)) would do... but it's definitely not the right thing to do.
2020:04:22 02:15:45 ashnur how can I use :require :refer to import a spec definition?
2020:04:22 02:16:06 seancorfield Ah, probably because of function specs: s/fdef uses the fully-qualified function name (symbol) so that's probably why s/def will accept a symbol.
2020:04:22 02:16:14 seancorfield You can't refer in a single spec.
2020:04:22 02:16:38 seancorfield Once you've loaded the ns in which specs are defined, they are globally available.
2020:04:22 02:16:49 seancorfield They're not like Vars or functions.
2020:04:22 02:17:26 seancorfield So you could require the specs ns in whatever you app entry point is and you would have access to those specs (as keywords) everywhere in your program.
2020:04:22 02:21:12 ashnur I can't even require them at this point
2020:04:22 02:21:41 ashnur how to "load the ns", I usually just do :refer or :as, neither which seems to work here
2020:04:22 02:22:21 seancorfield Just require the namespace like any other. Then the specs are in the registry and available globally.
2020:04:22 02:22:35 ashnur oh ok, so i can just write the ns :as, but then i have to completely ignore it
2020:04:22 02:23:57 ashnur and instead always have to read the file of the spec, because ns-publics don't list it
2020:04:22 02:24:16 seancorfield Yeah, if you're not using the :: auto-resolve syntax, you don't even need :as
2020:04:22 02:24:16 ashnur and it's available via some (black, that is, unobservable) magic
2020:04:22 02:24:41 ashnur but how? just (:require [my.specs])?
2020:04:22 02:24:45 ashnur ok 🙂
2020:04:22 02:24:51 ashnur thanks again
2020:04:22 02:25:27 ashnur when I think I understand something, it turns out that thing is specific to that particular area, and something else using the same syntax behaves completely differently
2020:04:22 02:28:39 ashnur the problem, which you have to realize is a problem, that I've had this page open for days and I have read it through from top to bottom and bottom to up several times
2020:04:22 02:29:38 seancorfield Don't worry, sometimes it taking a lot of reading the Clojure docs before it actually makes sense and you get to see a big picture!
2020:04:22 02:30:09 ashnur I see the big picture, I just don't see the details.
2020:04:22 02:30:35 ashnur 🙂 i've been trying to use clojure for 8 years at least
2020:04:22 02:31:16 ashnur this time I will succeed because finaly I can do stuff like this, waking up at 1:30 am, then suffering for an hour before you come to help 🙂
2020:04:22 02:31:26 ashnur oh and because I can use shadow-cljs
2020:04:22 02:31:38 ashnur so the tooling is solved more or less
2020:04:22 02:33:14 seancorfield I hear great things about shadow-cljs -- when (if) I ever go back to trying to do ClojureScript, that's the path I'll take.
2020:04:22 13:58:23 lambdam Hello everyone,
I am defining some specs in a .cljc file, some only exist in a #?(:clj (do ...)) , others in a #?(:cljs (do ...))
I get an error on a keyword at parsing time:
Here the ::const namespace is declared only for .clj parts of code (:require ... #?(:clj [... :as const]))
When I split this .cljc into two files .clj and .cljs, there is no error.
Would it be related to spec or is it a more general error? It is the first time that it happens.
I really don't have any clue.
Thanks
2020:04:22 14:04:14 alexmiller it's more general - ::const relies on having a clojure runtime namespace context to resolve it - I'm not sure what the exact problem is without more context but generally you'll want to avoid these in reader conditionals
2020:04:22 14:05:01 alexmiller you should be able to just use :foo/const or whatever instead
2020:04:22 14:41:00 lambdam Thanks @alexmiller it works.
It's not as convenient as the shortcut version but it's still better than two .clj and cljs files.
If you want more info, don't hesitate to ask for. I'll send in a private message.
2020:04:23 17:07:55 fricze I’m curious about specing functions. There’s probably something I don’t understand, but why function specs made with fdef are not reusable? I can easily imagine, and even find in my codebase, few functions that share the same spec, but still fdef have to be called with all the arguments for each function. If anyone has some insight into this decision, I’d love to hear something 🙂 thanks
2020:04:23 17:15:42 kenny Not sure if I'm following what you mean but you can do this:
(gen/sample (s/gen (s/select {:a int? :b keyword?} [:a])) 5)
Error printing return value (ExceptionInfo) at clojure.test.check.generators/fn (generators.cljc:435).
Couldn't satisfy such-that predicate after 100 tries.
I'm on 8498f9cb352135579b6d3a0a5d15c40e5c2647ce , which seems to be the latest commit on master.
Sorry if it's already been reported before. I don't really know where to look for stuff like this.
2020:04:27 02:11:53 alexmiller yeah, there's a bug in this area
2020:04:27 14:28:16 dspiteself I am sure you have already seen this, but unqualified keys are not being looked at for closed specs in spec-alpha2
2020:04:27 14:32:16 dspiteself Creating Specs Programmatically is so pleasant. I have really been enjoying spec-alpha2.
2020:04:27 14:42:48 alexmiller I'm sure there are any number of bugs in the newer stuff right now, it's not done
2020:04:27 14:50:37 dspiteself yep I was just checking if it would be valuable to report it. Thanks for the awesome work.
2020:04:28 09:23:28 arnaud_bos There's this line in spec-alpha2's wiki page on "Schema and select" (emphasis mine):
> General form: (s/select schema selection)
> • schema (required) - can be a registered schema name, schema form (like s/union), or literal schema
But I can't get the "schema form" to work.
(in-ns 'user)
=> #object[clojure.lang.Namespace 0x54029c23 "user"]
(s/def ::a int?)
=> :user/a
(s/def ::b keyword?)
=> :user/b
;; Literal schema: OK
(gen/sample
(s/gen
(s/select
[::a ::b]
[*]))
2)
=>
(#:user{:a -1, :b :B/d}
#:user{:a -1, :b :Q/r})
;; Schema name: OK
(s/register ::ab (s/schema [::a ::b]))
=> :user/ab
(gen/sample
(s/gen
(s/select
::ab
[*]))
2)
=>
(#:user{:a -1, :b :B/_}
{})
;; Union name: OK
(s/register ::ba (s/union [::a] [::b]))
=> :user/ba
(gen/sample
(s/gen
(s/select
::ba
[*]))
2)
=>
(#:user{:b :./l?}
#:user{:a -3})
;; s/schema form: Err
(gen/sample
(s/gen
(s/select
(s/schema [::a ::b])
[*])))
Execution error (IllegalArgumentException) at clojure.alpha.spec.protocols/eval1814$fn$G (protocols.clj:20).
No implementation of method: :keyspecs* of protocol: #'clojure.alpha.spec.protocols/Schema found for class: clojure.lang.PersistentList
;; s/union form: Err
(gen/sample
(s/gen
(s/select
(s/union [::a] [::b])
[*])))
Execution error (IllegalArgumentException) at clojure.alpha.spec.protocols/eval1814$fn$G (protocols.clj:20).
No implementation of method: :keyspecs* of protocol: #'clojure.alpha.spec.protocols/Schema found for class: clojure.lang.PersistentList
Looks like I've messed something up wrt "symbolic" vs "object"?
2020:04:28 12:52:36 alexmiller No, just bugs I think
2020:04:28 12:53:47 alexmiller Spec 2 is not ready for use
2020:04:28 14:32:09 arnaud_bos Yeah, sorry to make you feel like you have to repeat it again and again. I'm just toying around 🙂
2020:04:28 14:32:52 alexmiller the whole schema/select impl is due for a rewrite, just don't have time to work on it right now
2020:04:28 14:37:08 arnaud_bos I see, thank you 🙂
2020:04:28 14:35:44 flipmokid Hi all, I'm playing around with the lastest spec alpha 2. I'm not sure if I'm doing something wrong but:
2020:04:28 18:31:41 Frankline Apiyo Hey guys 👋 , I'm tring to get started on using clojure.spec and I hit an unexpected/frustrating roadblock...
2020:04:28 18:31:52 Frankline Apiyo
user=> (s/exercise false?)
Execution error (FileNotFoundException) at user/eval1428 (form-init5827540222428447763.clj:1).
Could not locate clojure/test/check/generators__init.class, clojure/test/check/generators.clj or clojure/test/check/generators.cljc on classpath.
2020:04:28 18:32:27 Frankline Apiyo This error is thrown for any spec for which I call s/exercise
2020:04:28 18:32:40 Frankline Apiyo Any help with be appreciated
2020:04:28 18:33:12 jaihindhreddy You need to add test.check library as a dependency to your classpath
2020:04:28 18:33:36 Frankline Apiyo cool... I'll go ahead and do that
2020:04:28 18:33:37 jaihindhreddy spec uses it for generation
2020:04:28 19:25:41 shooit I don’t know if there is a spec that you can use but you could alternatively call seq on your eduction to turn it into a LazySeq which would then pass the spec
2020:04:28 19:29:24 dominicm @kenny I'm not sure, but I don't think eduction returns a collection, it returns a reducible :). That doesn't help but it does indicate that coll-of probably isn't the right tool.
2020:04:28 19:30:10 kenny Right. I'm curious what folks use to validate against it. Hmm, I could do that @shewitt!
2020:04:28 19:32:10 alexmiller eductions are suspended computations so there is no data to validate without forcing it
2020:04:28 19:34:36 kenny This use case is for in a test so I think @shewitt's idea solves it. In general, it probably wouldn't make sense to call valid? on an eduction for that reason, right? Even if checked with s/every, it'd still need to recompute *coll-check-limit* when you finally use the eduction elsewhere.
2020:04:28 19:42:25 alexmiller Right
2020:04:30 02:24:08 bbrinck I noticed that instrumentation doesn’t work for certain core functions. Is this related to the inline metadata?
2020:04:30 03:54:09 alexmiller you will also find that adding specs to core functions will not be checked in calls from one core function to another as core is compiled with direct linking
2020:04:30 15:16:15 bbrinck Thanks! That’s good to know.
2020:04:30 15:20:57 bbrinck The reason why the “inline” limitation comes up is I’m thinking about ways to use instrumentation to help provide some different error messages when using core functions, but this means that some functions cannot be instrumented.
2020:04:30 15:22:09 bbrinck I wonder if there’s a way around this. I can also see how many functions are actually inlined. Maybe it’s not a huge amount in practice.
2020:04:30 15:23:38 bbrinck I may be misremembering, but I seem to remember some mention of spec2 instrumentation changing things. Do you have any idea if this inline limitation will remain in the new instrumentation?
2020:04:30 15:23:54 alexmiller no difference in how instrumentation is implemented
2020:04:30 15:33:26 bbrinck OK, thanks. A quick search in clojure.core indications about 85 cases of :inline. Maybe it’s not a huge deal to just skip those functions or maybe there’s a non-instrumentation-based way to help with errors. I’ll think more.
2020:04:30 15:39:28 alexmiller the biggest thing to be concerned about is probably performance. Some of the tests they did with https://github.com/borkdude/speculative made instrumented core pretty much unusable.
2020:04:30 17:58:55 bbrinck Agreed. I tried instrumentation on my test suite and it was way too slow. I have an idea about a different way to instrument that might be better wrt performance (basically by only doing spec checking if there is an error) but the inline limitation will require some additional thinking
2020:04:30 18:16:17 alexmiller Sounds like a good idea
2020:04:30 16:18:38 ballpark To use spec.alpha2, do I need to clone the repo? I don't see it in maven-central
2020:04:30 16:20:23 alexmiller it has not yet been released
2020:04:30 16:20:42 alexmiller there are instructions in the readme on how to use it as a git dep
2020:04:30 16:21:25 alexmiller it is still a wip, actively changing, and buggy. you have been warned. :)
2020:04:30 16:21:52 ballpark understood, thanks!
2020:05:03 20:02:22 oskarkv I tried one of the examples on http://clojure.org, namely this one:
If I pass in a string, I get a complaint. But as you can see, I added a 1 in the body of adder so that the :fn check wouldn't hold. But nothing happens when I call adder. What's going on?
2020:05:09 19:22:50 dvingo That was my issue! I was trying to use a cljc file and updated them to separate cljs and clj files. it's working now 🙂
2020:05:10 19:49:27 Amir Eldor Hello, rather new to this. How would you instrument specs only in development? I'm using Leiningen, I guess there's some way to check what profile we're running? If so, would you do this (if) at the bottom of each namespace with specs??
2020:05:10 21:37:28 seancorfield @amir In several of my projects, I have the tests run with instrumentation in place, so each test file has a test fixture that turns instrumentation on (or just a top-level form that does it as the test file is loaded). Since I tend to be loading/running tests a lot during development, that usually means I have instrumentation turned on during development -- but it's easy enough to just eval an instrument call from a comment in your source file.
2020:05:10 21:38:42 seancorfield Not sure what profiles have to do with that. If I'm developing, I'm working with a REPL. If I'm testing, I can have the tests turn instrumentation on. Otherwise the code won't turn instrumentation on by default, which is the behavior I want.
2020:05:14 17:10:25 dvingo this is super useful, thanks for the post! I have a use case where I want to generate some crud operations as well for some map specs - do you have any pointers for functions you used to parse the specs? I see s/describe and s/form did you use those or something else?
2020:05:14 17:38:20 seancorfield Just those. To get at the primary list of required/optional keys.
2020:05:14 17:41:31 dvingo perfect, thanks!
2020:05:11 09:50:57 Petrus Theron How do I use Spec with core.async? E.g. how do I spec a function that returns a channel where I'll put! another spec on? Or do I need to wrap it in some kind of record for this?
2020:05:11 16:45:21 jkxyz you can spec that something is a Channel, but that’s not very useful, so you probably want to spec the values at the points where you put/take them
2020:05:16 20:19:17 Petrus Theron Yes, I realized that I could spec the channel transducers that interpret incoming values on put!.
2020:05:11 22:44:11 rafael I'm seeing strange behavior in spec alpha2 (latest github commit). If I define a schema using the map form, to specify non-namespaced keywords, refer to it from another schema, and later try to generate data from a selection of latter schema, I get an "Unable to resolve spec" exception:
; Context
(s/def ::foo int?)
(s/def ::inner (s/schema {:foo ::foo}))
(s/def ::outer (s/schema [::inner]))
; Works as expected:
(s/exercise ::outer)
(s/exercise (s/select ::inner [*]))
; Fails with "Unable to resolve spec: :foo"
(s/exercise (s/select ::outer [*]))
2020:05:11 22:44:34 rafael Is this expected, should I change something in my code, or is it a bug that I should report somewhere?
2020:05:11 23:15:05 seancorfield Spec 2 is very much a work in progress and isn't ready for folks to use -- and, yes, it has numerous bugs in it @rafael
2020:05:11 23:25:58 rafael Yeah, so far I'm finding workarounds. I think I can work around this one as well with s/keys
2020:05:12 02:09:03 seancorfield As long as you're not building anything with it that is more than a personal toy project @rafael...
2020:05:12 09:22:16 fmnoise hi everyone, is it possible to add meta to spec?
(s/def ::my-custom-int ^{:error-fmt #(str % " is not valid int")} int?)
when I call (meta (s/get-spec ::my-custom-int)) it doesn't contain my meta
2020:05:12 09:26:58 hkjels @fmnoise I don’t know the answer, but I’m pretty sure you’re checking the wrong thing. Try (meta ::my-custom-int)
2020:05:12 09:31:26 vlaaad
(s/def ::my-int (with-meta (s/spec int?) {:err-format #(str % " is not a valid int")}))
2020:05:12 09:32:02 vlaaad @fmnoise then (meta (sp/get-spec ::my-int)) will work
2020:05:12 09:56:18 fmnoise thanks @vlaaad, this works!
2020:05:12 12:45:59 alexmiller While this may work, please that it won’t work on all specs and some spec transformations like with-gen will not preserve meta. This is more accidental than intentional
2020:05:12 12:49:09 alexmiller Adding meta and doc string support to specs is the most requested thing in the tracker and I expect we will add it in spec 2
2020:05:12 18:16:32 sgerguri I have a design question - I have a Kafka Streams topology where I use conformance to automatically sanitise input, then split the stream into two - those where conformance resulted in ::s/invalid and a happy path. In the happy path, I call a pure function that transforms the conformed message.
This transformer has a function spec attached to it, but because I have already conformed the input according to the arg spec before calling the function, it will fail the conformance check.
2020:05:12 18:17:08 sgerguri I seem to be facing two options - either remove the function spec, or create yet another spec for the already-conformed input, a "derived spec" of sorts. Neither really feels satisfactory.
2020:05:12 18:18:03 sgerguri Yet another alternative would be to simply validate with s/valid? earlier in the stream, then conform inside the pure function - which feels like the right thing to do, but it also feels like I am validating the data twice effectively. What would be the cleanest solution here?
2020:05:12 18:59:34 seancorfield @sgerguri The approach we've taken at work is to treat that initial Spec layer as a boundary and assume it maps from "external data format" to "internal data format" and is applied consistently. Thus everything "inside" is Spec'd in terms of the (already conformed) data produced at the boundary.
2020:05:12 19:00:37 sgerguri So if I understand it correctly you have a second spec layer for the internal representation, that represents things that have been conformed, correct?
2020:05:12 19:07:42 seancorfield Yeah, an API Spec, a Domain Spec, and actually a Persistence Spec. The API Spec is conforming, from external data (often strings) to internal data (numeric, Boolean, date, etc). The Domain Spec describes the data formats used inside the API. The Persistence Spec describes the data that goes into the database (flat hash maps with JDBC-compatible types). We don't use all three in all cases, but it seems to have become a useful way to split things up -- and it clearly identifies two boundaries (API input and database output).
2020:05:12 19:09:06 sgerguri That's excellent, thanks. I have used a similar approach in another service, the only thing I'm struggling with in this one is the fact that the internal layer is very thin and quite close to the input layer, thus it feels like duplicating the specs, but might be worth it for the greater clarity.
2020:05:12 19:10:26 mpenet spec-coerce is quite good for this too
2020:05:12 19:11:05 seancorfield I would caution against leaning too heavily on conforming with Spec -- I don't really like what spec-coerce enables.
2020:05:12 19:12:07 mpenet By default it will do the right thing to coerce x to your spec expeceted values, and the other way around (serializing to db type, ex keyword -> string) just requires to override those cases
2020:05:12 19:12:36 mpenet No spec duplication that way
2020:05:12 19:12:46 mpenet No polluted specs either (no conform abuse)
2020:05:12 19:13:30 sgerguri I have used it for some fairly tricky transformations in the past, but I've found it really works well for emulating pattern matching along with multimethods by tagging individual cases through an s/or. In this particular example I only wanted to trim whitespace but for some reason am having trouble fitting it neatly in without overhauling one part of the service or another.
2020:05:12 19:15:06 sgerguri This particular situation also left me wondering whether conformance should either be the final station in your transformation/validation stack or whether it should yield something that also has a spec for it (without further conformers), like @seancorfield described.
2020:05:12 19:15:45 sgerguri In the grand scheme of things I could just trim the fields I need to but with spec around that just feels like the wrong approach.
2020:05:12 19:16:10 seancorfield There are folks on both sides of that decision 🙂
2020:05:12 19:17:09 seancorfield Some feel that input should be cleaned and parsed first, then checked for validity with Spec (and therefore next to no conforming). Others feel that input conformance to valid data is a reasonable use of Spec.
2020:05:12 19:17:51 mpenet I used to have dual specs like you mention, but it requires more work and is more brittle imho
2020:05:12 19:18:03 mpenet Then as you say ymmv
2020:05:12 19:18:17 seancorfield I am slightly on the conforming side of center. spec-coerce is quite a lot further out on the conforming side. I think Alex (and maybe others at Cognitect) are on the non-conforming side of center.
2020:05:12 19:18:56 alexmiller I actually like spec-coerce :)
2020:05:12 19:19:17 seancorfield Really? I was sure you had complained about it in principle in the past? 🙂
2020:05:12 19:19:48 alexmiller I don't like spec-tools
2020:05:12 19:21:50 sgerguri Food for thought. Thank you everyone, always a pleasure to come here for another opinion on how folk do things. 🙇
2020:05:12 19:26:39 alexmiller spec-coerce builds a separate registry of coercions that leverages specs, which I think is a good approach
2020:05:12 19:27:20 alexmiller I haven't used it in anger but that seems in line with how I would approach it
2020:05:12 19:29:19 mpenet It's gotten better lately, it was missing multi-specs, merge & tuple until recently but now it's quite feature complete
2020:05:12 19:30:16 mpenet It's also 300ish lines of code, easy to modify, fit to your taste if needed
2020:05:12 19:43:51 seancorfield @alexmiller Ah, thanks for the clarification. Then I suspect I'm getting the two libraries confused and maybe I should look at spec-coerce in more detail? 🙂
2020:05:12 21:40:53 seancorfield Wow, I don't even remember that. Looks like I'd started to look at spec-coerce as a possible alternative to our existing "web specs" at World Singles Networks -- and I guess by the time that PR got accepted I'd decided not to go down that particular path for some reason...
2020:05:12 20:31:02 Amir Eldor Hello, I hope it's ok to post some code. I'm trying to spec a function that has ehmm, this optional arguments thing? I'm not sure how it's called in Clojure. I'm having some trouble with the spec for it.
Right now I specifically use (st/instrument) in a test as suggested and the following code breaks with the following exceptions:
(ship (:id p)) ; (:id p) is surely a uuid
|| :cause "class java.util.UUID cannot be cast to class java.lang.Number (java.util.UUID and java.lang.Number are in module java.base of loader 'bootstrap')"
2020:05:12 20:32:36 Amir Eldor It seems like it's related to the ::speed thing, being an integer. I must have made something funny in the spec, if someone can notice. Thanks!
2020:05:12 20:46:14 Amir Eldor Ah yes, thet exception I gave is invoked on the s/def of ::speed 😕
2020:05:12 21:35:26 fmnoise what is (random-id) ?
2020:05:12 21:40:14 Amir Eldor It's my own function, which calls (UUID/randomUUID) wth java.util.UUID
2020:05:12 21:43:53 Amir Eldor The code itself works without the spec, I must be defining something badly
2020:05:12 21:45:56 fmnoise what about default-ship-speed?
2020:05:12 21:46:58 Amir Eldor (defonce default-ship-speed 20)
2020:05:12 21:51:08 seancorfield What is DateTime?
2020:05:12 21:51:55 Amir Eldor Though I use clj-time, so I might be wrong here when I think of it
2020:05:12 21:52:12 seancorfield 'k... Joda Time is deprecated. If you're on Java 8 or later, you should use Java Time really.
2020:05:12 21:52:19 seancorfield clj-time is also deprecated.
2020:05:12 21:52:24 seancorfield (for the same reason)
2020:05:12 21:52:51 Amir Eldor Oh, thanks for letting me know. I'll google for Java Time
2020:05:12 21:53:04 fmnoise (s/def ::speed (fn [v] (and (int? v) (> v 0))))
this definition works
2020:05:12 21:54:07 fmnoise but dunno why it assumes uuid there by default when doing coercion
2020:05:12 21:57:37 seancorfield The problem is that all three optional arguments are optional independently
2020:05:12 21:58:15 seancorfield So dest-planet-id could be omitted and the speed and departure-time could still be provided.
2020:05:12 21:59:09 seancorfield In other words, it tries to check (ship src-planet-id src-planet-id) against a signature that is essentially [uuid? #(> % 0)] and that's causing the exception.
2020:05:12 22:00:24 seancorfield And that's why adding (and (int? %) ...) into ::speed "works" -- it guards the > operation from being called on non-numeric values.
2020:05:12 22:01:07 seancorfield Because then the ::speed spec successfully fails to match (instead of blowing up) and Spec goes on to try the other options.
2020:05:12 22:01:44 seancorfield So...
2020:05:12 22:02:48 seancorfield I'd suggest spec'ing ship a bit differently, perhaps using s/alt over the four arities, each spec'd with no optional parameters.
2020:05:12 22:03:10 seancorfield Or at least across the first three and leave just :departure-time as s/?
2020:05:12 22:03:37 Amir Eldor Thanks a lot! However I don't fully understand how this happens:
> dest-planet-id could be omitted and the `speed` and `departure-time` could still be provided.
2020:05:12 22:05:16 Amir Eldor I'm sorry I have to go now as it's getting late. I hope it's ok if I ping you tomorrow in this thread if I still have some trouble. Thanks!
2020:05:12 22:09:18 seancorfield You have src dest? speed? time? -- each of those three are optional, so each could be omitted while the others are passed.
2020:05:12 22:09:44 seancorfield so src speed is "valid", as is src time or src dest time or src speed time.
2020:05:12 22:09:54 seancorfield That's what your fdef says.
2020:05:12 22:10:56 seancorfield So when Spec sees (ship src-planet-id src-planet-id) it's going to try src speed, src time, and src dest in some random order.
2020:05:12 22:11:26 seancorfield Since your speed spec just tries to compare the value > it will throw an exception if passed a non-number: which a UUID is.
2020:05:12 22:11:49 seancorfield Because Spec encounters an exception, it won't try the other options -- it just propagates the exception.
Is there a way to get rid of the repetition here without introducing more nesting and keywords in the conformed data?
2020:05:13 14:12:07 oskarkv There are always macros of course but I was hoping for something easier.
2020:05:13 14:13:03 alexmiller doesn't the first one include the second?
2020:05:13 14:16:32 oskarkv But I want a spec for data where a var is forbidden. And one where it's optional.
2020:05:13 14:16:45 alexmiller ah
2020:05:13 14:16:45 oskarkv I'm not sure I understand what you meant.
2020:05:13 14:17:16 alexmiller so, you can create a new spec that is an s/cat that just contains the common portions, then reference it in both ::fact specs
2020:05:13 14:22:41 alexmiller can't say I've done this before, might need to tweak that predicate more
2020:05:13 14:23:02 oskarkv Hm, gotta read up on &. Thanks!
2020:05:13 14:24:16 alexmiller & just lets you and arbitrary predicates into the regex spec
2020:05:13 14:24:31 oskarkv Ah, I see. Could work. Thanks!
2020:05:14 19:08:51 Amir Eldor How do you define a map spec with keys of similar spec? e.g.:
(s/def potato-name string?)
(s/def potato-happiness number?)
;; This is a potato:
{:name "potato name"
:happiness-1 12
:happiness-2 54}
Similarly, I see that (s/keys) requires a fully qualified name for each key and it's strange to me. What if I got two specs with the same key but different specs?
2020:05:14 19:49:38 alexmiller the idea behind spec is to give attributes global semantic meaning, by using namespaced attributes to differentiate and not do that
2020:05:15 18:54:45 Amir Eldor So it's not intended, for example, to use things like ::mygameobject-id on other namespaces of specific 'gameobject' each?
Among the things I'm playing around with is having a ::gameobject-id defined in some-ns.game and use that in some-ns.game.dog and some-ns.game.bat, each of which has an :id which is a gameobject-id. Am I doing things in a non-Clojure way?
2020:05:15 19:06:36 alexmiller :: means to auto-resolve the keyword in the context of the current namespace
2020:05:15 19:07:15 alexmiller so if you're using ::id in different namespaces, you'll get different keywords specific to each namespace, and there is no conflicting name
2020:05:15 19:08:09 Amir Eldor and if I want to use the same spec definition for each? just use a common function?
2020:05:15 19:08:36 alexmiller you can (s/def ::id ::common/gameobject-id) in each namespace
2020:05:15 19:08:52 alexmiller so have one common spec and then make namespaced specs that refer to it
2020:05:15 19:12:12 Amir Eldor I was trying to do just that and it didn't work, and not it works. Great. Maybe I had a funky repl state before :man-shrugging:
Thanks!
2020:05:15 19:12:24 alexmiller good!
2020:05:15 19:14:20 Amir Eldor I see that when I do something like
2020:05:15 19:14:46 Amir Eldor Then the thingy gets an :id property, and not { :common/gameobject-id "whatever" }
2020:05:15 19:14:51 Amir Eldor Which is good for me
2020:05:15 19:15:38 Amir Eldor (or maybe :gameobject-id, I am mixing up some namings between this chat and my actual code)
2020:05:15 19:16:49 alexmiller it doesn't "get" a :gameobject-id property, it describes a map expected to have a :gameobject-id key which matches the :common/gameobject-id spec
2020:05:15 19:21:40 Amir Eldor I meant that when I conform something to ::thingy, the keyword in the conformed map is { :gameobject-id "something" } and not { :common/gameobject-id "something" }. I guess it omits the namespace from the keyword?
2020:05:15 19:21:58 alexmiller :req-un means "required unqualified key"
2020:05:15 19:22:03 alexmiller (as opposed to :req)
2020:05:15 19:22:28 alexmiller so more importantly it's validating the incoming map expecting an unqualified key
2020:05:15 19:22:40 alexmiller conforming just follow that
2020:05:15 19:23:16 Amir Eldor I seeeee I made a thingy2 now with :req and it wants the fully qualified key
2020:05:15 19:23:53 Amir Eldor Interesting and cool.
2020:05:15 19:40:11 Amir Eldor Do I have to require common for using :common/gameobject-id or something?
2020:05:15 19:43:28 Amir Eldor Oh nevermind.
2020:05:15 19:52:04 alexmiller no
2020:05:16 01:02:47 Kyle Brodie Whats the best way to spec numbers as strings so something like "12.53" will be generated?
2020:05:16 02:22:35 seancorfield It has specs that accept strings or longs and generate as strings that are parseable as longs. Should give you something to work from.
2020:05:16 02:49:00 Kyle Brodie @seancorfield Thank you! That library looks very helpful. I might have to write my own date though because mine are "YYYY-MM-DD". Unless "yyyy-M-d" is the right format for zero padded in java-time
2020:05:16 03:09:45 seancorfield Yeah, it'll do zero-padded. And you want yyyy and d lowercase for dates.
2020:05:16 11:41:09 Kira McLean Hi there.. I’m just getting started with spec, have a somewhat noob question that I’m having trouble finding an answer to: Is it possible to have two different specs for the value of a given un-qualified keyword? I.e. I have a map and I know that the values might have different requirements in different contexts, but I would like to keep re-using the same key. So I have specs like this (`fs` is [datoteka.core :as fs] ):
(s/def ::path fs/path?)
(s/def ::html-path (s/and ::path (partial matches-ext "html")))
(s/def ::content string?)
;; This one is great -- checks my maps like {:path ,,, :content ,,,}
(s/def ::page (s/keys :req-un [::path ::content]))
;; Here is the issue: I want to check a map that also has the keys [:path
;; :content], not one with an html-path key
(s/def ::html-page (s/keys :req-un [:path ::html-path ::content]))
In the ::html-page spec I want :path to match the html-path spec but for the key to still be called :path . Is this possible?
2020:05:16 11:51:20 ghadi If you define two specs, :a.b.c/path and :x.y.z/path you can use them to spec unqualified keys in different maps
2020:05:16 16:14:19 Kira McLean Makes perfect sense, thanks.
2020:05:16 18:17:12 David Pham Are there good library to use with clojure.spec? 🙂
2020:05:16 18:49:16 salam @neo2551 you can find spec libraries (along with other libraries) here: https://www.clojure-toolbox.com/
2020:05:16 20:57:07 Kira McLean Can you write a function spec for a multi-method? If so, any chance anyone can share an example of this somewhere?
2020:05:16 21:31:00 favila You can spec the method, but afaik not the implementations
2020:05:16 21:32:08 favila Caveat with fspec on mm: instrumentation changes the type to a fn, so future defmethods will fail with a type error
2020:05:16 21:41:00 favila You need to unstrument before evaluating more defmethods
2020:05:16 23:29:48 Kira McLean Interesting.. good to know. Yeah I’m curious about whether it’s possible to spec the implementations of a multimethod. I couldn’t find any examples online, but so far I’ve found with clojure that doesn’t always mean it’s not possible.
2020:05:16 23:41:52 alexmiller You can spec the dispatch method used in the mm if that’s useful
2020:05:16 23:42:05 alexmiller But mms themselves, no
2020:05:16 23:42:49 Kira McLean That’s something! But good to know, thanks!
2020:05:21 21:33:18 Kira McLean Ah this is really cool! Never thought of separating out the args and mm to spec separately. Perhaps a little unorthodox or is this normal? But definitely interesting. Thanks!
2020:05:18 12:41:55 Frankline Apiyo How can I fix this:
2020:05:18 12:41:58 Frankline Apiyo
Error Error: Require clojure.test.check and clojure.test.check.properties before calling check.
2020:05:18 12:42:16 Frankline Apiyo I get this when I call (stest/check `filter-list)
2020:05:18 12:45:00 Frankline Apiyo picard-facepalm never mind, I should have just read the error messsage
2020:05:18 13:09:53 mpenet @seancorfield @alexmiller @sgerguri fyi we released our internal fork of spec-coerce https://github.com/exoscale/coax
2020:05:18 13:10:28 sgerguri Nice, I'll have a look today - appreciate the heads up!
2020:05:18 18:07:55 Felipe Marques Hi, there, I'm using spec-tools with a multi-spec, and I get the following error when trying to coerce some data:
TypeError: Cannot read property 'cljs$core$IFn$_invoke$arity$2' of null
at type_priority (/js/cljs-runtime/spec_tools.parse.js:270:22)
at eval (/js/cljs-runtime/cljs.core.js:7726:102)
at eval (/js/cljs-runtime/cljs.core.js:7638:85)
at stableCompareFn (/js/cljs-runtime/goog.array.array.js:640:12)
at Array.sort (<anonymous>)
at Object.goog.array.sort (/js/cljs-runtime/goog.array.array.js:626:7)
at Object.goog.array.stableSort (/js/cljs-runtime/goog.array.array.js:642:14)
at Function.eval [as cljs$core$IFn$_invoke$arity$2] (/js/cljs-runtime/cljs.core.js:7686:12)
at Function.eval [as cljs$core$IFn$_invoke$arity$3] (/js/cljs-runtime/cljs.core.js:7724:23)
at Function.eval [as cljs$core$IFn$_invoke$arity$2] (/js/cljs-runtime/cljs.core.js:7720:26)
Does anyone have a similar problem?
2020:05:19 06:01:59 joefromct hi, quick question, given a spec ::my-map based on req and opt how could i get a list of the keys that this spec pertains to?
Should i be parsing s/form or is there an easier way?
2020:05:19 08:56:44 ikitommi @marques.goncalves.fel haven’t seen that. please write an issue to spec-tools and you could check also spec-coerce or coax - many libs doing mostly/exactly the same.
2020:05:19 09:26:36 mpenet 🍅 vs 🍅
2020:05:19 09:28:50 mpenet I haven't tested extensively the cljs part personally, tests pass but that's it for now
2020:05:19 12:01:18 jrychter This just bit me while writing :postconditions, where I wanted to assert that the return value was (among other things) a "status", but did not want to assert anything else: this is at an edge API where data has been transformed into external representations (timestamps to ISO8601 date strings, for example):
(s/def ::a string?)
(s/def ::b string?)
(s/def ::foo (s/keys :req [::a ::b]))
;; Let's define ::bar, which is not concerned with ::b at all:
(s/def ::bar (s/keys :req [::a]))
;; This is expected:
(s/valid? ::foo {::a "a" ::b "b"}) => true
(s/valid? ::foo {::a "a" ::b 1}) => false
;; But even though ::bar is not concerned with ::b, validation fails. I was hoping to only check if something
;; is a valid ::bar, without asserting anything about keys that are not in :req or :opt in ::bar.
(s/valid? ::bar {::a "a" ::b 1}) => false
I expected the last form to return true, to this day I did not realize that spec is eagerly checking all keys, not just those listed by s/keys.
2020:05:19 21:06:30 Eddie Good to know! Thanks for mentioning it.
It seems odd to me that we would associate a value that is not a valid ::b with the very specific qualified keyword of ::b in a map. Perhaps this indicates that in the :bar maps should be using a different key, like :b.
2020:05:19 13:06:29 alexmiller all keys are checked by s/keys (even (s/keys) is a valid and useful spec)
2020:05:19 13:07:53 alexmiller if you want to limit, you could select-keys on the data first, or validate the attributes specifically
2020:05:19 20:40:40 jrychter That was unexpected, I somehow got used to reading s/valid? as "tell me if this thing is a valid ::bar". Especially since it takes a parameter — (s/validate {:data :foo}) is intuitively different from (s/valid? ::foo {:data :foo}).
2020:05:19 20:42:10 jrychter Perhaps I'm using it wrong, but I would find the ability to check just "is this a valid ::bar ?" very useful.
2020:05:19 21:09:33 seancorfield @jrychter That makes sense for unqualified keys but part of the idea behind Spec for qualified keys is that they are globally unique (within your program) and therefore if :foo/bar is present in a hash map, it should conform to the Spec for :foo/bar if any. s/keys is more about required/optional anyway (and that's emphasized in Spec 2 via schema and select).
2020:05:19 21:28:00 alexmiller the big idea is that spec gives named attributes global semantics
2020:05:19 21:28:39 alexmiller if you have global semantics, it's ok (and good) to validate them when you see them
2020:05:19 21:29:43 alexmiller if that's bad then either the spec is bad (b/c it doesn't fully encompass what that attribute can mean), or your data is bad (b/c it doesn't conform to the global semantics) or your version of "valid" does not match spec's semantics :)
2020:05:20 05:12:33 David Pham Is it possible to share spec over wire? Or to serialize them?
2020:05:20 05:14:14 seancorfield @neo2551 You can get the code form of a spec with s/form -- and encode that send it over the wire -- but any predicates are going to depend on the code itself (anything beyond simple core predicates).
2020:05:20 05:14:42 seancorfield I'm not sure how easy it is in Spec 1 to reconstitute a spec from its form tho'. That's something that is easier with Spec 2.
2020:05:20 12:49:39 alexmiller Just eval the form you get
2020:05:20 05:15:10 David Pham Great! Thanks a lot!
2020:05:20 07:24:41 jrychter Hmm. I think I understand the "global semantics" goals, but what about the specific case I have now? I think I am falling into answer (3) in Alex's list above. I am preparing API responses and I do want to preserve the same key names, as they will be sent to the API callers somewhere down the way. However, those responses fall outside of my app semantics boundary, and some fields will contain data in a different form (like timestamps). That is OK, but I would still like to verify some aspect of those responses — whether they are a valid ::status/status, for example.
I think there is also a conflict with the "open maps" idea that Rich has been talking about. I would like to be able to validate data as being valid in terms of the supplied spec, and whatever else is in the map should not matter. I do understand the global semantics idea, just pointing out that these two might sometimes clash.
I would find the ability to check only "is this a valid ::bar ?" very useful.
Anyway, just wanted to describe the use case, as material for thinking.
2020:05:20 18:03:12 seancorfield @jrychter FWIW, Spec 2 has support for checking specs with closed maps as an option (it's a check-time option, not a spec definition time option).
2020:05:20 18:04:01 seancorfield Also, if it helps: what we do at work is have different (but related) sets of specs for domain, API, and persistence -- since API and persistence are constrained by external systems whereas domain is not.
2020:05:20 18:04:20 seancorfield Then we transform data at the boundaries between those layers as needed.
2020:05:20 22:35:57 Kyle Brodie I am trying to spec CSV rows that are parsed into Clojure maps but I'm hitting StackOverflowError in (s/keys :req-un [::kw1 ... ::kw156]) and right now the keywords are all (s/def ::kwN string?). I mostly want this for generating examples while developing so I don't have to use spec. It works if I give Java more stack space (I'm giving it -Xss512k to match Heroku's smallest dynos) but I think I'm using spec wrong
2020:05:20 22:46:27 seancorfield @kyle That's certainly a lot of keys in a hash map to require all be present... I assume they have more meaningful names than kw1, kw2, etc?
2020:05:20 22:50:53 Kyle Brodie Yeah its because the CSVs are denormalized so the columns are always there even if there is no data. I can get example data and make them into Clojure maps for my development. Just thought it would be cool to test out my functions with generated data
2020:05:20 22:51:12 Kyle Brodie The data will be reduced to less keys after a few steps so those I should be able to spec then
2020:05:20 22:51:57 seancorfield If you wanted random key/value rows with random keywords and random strings, just for testing stuff, you could use
Error printing return value (ExceptionInfo) at clojure.test.check.generators/fn (generators.cljc:435).
Couldn't satisfy such-that predicate after 100 tries.
class clojure.lang.ExceptionInfo
2020:05:20 23:53:12 seancorfield @kyle Ah, yeah, that's because it is trying to randomly generate elements from the full set for each key but each key in the generated map needs to be distinct 😞
2020:05:20 23:53:54 seancorfield You'd have to wrap the key part spec with a generator that produces the full set (in some random order, if you cared).
2020:05:20 23:55:30 seancorfield I suspect it would be easier to wrap the whole s/map-of spec in a custom generator since all it really needs to do is zip together that set of keys with a bunch of random strings...
2020:05:21 00:01:53 Kyle Brodie So I'd do (s/def (with-gen (my-spec-here) (fn [] (something-using-test-check-functions))))
2020:05:22 12:46:12 practicalli I am assuming that if I am using spec keys macro, I do not also need to check the data is a hash-map. So in the following code, the map? predicate would be redundant. Can someone confirm?
2020:05:22 12:54:29 alexmiller s/keys does a map? check
2020:05:22 20:08:38 David Pham Hello everyone. I really like Clojure.spec, it is really fun to use. I am currently wondering if there was a way to extract the default generator of a specs when using with-gen? The reason I am asking is I wish to provide more specific example than the default generator, but would still like to use the randomness of the default one. I know we can use gen/one-of, or gen/frequencies, but can we define the specific generator directly with with-gen? If not what would be a good/concise alternative to my problem, which is to define a spec with a custom generator which combine the default generator as well?
2020:05:22 20:21:03 kenny @neo2551 If I understand you correctly, we do this sort of thing a lot. Typically it's because of having a s/and. Here's an example of what we often have:
2020:05:22 21:33:16 David Pham @kenny It is more if you define the ::color spec as keyword? and provide the #{:blue :red} as examples? In this case the base gen is concise, but in a more general case, I would need to copy/double the definition of my spec, will I?
2020:05:22 21:35:02 David Pham I would still like to retain the ability to generate random keywords, while using my blue and red examples. One solution is gen/one-of, but I don’t see how I can get the random one without copying the base spec again.
2020:05:22 21:42:51 kenny I'm not really sure I understand your problem 🙂
2020:05:23 07:18:34 David Pham In your answer with with gen/one-of, you repeated keyword? which was the main predicate. In that case you repeated the main predicate twice, once for defining the validity, once in the definition of the generator. Can we avoid this repetition?
2020:05:23 07:19:29 David Pham I think it is a common pattern to give some examples to a generator for documentation, but it reduce the power for testing.
2020:05:23 21:13:52 niclasnilsson Recursive specs? I’ve tried all old examples I’ve found, but they don’t seem to work (any longer?). In spec or spec2, are recursive specs possible, or is eager lookup making it impossible? I mean things like this:
(def ex '(+ (+ 1 (* 2 3)) 200))
(defn operator? [x] (#{'+ '- '* '/} x))
(s/def ::operator operator?)
(s/def ::operand int?)
;; I've tried many different ways, but fail to get something like this working
(s/def ::expr
(s/or :operand ::operand
:expr (s/cat :operator ::operator
:left ::expr
:right ::expr))
2020:05:24 13:56:29 Charles Fourdrignier @niclasnilsson
You need to define a reference ::expr before the recursive one.
(s/def ::expr any?)
2020:05:25 06:32:03 niclasnilsson @charles.fourdrignier, well, yes and no. It compiles then, but it still doesn’t work.
The following compiles, but the expected output is not useful, and not what’s expected from a recursive spec. (see output from explain and conform)
2020:05:25 11:11:23 niclasnilsson @charles.fourdrignier, this is spec 2, but I’ve tried previously on the spec that’s in the box with 1.10 as well.
2020:05:25 11:31:13 Charles Fourdrignier Sorry, I don't get you were using Spec 2.
2020:05:25 11:40:27 niclasnilsson But it seems to work in spec 1, even without the any? trick. Very odd?
2020:05:25 11:44:24 Charles Fourdrignier Yes.
I use the any? trick a couple of months ago in Spec 1.
Pretty sure some error push me to use it...
Very strange.
2020:05:25 11:28:57 niclasnilsson Hmm… But this actually seems to work in “spec 1”, (clojure 1.10.1) but not in spec 2.
Unhandled java.lang.IllegalArgumentException
No implementation of method: :conform* of protocol:
#'clojure.alpha.spec.protocols/Spec found for class:
clojure.lang.Keyword
2020:05:26 19:44:52 alexmiller it's a known issue for aliased specs
2020:05:26 19:45:01 alexmiller in that they don't work right
2020:05:26 19:47:48 wagjo thanks, I'll use (s/def :foo/b (s/and :foo/a)) as a workaround for now
2020:05:28 20:32:35 David Pham Saying you a spec ::a , you can get back the definition with s/form. Is it possible to extract a generator from the s/form?
2020:05:28 20:46:58 alexmiller sure, just eval to to get a spec object, then s/gen
2020:05:28 20:47:30 alexmiller but if you have ::a, you're already there, so not sure what you're driving at
2020:05:28 20:48:07 alexmiller one big caveat is that that s/form does not currently capture when a generator is overridden with like s/with-gen
2020:05:29 04:29:24 David Pham How do You eval? I actually want the default generator because I override it, but for testing i would also like to test the generic one.
2020:05:29 13:14:49 alexmiller with the eval function?
2020:05:29 19:30:59 jacklombard Hello, I want to introduce clojure spec to validate API endpoints. The parsed response naturally does not have namespaced keywords. How do I go about validating the data? Should I simply use unqualified keys? Where should I keep my spec?
2020:05:29 19:31:36 jacklombard I know it is a very common question of where to put spec but can't find the right answer
2020:05:29 19:34:50 alexmiller I answered these in #beginners - in the future, it's best to put questions in just one channel
2020:05:29 19:55:05 David Pham Is it not a bad thing to use the eval function?
2020:05:29 20:16:13 alexmiller it's not inherently bad, it's just a tool
2020:05:29 20:17:20 alexmiller Clojure evals all of your expressions after all
2020:05:29 20:58:14 practicalli I have a simple function and an associated fdef specification, but am not getting an error when calling the function with incorrect arguments. The function should take a map that is a ::customer specification. Have I misunderstood something?
I would have expected a call to the customer-fullname to fail when the wrong kind of arguments are passed
(customer-fullname "customer")
2020:05:29 21:01:28 practicalli The ::customer spec works with spec/valid? and spec/assert when I use them with :pre and :post conditions in a function definition, but seem to be missing something when using spec/fdef.
2020:05:29 21:25:53 practicalli Ah, I have now learned about instrumenting the fdef specifications, and now it works, well fails when I expect it too...
(spec-test/instrument `customer-fullname)
2020:05:29 21:31:08 practicalli I am not clear on if it is valuable to use fdef without instrumenting them. Some discussions suggest is it but I have not really understood why as yet. I am still not that clear on the :fn aspect of fdef so will be on the look out for more examples. I still have a lot to understand about spec.
2020:05:29 21:34:55 alexmiller you can see the specs in (doc customer-fullname)
2020:05:29 21:35:12 seancorfield @jr0cket You can also spec-test/check them, separate from instrumentation.
2020:05:29 21:35:17 alexmiller ^^
2020:05:29 21:35:35 alexmiller those are the main benefits
2020:05:29 21:36:35 seancorfield In particular, instrumentation checks :args passed in are correct (and ignores :ret and :fn). Check is generative and passes in conforming random arguments (per the :args spec) and then checks :ret and :fn are satisfied.
2020:05:29 21:38:50 practicalli I can see the specs in the docs, yes that is very useful (had forgotten that in my frustration to get the code to fail). Will try out spec-test/check in the morning, sound promising. Thank you both.
2020:05:31 11:07:47 plexus I'm trying to figure out how to prevent my recursive spec-based generators from StackOverflowing. From what I can tell the checks that are built in based on *recursion-limit* are moot as soon as you use with-gen.
2020:06:07 13:02:06 Vincent Cantin That’s really a coincidence, because I am currently working days and days on overcoming this problem.
2020:06:07 13:02:30 Vincent Cantin You might want to give Minimallist a try.
2020:06:07 13:04:06 Vincent Cantin @U07FP7QJ0 let me know if you want to try the pre-release repo.
2020:06:08 10:27:59 plexus what's your sales pitch? why should I consider Minimalist over Malli (my preferred solution nowadays), and does it solve the problem I linked to?
2020:05:31 11:11:16 plexus The second one I could get away without the custom generator, it'll just do some extra work until it finds a valid value, but the first one is annoying. The spec boils down to (s/and (s/cat ...) vector?), but (s/cat ...) never generates vectors so when generating this tries 100 times and gives up
2020:05:31 11:11:59 plexus maybe there's a better way to write that? can I use regex specs but still convey that I really only want to match/generate vectors?
2020:05:31 14:48:24 alexmiller Not easily (this is something we’ve added in spec 2)
2020:05:31 15:41:33 clayton Hello!
I've recently done some reading on clojure spec and have added it to some side projects of mine, but I'm a little lost on how/when to use instrumentation (or orchestra). I've spec'd multiple functions in my project, but realize those are useless (aside from readability) without running some sort of instrumentation. Are there any resources explaining how to integrate instrumentation in a clojure project? I don't want to permanently leave a call to instrument inside my clojure files, and only want them to really be run during development phase. Aside from just running instrumentation in the REPL during the dev process, are there any best practices for instrumenting in my project? Thanks 🙂
2020:05:31 16:48:57 alexmiller most useful in dev at the repl
2020:05:31 16:49:07 alexmiller but could also be useful to turn on/off in a fixture around tests
2020:05:31 16:55:49 clayton Thanks! I've only ever used instrumentation in tests or the repl, but was thinking I was missing something. Back to the specs I go 🏃
2020:05:31 18:10:44 seancorfield @UB4EZH42F In the tests for seancorfield/next-jdbc I turn on instrumentation as part of test setup in several of the test namespaces.
2020:05:31 18:11:17 clayton will check out the repo for examples - thanks!
2020:06:01 01:21:51 Eccentric J A friend shared this with me https://gizmo385.github.io/clojure/programming/2015/08/11/adts-in-clojure.html which ended with a macro for (data Tree = Empty | Leaf value | Node left right). I'm trying to write the spec for something like that just for learning purposes. I got to (s/def ::tree (s/or empty?)) before getting a bit stuck 😅
2020:06:01 01:24:02 Eccentric J My first question is can you compose specs together like (def ::leaf ...)(def ::node ...)(def ::tree (s/or empty? ::leaf :;node))?
2020:06:01 01:30:38 seancorfield I think you'd need "tagged" data structures and multi-spec for that @jayzawrotny
2020:06:01 01:31:30 Eccentric J Ah thanks for the pointer!
2020:06:01 14:55:57 plexus regarding my question from yesterday re. infinite recursion in generators, I managed to work around it with size/`resize`, basically making sure that nested generators get increasingly smaller size parameters, and always emitting terminal tokens at small sizes https://github.com/lambdaisland/regal/blob/master/src/lambdaisland/regal/spec_alpha.cljc#L21-L33
2020:06:01 14:56:52 plexus maybe that's useful to someone 🙂
2020:06:03 08:20:13 MorongÖa I want to use phrase to produce human friendly errors messages. Has anyone used this before? What are your comments?
2020:06:04 02:20:17 nmkip I have a map that has this shape {"account" {"active" true "balance" 30}} , what's the best way to spec the "account" key?
I've tried these and I'm using the first one.
2020:06:04 02:24:00 seancorfield Inventive... but certainly "overkill" 🙂
2020:06:04 02:24:24 nmkip indeed
2020:06:04 13:05:48 dev.4openID Learner.
Maybe a dumb question. I have built up a detailed spec using s/def s/valid? s/conform. It works fine. Following best practices all my keywords use :: (e.g. ::timestamp). However, my data covered JSON to map, all keywords are single colon (e.g. :timestamps)
When I construct the data with two colons the valid? and conform has no problem whereas the mapped data with single colon fails.
I was understanding the :: indicates the keyword in this namesapace so having :: in my spec vs. the single : in may data should not matter.
Any clarifications on this? 🤔
<script src="https://gist.github.com/dev4openid/1becc5be61b764e330edc82d16f99eb6.js"></script>
2020:06:04 13:05:48 dev.4openID Learner.
Maybe a dumb question. I have built up a detailed spec using s/def s/valid? s/conform. It works fine. Following best practices all my keywords use :: (e.g. ::timestamp). However, my data covered JSON to map, all keywords are single colon (e.g. :timestamps)
When I construct the data with two colons the valid? and conform has no problem whereas the mapped data with single colon fails.
I was understanding the :: indicates the keyword in this namesapace so having :: in my spec vs. the single : in may data should not matter.
Any clarifications on this? 🤔
<script src="https://gist.github.com/dev4openid/1becc5be61b764e330edc82d16f99eb6.js"></script>
2020:06:04 13:12:40 aisamu The conversion from JSON will probably give you unqualified keywords, since no such concept exists there.
You have to either qualify your keywords on a separate post-processing step, or make a spec for the payload that takes unqualified keys (e.g. :req-un instead of :req)
2020:06:04 13:23:06 dev.4openID @U3UFFB420 Thanks for the explanation. But..... why is the :timestamp not found in the namespace or is it global? If global than how do I relate it to my incoming data - as it were. Because the spec is not of use right now. Obviously I do not see the connection
2020:06:04 13:24:05 dev.4openID So that is the purpose of req-un then?
Does that make the :keyword global?
2020:06:04 13:30:56 franquito Maybe you are mixing concepts. Keywords are just keywords, they can be qualified (`:test/timestamp`) or unqualified (`:timestamp`). :: It's a shortcut to write a qualified keyword with the same namespace as the current file.
2020:06:04 13:31:30 aisamu Keywords can be qualified or unqualified.
Unqualified can be thought of as "global", but it's better to stick with "lacking a namespace".
When you type in ::timestamp, Clojure uses the current namespace and creates a qualified keyword.
req-un means that the spec will match unqualified keys to the qualified keys, comparing just the name (i.e. discarding the namespace)
2020:06:04 13:32:47 dev.4openID So in a more complex map structure, where I have several collections do I have to req-un for all definitions or only at the top level and it is inferred?
2020:06:04 13:33:17 dev.4openID 'Cos I req-un all my defs and now it does not work
2020:06:04 13:42:05 aisamu Yes, you'd have to req-un all of them.
Try breaking it into smaller pieces to find where it's failing
2020:06:04 15:23:30 dev.4openID yes, without changing the above, how do you apply it to a map ex JSON) {:addr {:country "USA :street "A street"}}
2020:06:04 15:23:42 franquito As aisamu said before, if you are working with JSON you loose the namespace of the keywords, so you probably want to use :req-un in your specs.
2020:06:04 15:55:52 dev.4openID Yes, in theory that will work, however the data looks like this
:pieceID ["ABC1" "DEF2" "GHI3"]}]})
(def exampl2 {::abc [{::loca {::addr {::country "USA"}}
::dets {::some{::timestamp "2020-04-09T15:27:00"
::Url "https://mywebpage.com/answer"
2020:06:04 15:57:02 dev.4openID Look at line 1 of the gist
2020:06:04 15:57:29 dev.4openID so loca is a map of map of map
2020:06:04 15:58:54 dev.4openID The trouble I a,mm having is the compounded map that does not seems to play nicely
2020:06:04 16:06:37 dev.4openID
OK - it does conform to the definition Thx. However it is not as per line 1 - which is how the data is meant to be. So extracting from line 1: :loca {:addr {:country "USA"}}
2020:06:04 16:09:19 franquito Yes, you need to modify ::loca so it looks like this (s/def ::loca (s/keys :req-un [::addr]))
2020:06:04 16:09:32 dev.4openID @U3UFFB420 OK !!!! I think I have it ! I must take out the ::country in the s/def as the ::country is already defined (s/def ::loca (s/keys :req-un [::addr]))
2020:06:04 16:09:54 dev.4openID I think that is what you were trying to get me to see
2020:06:04 16:10:19 franquito Yes, exactly! 🙂
2020:06:04 16:10:38 dev.4openID Appreciated
2020:06:04 16:26:50 dev.4openID @U3UFFB420 Thanks for your patience and help. I got it now!! my complete script now works! Kudos for you 🙂
2020:06:04 13:12:38 franquito The difference is that :: expands to the current namespace. So if you're working in a ns called my.test then ::timestamp expands to :my.test/timestamp. The spec fails with :timestamp because it's also expecting the namespace.
2020:06:04 19:30:09 dev-hartmann Hi fellow clojurians, is anyone aware of a way to parse a swagger json to valid specs that can be used to generate test data?
2020:06:09 10:40:33 elimisteve That's a great idea and something I'm interested in, too
2020:06:09 11:04:01 dev.4openID This is what I did to learn spec. From this point on one can gen data. What you see is a map converted from a JSON structure. Now this is not a "take a swagger spec and convert" but is the basis of how to clojure spec based on a swagger that I did.
2020:06:09 14:19:52 dev-hartmann hey @UEGT2541J thanks for joining in. actually the challenge is not to create a clojure map from the json and then writing the spec. but parsing the json into specs . the difference being that I don't want to write them manually. but you are absolutely right, the chain is: json -> clojure-map -> transform clojure map entries to spec entries and compose them to upper level spec. then we can use the gen functions to create test data
2020:06:09 14:21:53 dev-hartmann if we have a working parser for that, we can just call the endpoint on webservices, slurp the swagger json and create specs for data models
2020:06:09 14:22:11 dev-hartmann which is a handy little tool 😄
2020:06:09 14:26:24 dev-hartmann I'm not an expert for specs, but searching a swagger yaml/ json for the definitions
2020:06:09 14:27:17 dev-hartmann and then using the type information there and maping those and the field names to specs sounds doable 😄
2020:06:09 14:27:51 dev-hartmann i like structured data 😄
2020:06:09 14:28:40 dev-hartmann I'll work on that a bit an will put a link to a gist here if I can create something useful
2020:06:09 17:44:15 flyboarder @U0J8XN37Y im really interested in this, we just rolled out swagger to our API’s both internally and externally, I would love to be able to have our apps handle specs for us
2020:06:09 17:46:50 dev-hartmann will keep you posted too.
2020:06:09 17:47:03 dev-hartmann I think it's easier than I thought.
2020:06:09 17:47:17 dev-hartmann .. I mean until I hit the brick wall 😄
2020:06:09 17:55:18 dev-hartmann at least the basics. translating the conditions on properties can take some time 😄
2020:06:09 20:29:34 dev.4openID OK, similar to what you are stsing then: look st the library serene - It autoigenerates the specs for REST and graphql APIs.
2020:06:09 20:30:43 dev.4openID I'll shut up now 🙂
2020:06:09 22:05:27 dev-hartmann no, please don't. never heard of that library and it looks like they are doing the same thing, but for a different format. 👍
2020:06:05 14:09:01 Felipe Marques @ikitommi
Hi, a few days ago you mentioned some other libs to perform spec coercion, but I forgot to take notes on its names to check it later.
Could you share the names again?
Also, I'm having some trouble with coercing some data using spec-tools. I'm not sure if I got the logic right.
Do you have any resource explaining how the coercion works in spec-tools?
Thanks very much!
2020:06:05 14:41:47 ikitommi Hi @marques.goncalves.fel! The slack history (& my message) is here: https://clojurians-log.clojureverse.org/clojure-spec/2020-05-19. I believe all spec-coercion libs are doing it in the same way, : given a spec, it's form is parsed and for all different specs (`or`, and, keys, integer?, any? etc.) a transforming function is selected and applied to the value. Corcion is recursive and best effort. With spec-tools:
2020:06:05 14:41:47 ikitommi Hi @marques.goncalves.fel! The slack history (& my message) is here: https://clojurians-log.clojureverse.org/clojure-spec/2020-05-19. I believe all spec-coercion libs are doing it in the same way, : given a spec, it's form is parsed and for all different specs (`or`, and, keys, integer?, any? etc.) a transforming function is selected and applied to the value. Corcion is recursive and best effort. With spec-tools:
2020:06:05 14:52:33 Felipe Marques Thanks for the links and explanation. Sorry for not finding the history. I looked only in slack! 😅
2020:06:05 20:53:57 dev.4openID Learner here. Dumb Question maybe, maybe a mindset
I have a map (ex JSON) in it exists the following:
{:status {:timestamp "2020-04-09T15:27:00"
:status "abc"}
Yes, I suppose I could change the data so that the 1st :status has a different name but that is not very elegant. If the structure remains the same, I assume you cannot redefine the :status again within the spec
Is there a way int spec (some technique) that deals with this type of anomaly? I have a data set exactly like that. Any thoughts?
2020:06:05 21:08:14 alexmiller s/keys with :req-un will match the short name of the provided spec but can use different qualified specs
(you can use triple-backticks around your code to format it and make it easier to read)
2020:06:05 21:59:13 dev.4openID Hi @alexmiller and @seancorfield - thanks for the help - I am wrong: my fat fingers and compiling versions of the trial and error - I screwed up! 😭😀 - along day!
2020:06:06 07:05:04 practicalli Question about spec keys. It seems I can use either qualified keys with :req or unqualified keys with :req-un . I cannot use qualified keys with :req-un. So if I have keys that could be qualified or unqualified, would I have req and req-un sections in the keys
Or I assume I should just choose to use one or the other.
I could just add the #:: auto-resolve macro to hash-maps with unqualified keys...
2020:06:06 07:14:57 practicalli Adding a specific namepace in front of the hash-map, #:practicalli.bank-account-spec seems to work too and feels like a better approach
2020:06:06 07:26:30 ashnur I haven't really had time to watch all the videos, is there one where the organization of the namespaces (especially in relation with models, specs, entities)? I am coming from js where circular dependencies are allowed in certain circumstances and I find it a bit too much work to create new files to create new namespaces (note, it's not the problem that I have to create files, I love lots of tiny files, I just don't like the now additional work that means I have to constantly rewrite all my files if I want to reorganize just a couple.)
2020:06:06 07:28:51 practicalli @ashnur I will be covering a bit of organisation in the broadcast today in about 30 minutes https://youtu.be/jUgm4zh-vF4 - I create a src, test and spec namespace and have a design-journal namespace to show my working 🙂
It still fairly basic, so you dont need to watch the other videos
2020:06:06 07:29:33 ashnur thanks, I will be watching
2020:06:06 11:36:05 dev.4openID There seems some confusion about :req and :req-un in regards to specs. In regards to a s/valid? or s/explain-str a lot of error info is generated.
So you s/def uses :: and can compose other s/defs with ::
However when using it with s/explain-str and it has a :req means (e.g. (s/explain-str ::myspec {:test1 "a" :test2 "b"))} it will generate errors versus (e.g. (s/explain-str ::myspec {::test1 "a" ::test2 "b"})) - assuming test1 and test2 are s/def somewhere.
At the moment the workaround is just use req-un and that gets past the problem. However, that seems a hack. Anyone with a good explanation anywhere as to the appropriate use of :req and :req-un. This seems more complicated to the eyeball than "it's good enough".
I see similar being asked above
2020:06:06 11:36:05 dev.4openID There seems some confusion about :req and :req-un in regards to specs. In regards to a s/valid? or s/explain-str a lot of error info is generated.
So you s/def uses :: and can compose other s/defs with ::
However when using it with s/explain-str and it has a :req means (e.g. (s/explain-str ::myspec {:test1 "a" :test2 "b"))} it will generate errors versus (e.g. (s/explain-str ::myspec {::test1 "a" ::test2 "b"})) - assuming test1 and test2 are s/def somewhere.
At the moment the workaround is just use req-un and that gets past the problem. However, that seems a hack. Anyone with a good explanation anywhere as to the appropriate use of :req and :req-un. This seems more complicated to the eyeball than "it's good enough".
I see similar being asked above
2020:06:06 14:00:00 alexmiller First, spec know nothing about :: - this is just a Clojure shorthand to fully resolve a keyword within the current namespace at read time. Spec will just see the full kw as if you had typed :user/myspec.
s/keys will always validate all fully qualified keys in the map with their associated registered specs. It will check that the map has all of the specified :req keys (match fully qualified) or :req-un keys (match only the unqualified name). And specially for :req-un it will verify the values validate against the spec for the qualified key.
2020:06:07 20:59:57 dev.4openID @U064X3EF3 Hi Alex
I accept is works and maybe I'm being a little thick here: The solution depends of the "temp" creation of a namespace as a workaround. In this case a :o/status in order to differentiate from the actual :req-un namespace of :status.
My perception is this in not very elegant or consistent with the spec model. As is encouraged by the clojure team, developers are to use :: more often. In the simple example provided - the code is not "portable" as such.
I recognize it may seem may seem nitpicking - perhaps I have missed the understanding or this is just accept "it works this way"
In the real workplace there are many occasions of "recursive dup fields in JSON data.
Could you provide a comment or two on this?
Much appreciate your efforts
2020:06:07 21:29:41 alexmiller I’m not trying to encourage more use of :: and it’s typically not what you need (but it’s very convenient for examples)
2020:06:07 21:30:17 alexmiller Personally I most often use full namespaces for data specs
2020:06:08 09:32:03 dev.4openID Hi, I was not stating you were the advocate - the spec literature does - just to be clear
Any thought about my point of "making up kw names" in conjunction with the point of recursive/nested JSON map structures that employ a keword such as my example :status - is there no recursive strategy for example? I don't know thus I ask.
2020:06:08 09:32:12 dev.4openID Thanks for your patience
2020:06:06 13:18:50 tianshu Is it expected if :a/b is not defined in this case?
2020:06:06 13:43:01 dev.4openID No. Imagine they are defined
It's not about a/b nut when :req or :req-un is appropriate
2020:06:06 13:52:24 alexmiller I would say yes, this is expected. :req means it’s required (and it’s there). If there are specs for any keys, they must be followed.
2020:06:07 03:29:11 tianshu For the case (s/def ::a ::b), ::b must be declared before use, but in this case it is not. Should it be unified? (I mean always define before use or always check at runtime)
2020:06:07 13:33:35 alexmiller Yes, in general resolution should be delayed for all specs and it’s not entirely true right now. Not going to make any updates on this in spec 1 though, will work on it in spec 2
2020:06:06 13:34:58 hadils I am having a very difficult time organizing my specs with my code. This is for code that interfaces to an external API. I have a single namespace for the code and an specs.namespace for the specs. The problem is that I have to rename fields in different args and returns for the API code so they won't collide with different uses. Help!!!
2020:06:06 13:34:58 hadils I am having a very difficult time organizing my specs with my code. This is for code that interfaces to an external API. I have a single namespace for the code and an specs.namespace for the specs. The problem is that I have to rename fields in different args and returns for the API code so they won't collide with different uses. Help!!!
2020:06:06 13:45:23 dev.4openID I have just done this and it is OK for code and spec to be in different files
I assume you have converted the JSON to a structure. Now define the spec "structure" accordingly
Apply s/conform ::spec "JSON structure" - note the spec has :: and your JSON map has : for all elements - in spec make sure all :req are :req-un and it will work
2020:06:06 13:50:50 dev.4openID in my case the file my namespace is ns: myproj.trail.clj and myproj.trial.spec.clj
2020:06:06 13:52:11 dev.4openID Lesson learnt model the API JSON map example in spec file fisrst and all becomes more clear. I started from leaves to trunk and not trunk down as seemingly many do
2020:06:06 13:53:00 hadils Thanks! @UEGT2541J
2020:06:06 13:54:43 hadils What about for multiple JSON outputs in the same spec file? How do I do that?
2020:06:06 13:57:20 hadils That's the real problem I'm having.
2020:06:06 13:57:47 dev.4openID If you break down the JSON maps into components and composed elements that are reusable. You then compose complete "trees" for all the responses for all the API calls. There are commonalities in the APIs returns and it goes faster in the compositions as you do more. i.e do 1 at a time. I went with the most complex one (in order to lean spec) and it just became easier to gen the rest.
2020:06:06 13:58:23 hadils How do you qualify your spec keys?
2020:06:06 14:00:04 dev.4openID It becopmes more and more obvious. Just build the leaves and branches slowly - use s/valid? and s/explain-str against sample data - extracted from the JSON map eventually build more complex branches and tada! you will compose the trunk. It seem tedious at first (as I was leaning) but it become faster
2020:06:06 14:03:08 hadils I am hung up on the qualified keys like ::foo instead of using :node/foo. The latter seems more appropriate for my needs. How should I approach this?
2020:06:06 14:35:28 hadils Thanks a lot @UEGT2541J!!!!
2020:06:06 16:22:51 hadils @UEGT2541J what do you do when you are trying to spec 2 different JSON outputs with same names -- some of the fields are identical and some are not, but have the same name?
2020:06:06 16:23:15 hadils Do I need to split them out into separate files?
2020:06:07 16:45:02 dev.4openID @UGNMGFJG3 Do you mean 2 fields in the JSON structure, where 1 is subordinate to another? {:name {:person "abc" :name}}
or do you mean recurring pasterns that are in a map?
2020:06:08 03:13:15 seancorfield s/or requires labels for the alternatives: (s/or :qualified (s/keys :req [...]) :unqualified (s/keys :req-un [...]))
2020:06:08 07:10:56 practicalli Oh yes, labels. At 2.30am I tend to forget things :white_frowning_face:
I find fdef a bit opaque, some more reading and experimenting to do.
Thanks for spotting the bug in last-name, not intentional.
Thanks for the review
2020:06:08 13:53:04 practicalli I got the spec/or expression to wrap the :req and :req-un versions working.
I have my fdef :args validating after instrumenting.
Still working on testing the :ret in the fdef...
2020:06:08 16:46:16 seancorfield Right, :args is for instrument -- checking that your code is calling the function correctly -- and :ret/`:fn` are for generative testing, to make sure your function behaves correctly.
2020:06:08 16:47:35 seancorfield With instrument, :ret and :fn are ignored. With check, :args is used to generate conforming arguments and then the function is called and the :ret and :fn specs are checked against the return value and against the arguments/return value respectively.
2020:06:08 16:48:42 seancorfield A lot of people don't feel the docs are clear enough about :args/`instrument` and :ret/`:fn`/`check`
2020:06:08 17:50:41 practicalli I’m comfortable with instrument for now. I don’t seem to have something right with check yet, but will look again tomorrow. Thanks.
2020:06:09 15:28:23 dev-hartmann hey folks, excuse my noobiness, but is it possible to return a s/def from a function?
2020:06:09 15:28:54 dev-hartmann I'm trying to build them dynamically from a parsed json which is working okish to the point where I try to return them
2020:06:09 15:35:06 alexmiller s/def is a function which I don't think you mean. do you mean a spec form or a spec object?
2020:06:09 17:21:38 dev-hartmann Ah, yes. Sry, that’s what I mean. I want to create them dynamically, save their names to an atom and then do an s/def
2020:06:09 18:09:21 alexmiller one way you could do this is to directly manipulate the registry (as s/def's impl does). in spec 2 we've pulled out a non macro function s/register for this purpose.
2020:06:09 21:23:16 dev.4openID
(defn len [n] (and (< (count n) 6)
string?))
(s/def ::piece-IDs (and vector?
(s/coll-of len into [])))
(s/valid? ::piece-IDs ["92257" "12" "01234"] );; => true
(s/conform ::piece-IDs ["92257" "12" "01234"]) ;; => ["92257" "12" "01234"]
(s/explain-str ::piece-IDs ["92257" "12" "01234"]) ;; => "Success!\n"
In the above code the s/def is valid, however it depends on the defn provided - seems untidy and not best practice
Can this not be changed to incorporate the count condition soemwhat as below?
(obvs. the coud does not work as the count does not apply to the strings)
(s/def ::piece-IDs (and vector?
(s/coll-of #(and (< count 0)
string?) into [])))
Any ideas to eliminate the defn in the above case?
2020:06:09 21:46:16 alexmiller you don't need a custom predicate for that at all
2020:06:09 21:46:53 alexmiller coll-of has a :max-count modifier
2020:06:09 21:48:03 alexmiller also note that in spec 2, there is now a catv too that would slim this down to just (s/catv string? :max-count 5)
2020:06:09 22:24:22 dev.4openID Hi, @vlaaad and @alexmiller
(s/def ::pieceIds (s/coll-of string? :kind vector? :max-count 5))
(s/valid? ::pieceIds ["JD014600007821392257" "12" "01234567890123456789" "777777777" "66666666666666666666666666666" "77"]) ;; => false it is counting 6 strings in vector
(s/conform ::pieceIds ["JD014600007821392257" "12" "11111111111111112222222222222222222y"]) ;; => ["JD014600007821392257" "12" "11111111111111112222222222222222222y"]
(s/explain-str ::pieceIds ["JD014600007821392257" "12" "11111111111111112222222222222222222y"]) ;; => "Success!\n" it is not applying the string length limit of 5 chars
produces the count how many string there are; whereas the code I presented determines whether every string length is less than 6 (this being the challenge I find myself in)
Subtle difference to what I suggested. Any ideas?
BTW Alex: I am using leiningen so I am not sure how to exactly refer spec 2; i.e. I do not use an edn file per se. Perhaps some guidance on that would be helpful - It refers to git etc. so I am unsure
2020:06:09 22:49:59 seancorfield @dev.4openid Do you want a :min-count as well as a :max-count?
2020:06:09 22:50:34 seancorfield Oh, you want the strings themselves to be checked for length, not the vector?
2020:06:09 22:50:54 dev.4openID Yes
2020:06:09 22:51:28 seancorfield So you want (s/def ::short-string (s/and string? #(>= 5 (count %)))) and then (s/coll-of ::short-string ...)
2020:06:09 22:52:09 seancorfield :min-count and :max-count apply to s/coll-of, not to things inside the collection.
2020:06:09 22:53:41 dev.4openID Well, it looks like I had the general idea right 😉 but not the implementation. I will try now
2020:06:09 23:05:42 dev.4openID @seancorfield
No it must be missing something
2020:06:09 23:07:23 seancorfield Read what I suggested for ::short-string and compare it to what you wrote.
2020:06:09 23:09:14 seancorfield (I just edited mine BTW)
2020:06:09 23:09:39 seancorfield (and #(..) string?) is truthy
2020:06:09 23:10:37 seancorfield (s/and #(..) string?) is a spec
2020:06:09 23:11:59 seancorfield Also, it's safer to add the type predicate (e.g., string?) first before applying count otherwise this will blow up (s/valid? ::piece-IDs [42]) because it will try to call (count 42) before testing it is a string.
2020:06:09 23:14:11 dev.4openID Yep, I missed it 😠 now it works and I have to discipline myself on the s/. Bit of improvement work!! @seancorfield thanks for the help.
Will use string (type checks first) Thx
2020:06:09 23:15:21 seancorfield (somewhat ironically, my incorrect version -- (and string? #(>= 5 (count %))) -- worked by accident because it evaluated to just #(>= 5 (count %)) because (and truthy x) => x for any x
2020:06:09 23:16:11 seancorfield I only realized my version was broken when I tried the 42 example and mine blew up, even tho' it had correctly rejected your test example with "012344444"
2020:06:09 23:29:57 dev.4openID I am learning this language ad enjoying it. this is after the last serious coding I did 30 years ago. It is a steep curve, it feels like doing "maths" all over again - strict discipline and be aware of the subtilties . There are so many variations and tweaks I sometimes get lost! Too easy too make mistakes. Worth learning as it is pretty powerful. Thx for your help
2020:06:09 23:40:23 seancorfield @dev.4openid The REPL is your friend here. If you try out every single function as you write it, you might have caught the problem with your len function sooner:
user=> (len 42)
Execution error (UnsupportedOperationException) at user/len (REPL:1).
count not supported on this type: Long
user=>
2020:06:10 14:49:09 adam I am trying to validate a server-side form with spec (so Clojure not ClojureScript) and I am a little lost about where to start to formulate human readable error messages.
Is https://github.com/alexanderkiel/phrase the de-facto library for that?
2020:06:10 14:49:09 adam I am trying to validate a server-side form with spec (so Clojure not ClojureScript) and I am a little lost about where to start to formulate human readable error messages.
Is https://github.com/alexanderkiel/phrase the de-facto library for that?
2020:06:11 14:10:23 avi I’ve always used expound for that
2020:06:11 09:17:11 jacklombard Do you all unit test specs?
2020:06:11 09:17:11 jacklombard Do you all unit test specs?
2020:06:11 13:10:38 Jivago Alves Never unit tested specs. But I'm using it as part of the test. I'd test it if I was going to release it as a library.
2020:06:11 13:32:36 jacklombard for complex specs that i use to validate data, would like to know if the spec is right
2020:06:11 14:11:04 avi That’s… interesting! Because I think I already think of specs as, more or less, akin to tests themselves.
2020:06:11 14:12:05 avi I do of course sometimes find that a spec I’ve written turns out to not quite capture what I had in mind, or a requirement, etc… but I’ve done the same with “traditional unit tests” (i.e. example tests) …
2020:06:11 16:08:18 Jivago Alves I use spec to help me to parse some data but I'm testing the function that does that. The fact I'm using a spec is just a implementation detail. At least, that was most of my use cases.
2020:06:11 16:46:22 avi Ah, I see, yeah. That makes sense.
2020:06:11 16:46:33 avi In that case, yes, I would probably write some example tests.
2020:06:11 18:28:14 seancorfield My approach to developing complex Specs is to have a (comment ,,,) form under the Specs with expressions in it that I use to test validation, conformance, and generation.
2020:06:11 18:28:56 seancorfield I also make sure to "test" each individual part of a Spec (so I can catch generation problems early -- debugging generation problems in a large Spec is no fun!).
2020:06:11 15:53:15 misha can I express "this key is optional, but if is in map - that key has to be there too, " as s/keys out of the box? with some combination of :opt :req and or like
(s/keys :req [(or :a/foo :a/bar)]) or something?
Or should I s/and it?:
2020:06:11 15:55:43 alexmiller s/and'ing a predicate is prob best
2020:06:11 15:56:36 misha is it s/and in spec2 for the same use case?
2020:06:11 15:57:31 misha thank you
2020:06:11 15:59:26 alexmiller yeah, I would do the same in spec 2
2020:06:11 15:59:52 alexmiller or rather I would probably make 2 different s/selects for the same s/schema
2020:06:11 16:00:01 alexmiller s/keys is probably going away in spec 2
2020:06:11 17:58:49 dev.4openID @alexmiller Over what time-frame are you anticipating spec 2 to come out?
2020:06:11 17:58:49 dev.4openID @alexmiller Over what time-frame are you anticipating spec 2 to come out?
2020:06:11 18:38:41 alexmiller I’m deep in something else right now, but hoping to pop the stack back to that soon. Not sure, prob mostly gated in rework of function specs which rich has been hammocking on
2020:06:11 17:59:29 dev.4openID I am assuming it will be spec.alpha2 right?
2020:06:11 17:59:29 dev.4openID I am assuming it will be spec.alpha2 right?
2020:06:11 18:39:05 alexmiller I’m hoping it will just be clojure.spec
2020:06:11 18:25:45 seancorfield @dev.4openid I'm not Alex but my understanding is that Spec 2 will eventually become just clojure.spec when it is ready to come out of alpha -- and there's no timeframe for it yet, based on what I've seen/heard, since a lot of things are still being designed/revised.
2020:06:11 18:28:01 seancorfield For a while, I kept a branch of our codebase current against that repo but it involved quite a few changes (and it had to keep changing as Spec 2 changed). I stopped tracking it back in ... September/October I think? Our approach going forward -- once Spec comes out of alpha -- will be to use the new Spec for new code we write and slowly, over time, migrate our Spec 1 code to "Spec 2" as needed. Spec 1 will stay available as-is "forever" so folks can stay on the old version if they don't want to migrate.
2020:06:11 18:35:09 dev.4openID Thx
2020:06:11 21:06:12 plins how can I create a generator from an arbitrary function?
here’s a silly example but It would be something like (s/def ::name (s/spec string? :gen (constantly "Margaret")))
not sure if possible but id like to combine it with a lib like https://github.com/paraseba/faker to generate test data
2020:06:11 21:29:13 alexmiller s/with-gen
2020:06:11 21:29:13 alexmiller s/with-gen
2020:06:12 03:14:36 practicalli How can I specify the number of tests run when using clojure.spec.test.alpha/check ? By default its running 1000 tests for a check against 1 spec (the playing cards example from https://clojure.org/guides/spec#_a_game_of_cards ) and takes around 80 seconds to complete.
The docs mention :num-tests within clojure.spec.test.check/opts but either I have the syntax wrong or missing something
Apart from Clojure 1.10.1, the project includes the dependency :extra-deps {org.clojure/test.check {:mvn/version "1.0.0"}}
Requiring clojure.spec.test.check generates an error when the namespace is evaluated
java.io.FileNotFoundException
Could not locate clojure/spec/test/check__init.class,
clojure/spec/test/check.clj or clojure/spec/test/check.cljc on classpath.
2020:06:12 04:12:21 seancorfield Your ::clojure.spec-test-check/opts is going to fail because :: will try to auto-resolve clojure.spec-test-check which is not an alias.
2020:06:12 04:12:21 seancorfield Your ::clojure.spec-test-check/opts is going to fail because :: will try to auto-resolve clojure.spec-test-check which is not an alias.
2020:06:12 05:08:14 David Pham Spec2 seems so cool. Increased programmability is such a good advantage. I have been using custom macros to generate spec, and it was a bit odd.
2020:06:12 07:18:56 ashnur Sorry if this question has an easy answer discoverable that I missed, didn't do a deep dive. I see that https://github.com/clojure/spec-alpha2/wiki/Schema-and-select#select is available in alpha2 but not in alpha. Obviously lot of people use spec already, but which version?
2020:06:12 07:22:02 mpenet "spec1" (spec.alpha)
2020:06:12 07:23:21 mpenet it's not really clear what will/wont change in spec2 and there are also a few bugs
2020:06:12 07:31:15 ashnur Thanks!
is there something similar to spec/select already existing, perhaps written by someone else, published under a different name?
2020:06:12 15:42:38 seancorfield @ashnur Spec 2 will eventually become the official clojure.spec. It's just not ready for use yet as it is still being actively designed and changed. Spec 1 will remain available for everyone already using it. I don't think anyone has tried to copy Spec 2 -- because it is not yet complete (and why would anyone try to recreate an official part of Clojure when Rich himself hasn't fully figured out parts of the design?).
2020:06:12 16:03:21 ashnur I am just curious if I want to use select or something that does similar stuff to select, what are my best options currently.
2020:06:12 16:11:21 seancorfield @ashnur If you're just building toy stuff or learning/experimenting, you could use Spec 2. It's just not ready for production use.
2020:06:12 16:13:05 seancorfield We were tracking it at work, with a branch of our (95k lines) codebase, and I really like the changes in Spec 2 -- above and beyond the schema/`select` stuff -- but there's been a lot of churn in Spec 2 and Alex has said that Rich will likely overhaul s/fdef completely before it is released (and it may drop s/keys completely as well), so we stopped tracking it months ago.
2020:06:12 16:13:28 seancorfield We're just going to wait for it to be "fully baked" at this point.
2020:06:12 16:14:07 seancorfield Once Alex signifies that it is stable and just needs testing to help iron out the bugs, we'll pick it up again.
2020:06:12 16:14:25 alexmiller I will signify that by making a release :)
2020:06:12 16:22:37 ashnur so, if I understand that correctly, there is nothing else that targets the same problem domain to be used in production in the interim?
2020:06:12 16:26:54 seancorfield clojure.spec.alpha targets that domain and can be used in production. Spec 2 is the "next generation" of that and will be the preferred solution when it becomes ready.
2020:06:12 16:27:46 seancorfield Spec 2 is definitely "better" than Spec 1 -- because it's designed to incorporate lessons learned from the first version. But Spec 1 definitely has value in production.
2020:06:12 16:36:56 ashnur I like what s/select does and would like to use something like it in production, even if it's not necessarily exactly the kind of s/select that is planned by Rich, since as I understand it, there are still months, maybe years until that version will be ready.
2020:06:12 16:47:40 alexmiller god I hope it's not years :)
2020:06:12 17:27:27 misha how can not be expressed in spec? to maximize built-in generators reuse and composability (really just ability to wrap any spec in "it" without looking at spec/form I am wrapping)
2020:06:12 17:33:55 alexmiller not is weird and I would generally avoid doing it :)
2020:06:12 17:35:06 misha at this time, I am trying to generate spec as close to schema as possible, as code you than paste into file, and then might chose to change
2020:06:12 17:37:43 misha Alex, is there an (out the box) way to conform unqualified map and get qualified conformed map back?
2020:06:12 18:21:10 seancorfield I'm not Alex @misha but I can't think of any easy way to do that. You'd probably have to derive the (qualified) keys from the s/form of the Spec, and then zipmap with a version of those keys that had been mapped to unqualified keys, and then use clojure.set/rename-keys on your validated data.
2020:06:12 18:22:00 seancorfield (but that won't work with nested data structures/specs or anything more complex than just s/keys)
2020:06:12 18:25:16 alexmiller You could unform
2020:06:12 18:25:51 alexmiller I guess you still wouldn’t get unqual
2020:06:12 18:26:08 alexmiller So I’ll go with no :)
2020:06:12 18:57:11 misha I thought about just including both :req and :req-un sets of keys, but it does not solve "I have a map from example page, show me the specs it uses", and screws up the generators, which you probably want to generate either entirely qualified or entirely unqualified deep tree.
2020:06:12 19:03:13 seancorfield If we were starting again from scratch with Spec available, and using next.jdbc instead of clojure.java.jdbc, I think we would only have unqualified keys at the boundary of our system: either as API input or user input (forms, URLs), and at outgoing boundaries for JSON-based systems. So our use of :req-un/`:opt-un` would be a lot smaller, and we'd explicitly transform validated input into a domain model that always used qualified keys. Interacting with JDBC via next.jdbc means you can use qualified keys going out to the DB and you would get qualified keys coming in from the DB as well, automatically.
2020:06:12 19:04:19 suskeyhose I'm playing around with custom generators, since there's a type which it seems like spec is having a hard time generating for some tests.
The problem is that whenever I try to sample the machine-gen, it works fine, but if I try to sample the result of (s/gen ::machine) it always says that a such-that isn't met after 100 tries.
What would be the cause of this?
2020:06:12 19:04:19 suskeyhose I'm playing around with custom generators, since there's a type which it seems like spec is having a hard time generating for some tests.
The problem is that whenever I try to sample the machine-gen, it works fine, but if I try to sample the result of (s/gen ::machine) it always says that a such-that isn't met after 100 tries.
What would be the cause of this?
2020:06:12 19:22:04 misha Sean, my initial motivation is exploration, specifically of https://vega.github.io/
So I want to generate spec from schema (which is 9999km long), then take an example json, and with magic-qualify-conform see, which spec is that, and then navigate through keywords and specs in my IDE, instead trying to find things in huge json schema: https://vega.github.io/schema/vega-lite/v4.json or https://vega.github.io/schema/vega/v5.json
This, and, the usual spec goods: exercise, etc.
2020:06:12 19:23:32 misha so it seems I'd have to come up with "qualiform" too.
2020:06:12 19:34:28 seancorfield Yeah, I can definitely see the utility of this and it would be nice as an option in s/conform.
2020:06:12 19:35:27 seancorfield I think it's interesting that Spec 2 takes a different approach, where you can specify unqualified keys inline in a hash map spec or else qualified keys in a schema, to be select'ed
2020:06:12 20:26:24 suskeyhose Search is being very unhelpful for this problem, seems like few people run into it. @seancorfield would you happen to know of anything I could do to debug this issue or to alter the generator so that it'll work?
2020:06:12 20:30:57 seancorfield @suskeyhose what is gen/ in your code above? I gather it's not clojure.spec.gen.alpha...?
2020:06:12 20:35:56 suskeyhose My understanding was that clojure.spec.gen.alpha was just a namespace that re-exposed some of the test.check vars.
2020:06:12 20:37:55 suskeyhose Which seems to be true looking at the source.
2020:06:12 20:41:41 seancorfield Hahaha... OK, it took me a while... What is ::symbols? What does it generate?
2020:06:12 20:41:54 seancorfield And then in machine-gen, what type is symbols?
2020:06:12 20:42:35 suskeyhose symbols is just a set of ::symbol, which are just maps with ::name and ::value
2020:06:12 20:43:11 seancorfield ::symbols is a set. symbols in machine-gen is a vector.
2020:06:12 20:43:33 suskeyhose Oh boy, of course that's it
2020:06:12 20:44:04 seancorfield That was a nice Friday afternoon debugging diversion -- thank you! 🙂
2020:06:12 20:44:48 suskeyhose Thanks for helping me out! I feel so dumb when I just get my types misaligned like that 🙃
2020:06:12 20:46:04 seancorfield No worries. I couldn't see it either. And I was simplifying the ::machine spec trying to figure out what the problem was... I was quite bewildered by it!
2020:06:14 22:35:20 misha is there a sensible way to specify min/max items count for s/?s/* other than function predicate, like?:
2020:06:14 23:00:40 misha in your example conformed int it just int, so unforming is identity. but if you have some branchy spec (s/or, s/alt, s/cat) - it will change the shape of the data, so in custom predicates for s/and and s/& you either have to unform, or validate conformed value (but this requires you knowing exactly what kind of spec is as the first arg to s/ands/&)
2020:06:14 23:02:19 misha for the homogeneous collections I can use s/coll and it has :max-count and :min-count options. I was hoping s/cat has something similar
2020:06:14 23:03:46 sgepigon I see. Thanks for providing an example.
2020:06:14 23:04:02 sgepigon I believe in spec2 this is “non-flowing s/and”
2020:06:14 23:04:28 misha yeah, should have mentioned: need this for spec1 opieop
2020:06:14 23:04:55 sgepigon You can modify the latter one to go into the map and count from there if you really want to avoid s/unform
2020:06:14 23:05:19 sgepigon because you end up putting all the conformed values under the :1 key.
2020:06:14 23:05:54 misha what about
(s/cat :1 (s/+ (s/or :a int?))) :2 string?)
kappa
2020:06:14 23:07:27 misha the thing is: this is for translating json-schema to clojure spec, so all specs are dynamic, and I don't do this manually. So I need to find the most sane lowest common denominator
2020:06:14 23:07:27 misha the thing is: this is for translating json-schema to clojure spec, so all specs are dynamic, and I don't do this manually. So I need to find the most sane lowest common denominator
2020:06:15 06:55:08 ikitommi interesting. any code to share?
2020:06:15 08:29:10 ashnur second that. I will have to do something similar soonish, and I am not even sure where I will begin : )
2020:06:15 09:34:15 misha within this week, I hope
2020:06:17 08:23:32 tianshu @vlaaad Thanks for the advice!
But I want care about both key and value.
You remind me, since there's a map-of , so coll-of is only designed to work with sequences?
2020:06:17 08:29:13 vlaaad @doglooksgood figured it out after reading coll-of docstring:
2020:06:17 12:59:59 alexmiller "soon" is still many months :)
2020:06:20 08:16:02 Vincent Cantin what are the most difficult parts?
design or implementation?
2020:06:17 13:00:12 alexmiller I didn't say which Conj
2020:06:17 13:08:39 borkdude lol 🙂
2020:06:17 13:48:17 ashnur A school administrator in a speech said once (and I heard it with my own ears), "in 2000, by 9 o'clock, we will finish the new school building!"
2020:06:18 03:36:47 jumar @scott.silver asked this in #cider: https://clojurians.slack.com/archives/C0617A8PQ/p1592435925498900
I'm also curious what's your approach with stest/instrument and evaluating a clojure buffer since it must be a problem with other editors too.
In short, after you evaluate a ns buffer all the instrumentation is gone and you need to call instrument again manually which is cumbersome and easy to forget.
Developers then commit spec errors which are only found much later, etc.
2020:06:18 03:58:05 seancorfield You could make it part of you fixtures if you wanted to ensure it was added for every test you ran via the REPL from your editor but I think this is enough to ensure a full test run uses specs.
2020:06:18 04:29:41 jumar Interesting, but I'm more worried about real "src" namespaces, rather than tests... do you have instrument calls there as well?
2020:06:18 04:37:30 seancorfield No. Why would you?
2020:06:18 04:37:48 seancorfield You want instrumentation in place for testing, not production.
2020:06:18 17:57:22 Scott Silver I agree that you don’t want it in production, but it’s very useful in dev, not just in test, yeah?
2020:06:18 18:03:43 seancorfield I run tests during dev so... 🙂
2020:06:18 18:05:12 seancorfield (also, I only spec API boundaries in general, or functions that have some unusual aspects -- we have over 600 specs in our code but only 30-some function specs)
2020:06:18 18:06:08 Scott Silver do you run tests manually while developing? or do you use something like lein-test-refresh to run the test suite when you make changes?
2020:06:18 18:09:27 seancorfield I have hot keys bound to run: a) all tests in the current namespace b) all tests in a test namespace that corresponds to the current namespace c) the test under the cursor
2020:06:18 18:10:12 seancorfield So it is manual but very easy. I don't use lein at all and I don't like watcher-based tasks or any sort of reload/refresh workflow.
2020:06:18 05:38:16 jumar We enable instrumentation for all fdef-s in dev mode (when running in REPL) to catch issues early - it's not enabled in production
2020:06:18 13:11:55 practicalli I am assuming the following ways to define a spec are equivalent ? The seem to provide the same results with confirm/valid? etc.
Is it overkill to define a spec for a set as it already acts as predicate function? Or does it make little or no difference?
2020:06:18 13:29:07 alexmiller either is fine. s/form probably works better with sets
2020:06:19 13:50:01 David Pham I have a tricky bug: it says that I have an undefined variable in cljs.alpha.spec at line 479 at the gen* method. However, I never referece cljs.spec.alpha (I use clojure.spec.alpha).
2020:06:19 13:50:12 David Pham Anyone encountered something similar?
2020:06:19 13:52:06 David Pham This behavior only manifest itself when I am suing advanced compilation
2020:06:19 13:59:37 David Pham Okay, solve my problem by putting my specs namespace into the main module.
2020:06:20 01:29:21 practicalli Any suggestions on how to create a custom generator for this specification for an email address (or an alternative specification for an email address that can be used - I could just use string until I get chance to think about it properly...)
(def email-regex #"^[a-zA-Z0-9._%+-]
When I use clojure.spec.gen.alpha/generate on the returned generator from (clojure.spec.alpha/gen ::email-type) I get an exception (my other specs will all generate data, just not this. So I am assuming I need a custom generator (it will be my first)
The error I get from generate is
2020:06:25 13:23:21 djtango though if you're just doing this for fun, then w/e
2020:06:25 13:44:05 practicalli Everything I do is for fun, unless someone is paying me a 6 figure sum to do otherwise (although I did still have some fun working at the bank).
2020:06:28 14:02:17 djtango 🙂
2020:06:20 02:03:34 gfredericks check the test.chuck (no typo) library
2020:06:20 02:06:44 seancorfield Yeah, a big +1 for test.chuck -- we use the regex generator a lot!
2020:06:20 08:44:21 borkdude @jr0cket Additionally, lambdaisland/regal also has generators for regexes
2020:06:22 15:29:38 butterguns Is this the right place to ask about spec-tools ? If so: I'm trying out Swagger2 generation. I'd love to make use of https://swagger.io/specification/v2/#referenceObject to avoid duplicating the same definitions all over my json doc. Does spec-tools support this?
2020:06:22 18:07:10 ikitommi @mattmorten currently, no.. I think it has been in ring-swagger/plumatic schema for years, just not ported into spec-tools. PR welcome
2020:06:22 18:08:08 butterguns Thanks for the response @U055NJ5CC, I'll look into it
2020:06:24 23:12:43 misha
2020:06:25 08:36:15 ikitommi How complete are you aiming this to be? Could it generate something else than spec syntax? There is work ongoing in making JSON Schema -> #malli and would like to see this for #schema too
2020:06:25 08:38:22 misha I am doing this just to get familiar with vega enough to implement few charts for my projects troll
2020:06:25 08:39:27 misha so for now I am going to finish spec1alpha, and then (but don't know when) - spec2
2020:06:25 08:44:56 misha I need to finish few combinator use cases, such as oneOf , and see what can be done for string spec (all the regexes, etc.), then I'll package it up as a lib. The main purpose would be to generate spec forms, and it will be up to user what to do with them: paste into file, etc. Now, I am not aiming for it to be fast, just maintainable.
2020:06:25 08:49:52 misha I am not very familiar with plumatic-schema, and only have seen few slides of mali, but I think it would be even easier to translate to those, given: that would be second/third implementation, and those support nested inline anonymous declarations.
2020:06:25 16:16:05 denik anyone have an idea why this returns invalid?
2020:06:27 08:40:30 Eccentric J Does anyone have or know of any examples of using spec with reagent views?
2020:07:06 15:08:28 Joe Lane Hey @jayzawrotny, I'm a little late to the party, but I found this blog post to be very apropos https://juxt.pro/blog/cljs-apps
2020:07:06 15:10:04 Eccentric J Thanks @U0CJ19XAM
2020:06:27 09:18:42 borkdude @jayzawrotny I do have one example from a commercial app I've worked on for the last few years:
(s/fdef dre.components.widget/widget
:args
(s/and
(s/cat
:opts
(s/keys
:req-un [::title ;; title in header
::content ;; Contents to show in widget. It must be a Hiccup
;; vector. Widget does not support varying arguments in
;; content, they are for initialization only.
]
:opt-un [::id ;; unique identifier for widget (page wide, not app
;; wide)
::icon ;; icon to show in header
::widget-class ;; CSS class
::collapsable? ;; If it can be collapsed
::init-collapsed? ;; initialize the widget in a collapsed
;; state?
::help ;; help contents which are shown in modal
::controls ;; Arbitrary component rendered in the
;; header. What it does is up to you.
::selections ;; coll of dropdown/tabs
::dropdown ;; convenience option, singular version of
;; selections with
;; {:type :dropdown, :id :dropdown} by default
::tabs ;; convenience option, singular version of selections
;; with {:type :tabs, :id :tabs}
::loading? ;; Whether or not to display a loading indicator
::locked? ;; Whether or not content modifications are possible
::menu ;; menu
::on-will-unmount]))
(fn [{:keys [opts]}]
(if-let [unexpected
(seq (apply dissoc opts expected-keys))]
(do
(warn "Unexpected options" unexpected)
false)
true))))
2020:06:27 09:20:49 borkdude @jayzawrotny This is about the only Reagent component I've spec-ed because it has so many options
2020:06:27 09:22:39 borkdude Not sure if it's useful, but this is the app: http://covid19.doctorevidence.com/ (this is a free preview of it)
2020:06:27 18:29:03 Eccentric J Thanks @borkdude this really helps!
2020:06:28 15:52:16 sveri Would it make sense to incorporate alpha 2 into a new project? All the documentation I can find right now is the differences to alpha 1.
Is it enough to read the docs for alpha 1 and the differences page?
2020:06:28 15:53:16 alexmiller No, you should not use it yet
2020:06:28 15:53:35 alexmiller Unless you enjoy bugs and breaking changes
2020:06:28 15:55:03 sveri Ok, good to know, thank you @alexmiller
2020:06:28 16:00:11 sveri Honestly, when looking at a library I always look at a high level doc first instead of api documentation. On the other hand, that api doc is very exhaustive 🙂
2020:06:28 16:00:15 sveri So thanks for the pointer
2020:07:01 13:17:19 alexmiller there's a lot of tedious history here but keywords with numbers as the name part are technically not valid in the reader
2020:07:01 13:18:08 alexmiller due to a long-standing bug in the regex for keywords, things like :1 are actually read however and we've decided not to make them invalid
2020:07:01 13:18:54 misha backward compatibility is tough
2020:07:01 13:21:09 misha this is I am aware of, yes.
trying to choose generated keyword format, suitable for all of: item spec (:foo/i1) cat/or branch spec for that item (:i1), conformed value walking (idx) etc.
2020:07:01 13:25:08 misha @alexmiller is it possible to get quoted form of lambda verbatim as edn? e.g. #(+ % 2) so it could be printed out exactly like this #(+ % 2) ?
2020:07:01 13:25:30 misha
'#(+ % 2)
=> (fn* [p1__7217#] (+ p1__7217# 2))
2020:07:01 13:25:33 alexmiller no?
2020:07:01 13:25:42 misha ok :(
2020:07:01 13:26:28 alexmiller if you're worried about the gensyms, you can transform to (fn [%] (+ % 2)) - spec does this for forms
2020:07:01 13:28:06 alexmiller (as an aside, there is a ticket and some work on better controlling gensym construction - this is often a tricky case when trying to symbolically compare macroexpanded code chunks)
2020:07:01 13:29:20 misha I am generating lambda forms for generated specs, and often fn one is way too long, so I might as well just defn it:
(defn <=10? [x] (<= x 10))
(defn >=5? [x] (>= x 5))
(s/def :user/root (s/and number? >=5? <=10?))
2020:07:01 13:30:55 misha which is, arguably, better, than
(s/def :user/root (s/and number? (fn [x] (>= x 5)) (fn [x] (<= x 10))))
2020:07:01 23:55:56 Jan K You could also do #(<= 5 % 10)
2020:07:02 08:36:46 misha not gonna:
• complicates lib code
• less granular spec errors
• less similar to source json-schema, so when(if) you gonna debug things – you will have to recalculate another transformation in your mind
2020:07:03 15:39:32 misha > range from start (inclusive) to end (exclusive).
- only for ints
- (ex)Inclusiveness is baked in
- requires both limits
- specifically range spec was not the point, but readability of inline functions was
2020:07:01 13:32:13 misha but then, there are things like ratios, decimals, etc opieop
2020:07:04 20:01:57 adam How do I refer a spec defined in another namespace? The usual :refer throws an exception
2020:07:04 20:03:07 alexmiller you don't need to refer it, just use the fq name
2020:07:04 20:03:30 alexmiller you do need to load the namespace defining the spec (w/ require)
2020:07:04 20:09:12 adam Got it, thanks. Ring’s wrap-reload middleware got me confused because it is picking it up even without referring the namespace IF I make some changes to my spec file... it suddenly becomes available everywhere.
2020:07:04 20:10:31 alexmiller specs are loaded into a global registry
2020:07:06 06:01:18 tianshu I want to use s/multi-spec to validate a map that has a few different structures. And I'm using s/conform to test if the map is satisfy the spec, but I can't find a way to figure out what branch it is matched since the conform on a multi-spec just simply return a map without more information.
how can I know which branch it matched, like in s/or .
2020:07:06 12:36:39 alexmiller I assume you're not just checking a keyword in the map, it's something more complicated
2020:07:06 12:38:04 alexmiller I don't think you're going to automatically get this, but you could add a conformer to the different cases to "tag" the map with a branch identifier
2020:07:06 14:34:46 tianshu @alexmiller I end up call that dispatch function again to get the branch identifier.
2020:07:08 07:21:15 rival Hi everyone 👋
to get my head around spec, I decided to just pick the FIX protocol and spec it. The result is my first Clojure lib https://github.com/rvalentini/fix.core. However the way I used spec felt a little bit "forced" here and there. At some point, I had the feeling that I was wrapping my validation logic into a thin spec wrapper definition for the sake of using spec, without much benefits. I couldn't find a clean/nice way how to build the validation logic as composition of individual specs.
To illustrate what I mean: in
https://github.com/rvalentini/fix.core/blob/master/src/fix/spec/primitives_spec.clj
it felt for me like an "intended" usage of spec, where the individual spec definitions compose quite nicely. But here
https://github.com/rvalentini/fix.core/blob/master/src/fix/spec/message_spec.clj
it felt more like I misused spec, since the spec definition is just like a facade.
Any advice on how I could improve this is much appreciated!
2020:07:08 15:05:02 misha @riccardovalentini1854 Spec's strengths are granular errors, parsing, generators. Since you basically use/provide neither of those, and just give "yes/no" answer to "valid?" – I think, such validation would be "lighter" with instaparse, or just giant regex.
What can be alternatively useful, though, is a speced (with generators) edn DSL + an encode/decode functions to/from that edn data structure to FIX string messages. (pretty much what I'm doing for graphviz dot files, right now)
2020:07:09 07:04:49 rival @U051HUZLD Thanks a lot for your feedback!
As a next step I planned to improve the error reporting for the client, so that in case the validation fails, the client receives some message indicating which part of the FIX message is invalid. I could make use of spec's granular error reporting there I think. I will also have a look at custom generators and think about how I could apply them to the internal edn representation of the parsed FIX messages. A nice encoder to produce FIX messages also sounds like a good idea! :+1:
2020:07:08 15:13:10 misha user then can explore "WTH is FIX message, anyway?" with s/exercise, and assemble messages as familiar maps/vectors with core functions, validate, get granular errors, etc, and then convert them into message strings.
2020:07:10 22:36:05 walterl Hi everyone. I'm still pretty new to spec, and can't figure out what's wrong with this simple spec with a generator fn:
Execution error (AssertionError) at clojure.test.check.generators/such-that (generators.cljc:346).
Assert failed: Second arg to such-that must be a generator
(generator? gen)
Any ideas?
2020:07:10 22:37:08 gfredericks That's not a generator
2020:07:10 22:37:47 gfredericks Generators are special objects from spec's generators namespace
2020:07:10 22:38:32 walterl Ah!
2020:07:10 22:38:41 walterl Thanks, @gfredericks
2020:07:12 17:11:49 coby Hey all, I want to check my intuition about something I'm trying to model. I need to design a small API for taking calendar availabilities (start/end pairs of date-times) and returning available appointment windows (also as start/end pairs, and also taking existing appointment times into account). Spec provides inst-in which is very nice, but I need to do a bunch of date comparisons, for which it's much nicer (IMHO) to use clojure.java-time. So would the recommended approach be to convert all datetimes to/from java-time at the edges of the API, and take/return insts?
2020:07:12 19:24:06 alexmiller inst is backed by a protocol so you don’t actually need to convert, just ensure the type you’re using extends Inst
2020:07:15 12:27:40 tikotus Once in a while I'm toying with Clojure, currently I'm trying to learn how to benefit from specs. I just re-watched Hickey's "Maybe Not" talk to support some thoughts running through my head. I ended up wondering about a thing in the proposed schema solution:
In the selection we give ::user [::first ::last] etc.
Now what's the point of ::user? Why not just {::first ::last}? Just for communication? I'm not sure of the value it adds to communication.
One thought I had was that it's there to tell us that we might be using other stuff from the ::user too, like ::addr, we just don't require it. But isn't this then just a briefer, less informative way of saying {:req [::first ::last] :opt [::addr]}? Now we're just one step away from the old spec :keys. Where did ::user go, what was the value of having a schema?
Anyways, are there any news on the next take on spec that I'm missing? Couldn't find anything...
2020:07:15 18:44:03 jaihindhreddy with schema you can describe trees and not just one-level. For example, we can say, address is optional, but when you do give me an address, I need the zip code and the street. Before schema and select, this kind of a thing would result in an ad-hoc spec to be written, and we end up with a parochiality problem.
2020:07:15 22:03:17 tikotus I still have two questions:
1. What's the use of ::user at the root in this case?
2. Why is schema needed for a nested select? I don't need a schema for ::address to say that if ::address exists in the map, I require it to contain ::zip and ::street.
2020:07:16 12:16:52 jaihindhreddy Let's say we're dealing with a scenario where we're dealing with a seller and a buyer, and we need the address of both, then ::seller and ::buyer would be useful. With just ::user, it's not very useful, but if we don't have the ::user in there, that means it's assumed that the ::address is that of the user implicitly, making it context-sensitive. With ::user, we're making it clear that it is user information.
2020:07:15 21:00:29 mikeb Cool. What's best way to begin trying out spec2, or is that discouraged until it's further along?
2020:07:15 21:05:50 seancorfield @mikeb I'd say it is discouraged at this point. We were tracking it closely at work for several months last year but it's still very fluid and very buggy and there's clearly still some "hammock time" needed. After talking to Alex in September/October about it, we decided to wait until he says it is "baked" before we try it again.
2020:07:15 21:06:22 alexmiller instructions are in the readme if you want to try it as a git dep
2020:07:15 21:06:48 seancorfield I think it's going to be a big improvement over Spec 1, based on our foray into it last year, but be prepared for it to change/break quite a bit as it continues to evolve 🙂
2020:07:16 16:12:02 kenny Are there some cases where the generator override passed to st/check will not actually be the generator used? I seem to be hitting a case like this. I do see this note in the docstring for s/gen "Note that parent generator (in the spec or overrides map) will supersede those of any subtrees." I'm not sure if this is relevant here. It's also not totally clear what "parent generator" means in this case.
2020:07:16 16:14:52 alexmiller I believe there are some cases where this doesn't work. the note is talking about recursive cases I think
2020:07:16 16:15:22 alexmiller iirc one common issue in this area is with aliased specs
2020:07:16 16:17:14 kenny Yep, this is definitely an aliased spec
2020:07:16 16:17:49 kenny Guessing this fix will be to use the alias.
2020:07:16 16:18:08 kenny Not exactly a fit but I think it'll work in my case.
2020:07:16 16:19:49 kenny fwiw, my usage here is really a workaround to selecting which (deeply nested) keys are required for a test 🙂
2020:07:17 10:58:53 Charlie Briggs hello, we’ve recently started using specs and have a few questions regarding how to load them correctly
say we have a namespace identity-provider and wish to have specs for identity-providers in a s_eparate file_, how would we go about doing that? We’re looking at putting the actual specs into a separate namespace as otherwise we’re having cycling import issues
if we put our specs in a separate file/namespace (say specs.core, and named them as :core.identity-provider/id, :core/identity-provider would we need to then require :refer :all in consumer namespaces to ensure they’re loaded?
Would it be more sensible to have a separate file for each entity specs, e.g. specs.identity-providers ? How would the ‘root’ object for the identity-provider look in that case (rather than using the name :core/identity-provider )
2020:07:17 15:41:54 seancorfield You don't need to refer specs. Just requiring the namespace they're in will cause them all to be registered.
2020:07:17 15:43:05 seancorfield It's common to put data-describing specs in a separate namespace from code. As long as you require that spec namespace somewhere (anywhere, early in your execution path), those specs will then become globally available.
2020:07:20 08:31:28 Charlie Briggs thanks for your response, so would it be common practise to load all specs in the server entrypoint, or a user.clj dev-namespaced file for use in the repl, rather than loading them in the code which is using them?
2020:07:17 15:43:13 seancorfield @charliebriggs ^
2020:07:17 15:52:21 Daniel Stephens I don't know if this is at all good practice, but we have a ns per 'entity' for our specs to keep things readable, specs in one namespace can refer to others outside of their namespace by using fully qualified keywords (as normal), and then we have one ns which just requires them all.
Those are all package together as one dependency, then anything that wants to use them just imports the dependency and requires that one namespace.
2020:07:19 15:38:28 misha By “readable” you mean ::?
2020:07:20 08:22:43 Daniel Stephens Assuming this is related to my response above I just meant readable as in 'more easily read (by humans)', I prefer reading a file with 10 specs in all relating to the same entity than trying to find what I want in a file of hundreds.
2020:07:21 06:49:45 tikotus Is utilising specs in linters technically possible, or is it too slow? Is there any tool working on this?
2020:07:22 11:25:46 denis_krivosheev Hello everyone. I want to write some tests for my library but I really don’t want to write example based tests. I thought spec can be an instrument for property based testing, but couldn’t find a way to transform spec (or generator) into a test. Can someone please help me with that?
2020:07:22 12:43:41 alexmiller With spec, the main way you do it is by writing function specs and running “check”
2020:07:22 12:45:17 alexmiller Spec generators are also test.check generators so you can use test check property testing too
2020:07:22 18:43:28 denis_krivosheev Oh this should work. Thank you very much
2020:07:22 18:47:14 alexmiller in fact, I find using (s/gen a-spec) to often be easier than using test.check to create generators directly
2020:07:22 13:56:56 borkdude @jahvenni https://github.com/arohner/spectrum is one attempt at this. I'm not sure if it's actively being worked on.
2020:07:23 06:12:05 tikotus Thanks @borkdude, that's what I was looking for. But doesn't seem very active.
Is it just me or wouldn't such a tool be simply amazing? It would make transitioning from statically typed languages feel a lot safer.
2020:07:23 14:46:32 seancorfield @jahvenni I think its very misleading to think of anything Spec-related as some sort of analogy to static typing...
2020:07:23 14:53:10 borkdude Yes. spec is a runtime validation/conformation/generation library. And considering how complex core.typed gets, I don't think getting leverage out of specs at compile time is trivial. Using a tool for something it's not designed for in general is ... hard.
2020:07:23 14:55:28 borkdude I do think you can get <some> leverage out of it. E.g. a tool could inspect specs at runtime, parse out the trivial stuff like, is the first argument an int?? Then something like https://github.com/borkdude/clj-kondo/blob/master/doc/types.md can be generated and you will get <some> static analysis benefits for clj-kondo, for example.
2020:07:23 14:57:08 borkdude Malli / @ikitommi has demoed a similar approach at ClojureD.
2020:07:23 16:27:13 tikotus Yeah, figuring out the trivial stuff is what I had in mind mostly. I can see many trivial cases where a linter utilising specs could easily provide value. Things like order of parameters, names of keyword arguments or tricky get-in's are mentally really heavy for someone who is used to have these things given by autofill. To me these sound like trivial things to check in most cases if spec is provided. It would be a much softer landing to a dynamic language.
2020:07:23 16:32:20 borkdude @jahvenni The approached I outlined is open for anyone to implement or experiment with 🙂
2020:07:24 23:57:27 seancorfield What sort of keys are in your report-data map?
2020:07:24 23:58:55 kenny Keywords
2020:07:24 23:59:00 kenny Did you mean something else?
2020:07:25 00:01:42 kenny I can't create a repro locally either. It's something about report-data itself.
2020:07:27 11:11:51 ashnur where should I look for docs/guide for how to test sideeffects on the frontend with specs?
2020:07:27 15:03:34 vemv do you want to test that side-effects happen?
or to avoid them, so that things are more cleanly testable?
2020:07:27 15:05:30 ashnur test them
2020:07:27 15:05:46 ashnur sadly, no one pays me for code that is pure 🙂
2020:07:27 15:08:08 vemv what do you use for frontend?
with re-frame you should have effect segregation for free
2020:07:27 15:08:29 ashnur the state is handled by react hooks locally in react components so it's not easy to test state transitions. There are specific situations that I want to avoid, so I was thinking I could run the app with generated data and test that specific situations don't occur. Like there is always a button to click or always a feedback message is shown
2020:07:27 15:09:22 ashnur the time I started this project re-frame didn't support hooks, which were required by my boss and colleagues
2020:07:27 15:09:30 ashnur so I use helix
2020:07:27 15:19:11 vemv not sure if I'd run Spec's generative testing (specifically, spec.test.alpha/check) with side-effectful functions. IME it's the kind of thing that is painful/slow to implement, and later to debug
It might be cleaner to generate data with Spec, and pass said data to a vanilla integration test (whether a purely frontend-bound one, or a full-stack one aided by Webdriver)
2020:07:27 15:21:12 vemv For completeness, with re-frame I think it's feasible to craft a useful generative test that asserts that interactions always yield a valid application state
e.g. click X + click Y, in any order, generate events a, b, c all of which are valid and form a valid new app state
All this side-effect free (thanks to the indirection bought by events)
2020:07:27 15:46:13 ashnur The first suggestion with the integration test makes sense, it's what direction I was going, but I wasn't sure if there is no other way.
2020:07:27 15:46:29 ashnur The second part with re-frame I don't understand, or maybe I understand just don't see how it is relevant?
2020:07:27 20:05:59 robertfw I do stateful generative testing but not on the frontend, so I can't help with specifics there. But I can confirm what @U45T93RA6 says - it is a bit painful and slow to implement. I've gone through a few iterations and am finally at a spot where I think we are taming the beast, but it's taken some work getting there and we have some work to go to get it running in a more reasonable amount of time.
2020:07:27 20:10:44 vemv > The second part with re-frame I don't understand, or maybe I understand just don't see how it is relevant?
it's relevant because it attempts to show how one could write a generative, integration-ish side-effect-free test using a specific framework.
Don't know helix myself so yeah it may have limited utility :)
2020:07:27 20:25:20 ashnur I still don't understand, are you saying I should use a different library? Or framework, although I do not use frameworks so I hesitate to use the word framework.
2020:07:27 21:01:57 vemv no, I'm stating a technique that may be translated to other (but not all) libraries/frameworks
2020:07:27 21:02:37 ashnur Wouldn't that mean that I would have to start over and write the whole thing in a different style?
2020:07:27 21:02:51 ashnur so are you saying this for consideration for future projects?
2020:07:27 21:08:46 vemv IDK, I don't know helix so I don't know how practical the described techique could be for you. In the end it's just food for thought :)
2020:07:30 10:24:59 arohner Are there any libraries for parsing swagger/openAPI into specs? I see https://github.com/metosin/spec-tools, but that appears to only do generation, not parsing
2020:08:06 02:10:14 olivergeorge I keep wanting ::x/blah to expand to :current.namespace$x/blah when there's no x alias. It'd open up "private namespaces" for spec use.
2020:08:06 02:10:46 olivergeorge My use case is CLJS function arg specs. I have many fns in a ns. They expect different things from a :db key passed in as context. (re-frame handlers)
2020:08:06 02:13:26 olivergeorge With the proposed change I could do things like this
2020:08:06 02:45:32 seancorfield I suspect Spec 2 (a.k.a. what will eventually become just clojure.spec🙂 ) will help you here because you can define a schema will all the possible keys with their specs, all in one place -- and then you can select the specific keys that are required for each individual handler.
2020:08:06 04:29:32 olivergeorge Good point. If/when that lands it'll be an improvement.
2020:08:06 04:29:33 olivergeorge I guess it'd look something like this...
2020:08:06 04:31:56 olivergeorge (Guessing at the unqualified key bits)
2020:08:06 17:50:29 johanatan hi, does clojure.spec.test.alpha/check run its test cases in sequence or in parallel ?
2020:08:06 18:00:15 alexmiller parallel via pmap
2020:08:06 18:00:47 johanatan any way to tell it to do sequential? because my code under test is already parallel and is kicking off a bunch of promises
2020:08:06 18:01:08 johanatan and the combination of the two is overwhelming the browser i'm afraid
2020:08:06 18:10:01 alexmiller not currently
2020:08:06 18:14:21 johanatan ok, no worries. i can just limit the num-tests to a small amount for now
2020:08:09 22:19:18 johanatan hi, i have a spec'd function that is taking as input a collection of data where the individual elements have attached metadata via a with-gen for their spec. however, because stest/check conforms these inputs to their constituent components (specified via s/cat) the metadata which was attached to the outer element is lost. what is the recommended way around this? perhaps a way to override "conform" or otherwise propagate the metadata? something else?
2020:08:09 22:21:05 johanatan i could manually attach the metadata rather than to the outer value to one of its constituents but this doesn't feel like the proper data modeling for the actual situation. i could also add an extra component to the s/alt but that isn't desirable as it would expose "test-only" data to the code under test (and also generally violate one's tastes with respect to the actual modeling).
2020:08:09 22:25:09 seancorfield @johanatan Are you saying that stest/check is passing conformed data into the function under test? That doesn't seem right since the conformed shape won't necessarily match the input shape...
2020:08:09 22:25:47 johanatan no, it's passing it into the :fn verifier
2020:08:09 22:26:06 johanatan sorry wasn't clear. stest/check is passing the generated data to the actual function
2020:08:09 22:26:22 johanatan but that data is slightly deformed via conform to the actual :fn checker
2020:08:09 22:26:29 seancorfield Oh, right. So you're trying to rely on input metadata in the :fn verifier?
2020:08:09 22:26:30 johanatan (per the labeling)
2020:08:09 22:26:33 johanatan yea
2020:08:09 22:27:13 seancorfield I wouldn't expect metadata to be preserved through that. It doesn't seem like something you should be relying on in :fn.
2020:08:09 22:27:27 johanatan it would be very useful for testing purposes
2020:08:09 22:27:44 johanatan the generators can then inform the tests what the expected behavior should be
2020:08:09 22:27:51 johanatan I can't imagine any better way to accomplish that in fact
2020:08:09 22:28:20 johanatan and as long as the generators don't use s/alts/cat and the like, this is entirely possible
2020:08:09 22:28:28 johanatan (i've done it previously)
2020:08:09 22:29:01 johanatan i have to step out now so will respond back later this evening. thx
2020:08:09 22:31:59 seancorfield Could you use s/conformer in the spec to thread the metadata into the conformed result I wonder?
2020:08:10 02:44:06 johanatan @seancorfield i'm not sure how to force s/check to use my conformer but yes that would seem to work if it were possible to do so.
2020:08:10 02:45:40 johanatan i wonder if there is a "conformer registry" i could write into ?
2020:08:10 02:48:04 seancorfield No, I mean directly in your Spec. So it always runs when a Spec is conformed.
2020:08:10 02:54:37 johanatan suppose i have a spec defined like so:
2020:08:10 02:57:49 seancorfield Put s/conformer in the second/last slot of s/and
2020:08:10 02:58:25 seancorfield That allows you to modify the (conformed) value as part of the conform process without affecting the validation.
2020:08:10 02:59:13 johanatan ah, ok
2020:08:10 02:59:54 seancorfield (s/and ... (s/conformer #(with-meta % {:what :ever}))) -- untested but that's what I had in mind.
2020:08:10 03:01:28 johanatan hmm, ok. let me try that
2020:08:10 03:02:23 johanatan the problem with that is that the meta depends on a generated value
2020:08:10 03:02:37 johanatan and i need to somehow "attach" that value to the generated structure
2020:08:10 03:02:50 johanatan while allowing it to survive the first "conform" in the s/and
2020:08:10 03:03:41 johanatan i.e., the following doesn't work because ::re-frame-event has already stripped away the meta and destructured into labeled data
2020:08:10 03:04:40 johanatan i could "cheat" and stuff it into the last "param" and allow my conform to strip it out of there and put it in the meta
2020:08:10 03:04:52 johanatan but that kind of violates a sense of purity 🙂
2020:08:10 03:05:50 johanatan and would also only work for some specific subset of cases; i.e., we're lucky in the sense that params is an s/* of s/any which could accommodate the data
2020:08:10 03:06:50 johanatan perhaps i just make the single conformer the "spec" and forget about using s/and ?
2020:08:10 03:07:03 johanatan then do the base conform within that conform
2020:08:10 03:07:15 seancorfield True, you could have a completely custom "predicate" that conforms as a spec...
2020:08:10 03:07:45 johanatan yea, i think i'll go that route. probably the path of least resistance at this point
2020:08:10 03:13:01 johanatan this works:
(defn metadata-preserving [spec]
(s/conformer
(fn [v]
(let [m (meta v)
res (s/conform spec v)]
(if (= res ::s/invalid)
res
(with-meta res m))))))
2020:08:13 05:16:32 ikitommi a common question for spec-tools has been how to create custom specs that coerce correctly and emit valid JSON Schema (for web-stuff). I though of pasting a sample here too for adding support for ZonedDataTime
2020:08:13 05:17:08 ikitommi
2020:08:13 05:23:26 ikitommi would be happy to throw away the custom wrappers for adding meta-data for specs in favour of core spec supporting that.
2020:08:13 13:12:01 respatialized can I re-use a matched value within the body of spec/cat for further pattern matching later on by referring to its key? I'm trying something like:
2020:08:13 13:14:14 alexmiller you can s/& at the top level and check constraints with any arbitrary predicate inside the container
2020:08:13 13:18:26 respatialized ok, thanks! I think the space of possible keys in the lookup map I'd be relying on is small enough to fully enumerate using individual specs, so I think I'll just do that instead (probably what I should have been doing from the start, but I wanted to see if I could save myself some typing 😂)
2020:08:13 17:35:22 johanatan is it possible to define a spec for a function in a namespace that one doesn't control?
2020:08:13 17:35:53 alexmiller yes
2020:08:13 17:36:17 johanatan how?
2020:08:13 17:36:27 alexmiller the same way?
2020:08:13 17:36:51 johanatan so like
(s/fdef the-ns/the-func ...) ?
2020:08:13 17:36:58 alexmiller yep
2020:08:13 17:37:08 johanatan ah cool. thx!
2020:08:13 17:37:25 alexmiller fdef always takes fq symbols and those are just keys in the registry
2020:08:19 08:08:49 rickmoynihan FYI there is also a #rdf channel on here, it has infrequent but good discussions on it when they occur
2020:08:19 08:11:21 apbleonard Thanks. Good to know :)
2020:08:13 22:30:35 alexmiller while I find it's usually a bad bet to say that Rich is not aware of something, as far as I know he is not aware of that. certainly a lot of the design work precedes that talk
2020:08:13 22:38:22 alexmiller from a quick skim of the video, both Rich and are I well-versed in RDF and OWL and that inspired many things in the design of both Clojure and Datomic (this is covered in some more detail in the History of Clojure paper https://clojure.org/about/history). I'm not sure that SHACL has anything particularly novel in it and it also makes a closed world assumption that's actually intentionally absent from spec to a large degree
2020:08:13 22:39:33 alexmiller I actually started using Clojure at the semweb startup Revelytix about 10 years ago and wrote a commercial federated SPARQL engine there
2020:08:13 22:40:00 alexmiller all in Clojure, sadly, no code survives from the company dying :(
2020:08:14 18:58:05 apbleonard Very cool you started Clojure in semweb :) I should have asked whether SHACL informed spec2/select as IIUC its all about defining different shapes of data (including closed world for sure) that can be drawn from the same (open world) OWL ontology for different specific use cases - a bit like how select will (I think) fix a shape suitable for e.g. a given function, whereas specs focus on the unchanging characteristics of attribute values in all use cases? (Hope I haven't mangled spec2 there!) I don't think it's novel - I mean it's a web standard ... Thought the specification of a data shape using SHACL as a graph itself was cool.
2020:08:14 19:01:42 apbleonard SHACL actually stands for "Shapes Constraint Language" which I hadn't realised :)
2020:08:18 11:57:47 rickmoynihan @apbleonard: I’ve noted this similarity before too, but I think the similarity comes more from the property oriented starting point inherent in RDF and spec. i.e. spec and SHACL are similar because they both come from the same starting point, that properties/keywords are first class and more important primitives than the entities on which they exist.
@alexmiller As I understand it SHACL is also open by default; though you can close shapes with sh:closed true. It’s true though that I think one of the big usecases for SHACL is in being able to close the world; in particular setting the min/max cardinality of a shape to 1, though I think the excitement around SHACL is more that it helps a long-standing area of frustration in RDF, which is that in RDF essentially every property/join can be many->many. Which has historically made it somewhat harder to validate and build user interfaces against it.
In this regard datomic/clojure is I think arguably less open-world, but more pragmatic, in that it closes the world on some properties by allowing essentially cardinality of 1. Sure, OWL does let you sort of do this; by falling back on the non-unique-names assumption, but relying on reasoning everywhere can be impractical.
2020:08:19 14:24:08 bhaim123 Hi, Would appreciate some help 🙂
What am I doing wrong?
I have a spec for an empty map:
`
2020:08:19 15:14:59 bhaim123 Thanks, but I don’t want to send the keyword empty map, the someone_big should be either an empty map. or a map of 2 keys
2020:08:19 15:25:32 sgepigon I’m not sure I follow. The spec does capture that it’s either an empty map or a map of two keys. You’re not “sending” any keywords, you’re labelling alternatives. Both (s/conform ::final :empty-map) and (s/conform ::final :someone) are invalid for this spec.
2020:08:19 15:26:35 sgepigon The original ::someone_big spec you posted was a s/or with 1 branch: it could only be a map with two keys and it was labelled ::empty_map.
If you reversed the order it might be clearer why it’s wrong:
2020:08:19 15:27:05 sgepigon Notice the path from the s/explain you posted:
at: [:someone_big :<NS>empty_map]
2020:08:19 18:33:02 bhaim123 Thank you very much @U56NS7GMQ
2020:08:19 15:37:53 alexmiller as an aside, (s/def ::empty_map (s/and map? empty?)) would be better than what you have as it should gen when the other would not
2020:08:20 19:01:52 johanatan Hi, I'm running into a crash in some substrate of my testing environment. When calling stest/check on a particular function, if I increase the number of tests beyond a point or increase the size of the collections involved, i get the following (unfortunately not very illuminating output):
> clojure -A:test
2020-08-20 11:45:17.836:INFO::main: Logging initialized @5575ms to org.eclipse.jetty.util.log.StdErrLog
[Figwheel] Validating figwheel-main.edn
[Figwheel] figwheel-main.edn is valid \(ツ)/
[Figwheel] Compiling build test to "target/public/cljs-out/test-main.js"
[Figwheel] Successfully compiled build test to "target/public/cljs-out/test-main.js" in 5.052 seconds.
Launching Javascript environment with script: "./scripts/launch_headless.sh"
Environment output being logged to: target/public/cljs-out/test/js-environment.log
#error {:message "ClojureScript test run failed", :data {:type :end-run-tests, :fail 1, :error 0, :pass 18, :test 4}}
Error: ClojureScript test run failed
at new cljs$core$ExceptionInfo ()
at Function.cljs$core$IFn$_invoke$arity$3 ()
at Function.cljs$core$IFn$_invoke$arity$2 ()
at cljs$core$ex_info ()
at Function.eval [as cljs$core$IFn$_invoke$arity$variadic] (eval at figwheel$repl$eval_javascript_STAR__STAR_ (), <anonymous>:64:25)
at redacted$test_runner$_main (eval at figwheel$repl$eval_javascript_STAR__STAR_ (), <anonymous>:18:33)
at eval (eval at figwheel$repl$eval_javascript_STAR__STAR_ (), <anonymous>:1:26)
at figwheel$repl$eval_javascript_STAR__STAR_ ()
at
at Object.G__12816__2 ()
Execution error (ExceptionInfo) at cljs.repl/evaluate-form (repl.cljc:578).
#error {:message "ClojureScript test run failed", :data {:type :end-run-tests, :fail 1, :error 0, :pass 18, :test 4}}
Full report at:
/var/folders/jz/920rhb8h8xj864001s7_grh80000gn/T/clojure-8321569605138574991.edn
Neither the full report nor the js environment log contains anything of interest. Just stack traces in figwheel main.
I can currently execute fine with num tests of 20 but when increasing to 100 or more, the crash occurs.
2020:08:20 19:02:25 johanatan The crash only seems to occur under headless Chrome; when using figwheel-extra-main/auto-testing in the actual browser, it doesn't occur.
2020:08:20 19:04:17 johanatan Unfortunately I don't get access to the "seed" when the problem occurs and my printlns do not get executed either so actually diagnosing is difficult.
2020:08:20 19:22:55 seancorfield @johanatan At a guess, since it seems both environment-specific and size-specific, I wonder if your (apply (partial concat x) rest) in the :fn spec is causing a stack overflow due to a buildup of lazy concat calls?
2020:08:20 19:23:39 johanatan oooh! that's possible. in fact that was the issue that spurred the creation of concatv to begin with
2020:08:20 19:23:54 johanatan would (mapcat identity colls) be a better impl ?
2020:08:20 19:24:49 seancorfield I think I would approach testing this via test.check and properties/generators, rather than trying to use fdef with :fn.
2020:08:20 19:25:23 seancorfield After all, your :fn is pretty much re-implementing the function being tested, but using lazy concat instead.
2020:08:20 19:25:46 johanatan this does use test.check and generators though.
yes, but that's pretty much how all :fn s end up
2020:08:20 19:26:03 seancorfield You're misunderstanding my point I think.
2020:08:20 19:26:39 seancorfield You're basically testing a vector-based concat implementation against concat itself -- so your test is going to run into the same problems as concat.
2020:08:20 19:27:15 johanatan ah, true. that's why i mentioned (mapcat identity ...) though. so, basically what we need are "two strict implementations of concat". one to test the other
2020:08:20 19:27:18 seancorfield What you should be testing are properties of the result, e.g., count of the result is sum of count of each argument.
2020:08:20 19:27:43 seancorfield Set of values of result is union of set of values in each argument.
2020:08:20 19:28:44 seancorfield (I'm currently working through Eric Normand's three courses on Property-Based testing so this is top of my mind right now)
2020:08:20 19:29:02 seancorfield In the beginner course, he has a specific example of testing some properties of concat.
2020:08:20 19:29:07 johanatan well i agree that testing the actual sets is better. that supercedes the individual properties
2020:08:20 19:30:06 johanatan i think there is a difference of philosophy here. if you test for actual equality then you don't need to test for the properties
2020:08:20 19:30:24 johanatan this is what i typically do with my :fn tests
2020:08:20 19:30:40 seancorfield The key is to not re-implement the function under test though.
2020:08:20 19:30:45 johanatan i see the other sort of property-based tests as strictly less powerful / more limited / less robust
2020:08:20 19:30:54 johanatan no, re implementation is fine
2020:08:20 19:31:10 seancorfield Clearly it isn't 🙂 That's what is causing this problem.
2020:08:20 19:31:16 johanatan as long as the implementation is in a completely different fashion
2020:08:20 19:31:28 johanatan well laziness is (likely) what's causing this problem
2020:08:20 19:31:47 seancorfield And also, you may end up with bugs in your function under test because you accidentally replicated them in your :fn re-implementation of it.
2020:08:20 19:31:47 johanatan so if we have two completely separate strict implementations that both produce the same result, then we're good
2020:08:20 19:32:08 johanatan sure, but i don't do that 🙂 or i typically keep iterating until i eliminate those
2020:08:20 19:32:15 seancorfield You would probably change your position on this if you took Eric's course.
2020:08:20 19:33:35 johanatan i find a lot of his material to be a bit more "entry level" than fits my needs. (just in general, having seen a few of his other videos. there are definitely philosophical differences here and his audience is, like you said, beginner-esque)
2020:08:20 19:34:52 johanatan i've done property-based testing for a while now. it's not like my thoughts on this just formed recently.
2020:08:20 19:35:29 johanatan i do agree that there are times where checking properties is fine: the canonical example being checking the correctness of an NP-complete problem's solution
2020:08:20 19:35:49 johanatan but a lot of problems in practice do not have "easy checks for correctness"
2020:08:20 19:36:04 johanatan but yea i'll give it a listen
2020:08:20 19:37:29 johanatan still yet, and we're assuming that laziness is the problem here (which it very well could be), there would seem to be a "bug" in the way the failure is presented if you will. i.e., there is literally no trace of helpful information in this case in the actual output 🙂
2020:08:20 19:48:29 johanatan and actually just realized mapcat won't help here since it is a thin wrapper around apply concat
2020:08:20 19:50:18 johanatan also, btw, nothing about :fn says that you must do a full reimplementation. you can still do "property" checks in there. of course with the downside being perhaps less readable output upon a subexpression failure. i'm willing to make that tradeoff though for the streamlined / inline specification
2020:08:20 19:54:02 johanatan i think what i'll do in this case since a second "strict" version of concat seems elusive is a simple pairwise traversal comparing elements along the way
2020:08:20 20:20:19 seancorfield > nothing about :fn says that you must do a full reimplementation. you can still do "property" checks in there
I would say property checks are better than a re-implementation.
2020:08:20 20:22:15 seancorfield > i find a lot of his material to be a bit more "entry level" than fits my needs
I've been doing Clojure in production for a decade and I'm still picking up new ideas from even his "entry level" courses -- but I agree that his style is aimed at beginners in many courses. He has three PBT courses: beginner, intermediate, and advanced -- so I figured since I have a subscription, I might as well watch all of them (I run them at 1.5x speed).
2020:08:20 20:22:38 johanatan cool, sounds good. i'll give it a look. thanks for your help on this!
2020:08:20 20:27:38 seancorfield It sounds like fdef is going to get substantially reworked in Spec 2, based on what Alex has been saying about that. But it's still all in "design mode" right now.
2020:08:20 21:01:13 johanatan hm, in what way? btw, which properties would you test here if you were going the "property route" ?
2020:08:20 21:01:24 johanatan everything i'm trying to do is hitting the same problem / crash
2020:08:20 21:01:35 johanatan like sum of counts, sum of hashes etc
2020:08:20 21:05:38 johanatan perhaps the s/* is causing this to blow up really large. i've read somewhere that it can happen and that it's hard to tweak as "recursion depth" is not all that precise of a control on it
2020:08:20 21:05:53 seancorfield > (`fdef`) hm, in what way?
Alex hasn't said... just that it's going to be substantially reworked.
2020:08:20 21:15:07 alexmiller Rich is working on it, there have been many ideas explored, not sure where it's going to end up
2020:08:21 03:37:29 johanatan btw i think i found the problem
(apply concatv rest)
can blow up the stack / is not in tail position.
2020:08:21 03:40:23 seancorfield Oh, interesting... Does that introduce laziness somehow?
2020:08:21 03:40:27 johanatan (at least that is "one" problem)
2020:08:21 03:40:52 johanatan well, it's a recursive call
2020:08:21 03:41:20 seancorfield OH! Yeah, that sounds so obvious once you say it out loud! 🙂
2020:08:21 03:42:06 johanatan let me explain. sorry, i noticed that even if I remove the :fn I still get the crash with the original impl of concatv. but with this loop/recur one (inspired by the Stuart Sierra blog on concat) that crash won't occur:
(defn concatv
"Strict version of concat."
[head & tail]
(loop [seqs (cons head tail) acc []]
(if (empty? seqs)
acc
(recur (rest seqs) (into acc (first seqs))))))
2020:08:21 03:43:28 seancorfield Seems weird to have head & tail when you only cons them together and never use them again.
2020:08:21 03:44:01 johanatan oh, could just be [& args] ?
2020:08:21 03:44:06 johanatan good point
2020:08:21 03:44:40 seancorfield You might want to consider whether (concatv) should work and what it should produce (I'd say yes and [] respectively).
2020:08:21 03:45:18 johanatan i'm just trying to match behavior of the original concat (which seems to agree w/ you) 🙂
cljs.user=> (concat)
()
2020:08:21 03:45:45 johanatan and yea that concatv does similarly:
cljs.user=> (concatv)
[]
2020:08:21 03:46:39 seancorfield I just tried it in Clojure and got an error
2020:08:21 03:48:03 seancorfield Oh, it works after changing to & args and (loop [seqs args ...
2020:08:21 03:48:05 johanatan hmm, mine was from lumo
2020:08:21 03:48:12 johanatan yea, sorry
2020:08:21 03:48:14 johanatan didn't paste in here
2020:08:21 04:06:33 johanatan so yea i think i was kinda confounded earlier by the fact that there was a potential stack overflow in the code under test. so even when i got the test code "right" the code under test could screw me
2020:08:21 04:07:29 johanatan but it was somewhat non-deterministic on both sides. bit of an entangled heisenbug
2020:08:21 04:07:32 johanatan 🙂
2020:08:21 04:08:35 johanatan two entangled heisenbugs rather
2020:08:21 04:08:44 seancorfield Complected, even 🙂
2020:08:21 04:08:51 johanatan haha exactly
2020:08:21 19:04:30 johanatan small update: so ... there turned out to be 3 problems. my hunch about s/* was also correct. some of the crashes were emanating from it as well.
here's a "more final" version (at least until I get another random, intermittent failure some time in the future):
2020:08:23 08:23:39 misha I did not read entire discussion, so it might be irrelevant, but:
there is a :distinct option in s/coll-of
(s/coll-of int? :distinct true)
2020:08:23 08:25:19 misha or is it supposed to be "distinct for all 1st and 2nd level items"?
2020:08:24 19:40:18 Lennart Buit So spec (1) has a list of predicates it can create generators for, so (s/gen int?) works. I kinda forgot, is it possible to extend this on the predicate level, e.g. can I register a predicate so that this works (s/gen my-predicate?)?
2020:08:24 19:46:57 seancorfield @lennart.buit You'd have to write your own generator (and then supply it when you define the spec based on your predicate).
2020:08:24 19:47:40 Lennart Buit Yah right, so I can only define a spec with s/def and attach a generator that way
2020:08:24 19:53:13 Lennart Buit Yeah right, reason I’m asking, we have quite a lot of specs that use a (non-spec aware) predicate, e.g. (s/def ::my-spec my-predicate?). So I was kinda hoping I wouldn’t have to do like:
But what I’m finding in the source, I appear to be out of luck
2020:08:24 19:55:20 alexmiller yeah, this is not currently an open set
2020:08:24 19:57:26 alexmiller I think the issue is that if you could extend it, then predicates may gen in one place but not in another if you didn't extend it the same way (somewhat reminiscent of reader macros).
2020:08:24 20:01:30 Lennart Buit Right — because you have this global idea of what generators exists for what predicates and that could clash, or not be loaded or …
Yeah I can see that as causing pains
2020:08:25 05:11:48 stuartrexking Is it possible to add a req-un to an existing s/def to create a new s/def. I have a request map with a number of keys, and a response map with all the keys in the request, plus an additional id key. Is there a nice way to do this?
2020:08:25 05:13:45 seancorfield @stuartrexking you can use s/merge to merge two s/keys specs together.
2020:08:25 10:54:50 jacklombard Is there a way to get the results of s/valid? and s/expalin-str in one call?
2020:08:25 10:55:49 sogaiu you don't mean like by using juxt or something right?
2020:08:25 10:56:09 jacklombard Nope, want spec itself to return it
2020:08:25 10:56:33 jacklombard so that I can destructure say the validity and the error in the same call
2020:08:25 11:01:02 jacklombard I am using spec for api validation where I check whether the body or the params are valid, if not I return 400 with error message as s/explain-str against the same spec. Of course the error message is not humanized, but we can deal with it. Don’t want spec to run its validation twice since payloads can be huge
2020:08:25 12:10:59 alexmiller You could run s/explain-data, check that, then run the function to produce the string from the data
2020:08:26 08:43:36 ashnur What's the proper way to generate random emails for testing?
2020:08:26 08:46:11 dharrigan If you're using spec you could use a generator
2020:08:26 08:46:30 dharrigan shows one such approach
2020:08:26 08:47:03 dharrigan another
2020:08:26 08:51:24 ashnur Yes, I am trying to learn spec to use it : ) Would be a stretch that I am using it now : )
2020:08:26 08:53:14 ashnur I see, so these are valid emails but they only generate a relatively small subset of possible email formats
2020:08:26 13:49:18 Joe Lane @ashnur Check out https://github.com/miner/strgen , ignoring that it still uses clojure.spec in it's examples, I think everything else should work.
2020:08:26 13:50:02 ashnur I was just about to try test.chuck
2020:08:27 17:53:33 ashnur I tried out regal, and I like it, but I am not entirely sure I understand what is happening.I tried out regal, and I like it, but I am not entirely sure I understand what is happening.
2020:08:26 15:14:03 ashnur that link is broken 😞
2020:08:26 15:15:27 ashnur I am having an extremely frustrating experience while trying out test chuck. Even though the namespace is included, the example code doesn't work, methods like string-from-regexp are nil. I am not even sure in this situation where should I go to ask for help, probably #beginners ?
2020:08:26 15:30:42 alexmiller here's probably fine - can you share a snippet of what you're doing?
2020:08:26 15:53:46 ashnur Probably because java regex is not the same as js regex. But then I am probably better off with one of the other options that were mentioned.
2020:08:26 15:56:52 alexmiller it should work with both
2020:08:26 15:57:07 alexmiller did you (require '[clojure.test.check.generators :as gen]) ?
2020:08:26 15:57:53 alexmiller test.chuck builds on top of test.check, so you need to include both of the generator namespaces
2020:08:26 15:58:43 ashnur hmm
2020:08:26 15:58:49 ashnur I did, but this gives me an idea
2020:08:26 16:00:02 alexmiller it seems from those errors that you have issues with both the gen and gen' namespaces
2020:08:26 16:12:38 hadils I need help with multi-spec:
(def a #{1 2 3})
(def b #{4 5 6})
(s/def ::type #{:foo :bar})
;;; This is pseudocode
(if (= ::type :foo)
; I want ::new-sym to be defined as a
else (= ::type :bar)
; I want ::new-sym to be defined as b
; ::type and ::new-sym appear in the same map
I am pretty sure this is done with multi-spec but I am stumped as to how to make it work...
2020:08:26 16:23:12 gfredericks test.chuck's regex generator is jvm-only
2020:08:26 16:36:28 alexmiller ah, sorry
2020:08:26 16:38:29 alexmiller @hadilsabbagh18 s/multi-spec is for the case where you can look at the data and return different specs based on the data
2020:08:26 16:38:35 alexmiller what is your actual data?
2020:08:26 16:40:14 alexmiller in general, spec is designed primarily for the case where attributes always have the same spec so you are probably going against that grain a bit.
2020:08:26 16:41:22 alexmiller if the attribute is in a map, you could create (s/and (s/keys :req-un [::new-sym]) arbitrary-condition)) to tighten a more general spec
2020:08:26 16:41:49 alexmiller and then use an s/multi-spec to choose which of those applies
2020:08:26 16:50:30 hadils Thank you @alexmiller! Here is the actual snippet:
I want to make :budget-item/ordered-categories dependent on what :budget-item/main-category is set to. They will be in a map together...
2020:08:26 17:12:24 alexmiller yeah, I don't think you should do this in a spec
2020:08:26 17:13:27 alexmiller or just make it a valid-category? predicate
2020:08:26 17:15:15 alexmiller just spec ordered-categories as a string? or keyword? or whatever, then s/and the whole map predicate with a custom predicate that validates that extracts ordered-categories and main-category, and validates that one is valid according to the other
2020:08:26 17:15:55 hadils Ok, I'll do that. Thanks a lot for your help @alexmiller! I am really grateful.
2020:08:26 17:16:41 alexmiller dependent stuff is hard to do well - it's easiest to just specify that constraint at the level that contains all of the data
2020:08:27 10:43:27 ashnur So, about this email generation thing. I know I am asking for a lot 🙂. But this is clojurescript, so my expectations are high because of the hype. However, the task to put together a generator for this http://emailregex.com/ in either of the suggested DSLs seem at least as daunting as just doing it from scratch, using built-ins.
2020:08:27 10:46:04 ashnur Basically, I am blocked because it's such a huge task, I don't even want to start, I can write all the necessary tests by not generating emails at all. Some of the hand written email address generators that I have seen suggested or used in libraries are positively naive compared to what is out in the wild. : )
2020:08:27 11:17:10 Lennart Buit The pragmatic approach to validating emails: Check they contain an @ , and send an email to confirm 😉.
2020:08:27 17:55:23 ashnur Now, what is the pragmatic approach to generate emails in clojurescript? 🙂
2020:08:27 17:57:01 ashnur this is regal, but I am not sure if it will work
2020:08:27 18:15:14 vlaaad What is the problem you are solving @ashnur?
2020:08:27 18:22:30 ashnur Which one? 🙂
2020:08:27 18:22:59 ashnur I am writing automated integration tests.
2020:08:27 18:59:25 vlaaad for system that validates emails? 🙂
2020:08:27 19:10:20 ashnur I am not sure why validation is coming up. I didn't speak about validation at all.
2020:08:27 19:12:35 vlaaad Sorry, too snarky... You need to decide what properties of strings representing emails matter for your tests involving emails. Nothing really matters, it's just stored? Use generator #" , who cares. Test needs to successfully send an email, but still has to generate different emails? Use my.email+<random-stuff-here>@gmail.com .
2020:08:27 19:18:38 vlaaad btw your domain generator implies 2-nd level domains like {:tag :mailto:mefoo.commefoo.com, :attrs nil, :content nil} , but email hosts might be IP addresses (e.g. {:tag :a, :attrs {:href "/cdn-cgi/l/email-protection", :class "__cf_email__", :data-cfemail "016c64413033322f3033322f3033322f31"}, :content ("[email protected]")}). If your program runs in a corporate network, sysadmins might configure it to have internal resources as 1st-level domain (e.g. just {:tag :a, :attrs {:href "/cdn-cgi/l/email-protection", :class "__cf_email__", :data-cfemail "d9b4bc99bab6aba9"}, :content ("[email protected]")}).
2020:08:27 23:56:13 panda is there a really easy way to make human name generators with spec?
2020:08:28 01:53:04 alexmiller A set is a valid spec that gens. So override the generator to gen from a known set of names
2020:08:28 00:02:14 panda also is there a way to get the generators for different items in a nested map to be consistent? i.e. for instance if i had a map like:
how can i get begin-date to be less than end-date in the generated output? sorry if this is documented somewhere 🙂
2020:08:28 00:02:38 panda ^ appreciate the help in advance 🙏😄
2020:08:28 00:29:11 seancorfield @lpanda2014 You can wrap the s/keys part with s/and and add a predicate to check that #(some-date-lib/after? (::begin-date %) (::end-date %))
2020:08:28 00:45:27 panda thanks! for some reason that didn’t work though 😞 had to change it to ints since i dont have a datetime library in my repl but the below code gave me a null ptr when i went to generate. is my syntax off?
2020:08:28 00:50:32 panda actually nvm figured it out - the second predicate can’t be :: 🙂 thanks for your help!
2020:08:28 01:06:56 seancorfield https://github.com/stackoverflow/date-clj is a simple library for manipulating dates, that has before/after comparisons. We use that at work.
2020:08:28 00:30:00 seancorfield (most of the date utility libraries have a simple function to check if one date is after another -- don't know what or if you're using)
2020:08:28 00:31:34 kenny or built in to Date: #(.after (::end-date %) (::begin-date %))
2020:08:28 00:35:04 kenny Needs a custom gen too. Something like (gen/fmap #(zipmap [:begin-date :end-date] (sort %)) (s/gen (s/tuple inst? inst?)))
2020:08:28 00:41:52 panda thanks! this worked like a charm
2020:08:28 01:08:04 seancorfield Ah, good point. I couldn't remember whether java.util.Date had comparison built-in (and was too lazy to check the Java API docs 🙂 )
2020:08:29 19:06:42 Lennart Buit How would you go about spec’ing a (http) patch like api using nil as a sentinel for retraction. Let’s say you have an entity that has an optional attribute, I’d hate to say that this attribute is s/nilable in its global definition just because there is a patch endpoint using nil to signal retraction
2020:08:29 19:06:42 Lennart Buit How would you go about spec’ing a (http) patch like api using nil as a sentinel for retraction. Let’s say you have an entity that has an optional attribute, I’d hate to say that this attribute is s/nilable in its global definition just because there is a patch endpoint using nil to signal retraction
2020:08:29 19:08:41 Lennart Buit Like — if I send you this attribute, it will either not be there, or it will have a non-nil value.
2020:08:29 19:19:58 Lennart Buit I don’t know, it feels asymmetrical: I’ll always promise you either no value at all, or some value satisfying a spec, but you can provide me no value, a value satisfying a spec or nil, but only in this specific case. I guess it doesn’t help me that I can have only one definition for a namespaced keyword (here).
2020:08:29 19:56:06 seancorfield Isn't this just a non-required key in a spec?
2020:08:29 19:56:24 seancorfield It's either present (and conforms) or it is not present.
2020:08:29 20:19:57 Lennart Buit Yes, outgoing I would agree: I’ll (= server) never send you a nil for a key I have no value for and instead omit it. The problem is in how you (= client) tell me that you don’t want this value anymore: If you omit it in your (partial) update request, do you mean to retract it, or to retain it?
So some rest endpoints allow you to patch an entity, but with nil as value, meaning ‘let’s get rid of this value for that attribute’. So now we have a bit of asymmetry: I’ll promise you to never send a nil value, but you can send me a nil to indicate retraction on this patch endpoint.
2020:08:29 20:20:59 Lennart Buit I don’t know how to express, in spec, that I as a server make a stronger guarantee than you as a client have to when patch ’ing entities. If that makes any sense 🙂.
2020:08:29 20:34:15 Lennart Buit (Or more specifically, I don’t know how to do that when my keys are namespace qualified. If I were to use s/keys with :req-un I could maintain two specs.)
2020:08:29 20:41:56 seancorfield That's just two different specs (perhaps with reuse on the common stuff). A spec for the result of a call (where the key is optional but spec'd to be non-nil). A spec for the input value (where the key is nilable).
2020:08:29 20:43:53 seancorfield If it's an API spec, it's going to be for unqualified keys, surely? Since it will be a wire spec, e.g., JSON.
2020:08:29 20:49:23 Lennart Buit Right, we may have extended this nilability (or the spec, for that matter) too far into our system. We have a JSON handler that accepts this nil and is spec checked, that is unqualified, but then we have an update method shared between this JSON endpoint and other places that is also checked, but has qualified keys. Thats where friction occurs: the keys are qualified, but their semantics differ between what you supply and what you get.
2020:08:29 20:55:08 seancorfield Well, if you have an internal (qualified) name for that attribute, either it should be optional and non-`nil`, or it should be nilable -- in all cases. And if that's not possible, then the two semantics should have different names.
2020:08:29 21:04:50 Lennart Buit Sorry for the kinda fuzzy description… Right so in the latter of your options you say to extend the notion of valid values for this attribute to be a superset of all values it can take in all contexts. That means that consumers I could have promised no nils (because they consume the ‘outgoing’ format), have to consider nils because thats what I could promises them in my spec, right? I find that kinda sad.
2020:08:29 21:07:00 seancorfield I'm saying use different specs as needed and transform the data to match as you cross boundaries.
2020:08:29 21:13:48 seancorfield Another option is to chose a specific, unique, representation for a retraction (and, again, map from the inbound retraction to that representation).
2020:08:29 21:15:14 seancorfield (we had exactly this situation and we tried to blur the lines with nilable and optionality and it was a mess so we mapped it to a different representation altogether)
2020:08:29 21:15:38 Lennart Buit Right that makes sense
2020:08:29 21:19:36 Lennart Buit Yeah, I think we are going wrong in a similar way: have a single namespace qualified keyword for an attribute in all its contexts, but using it in different ways. I’ll put this in the hammock, thank you for the input, as always 🙂!
2020:08:31 02:26:45 rapskalian The fact that you’re destructuring the key doesn’t change the fact that budget-item-type simply accepts a map. Assuming you have a spec for :transaction/amount, your args spec would be something like
(s/cat (s/keys :req-un [:transaction/amount]))
2020:08:31 02:27:08 seancorfield @U6GFE9HS7 That's the answer they got in the other channel.
2020:08:31 02:27:20 seancorfield Please don't encourage people to cross-post questions here.
2020:08:31 02:27:59 rapskalian Oops, didn’t see that. Noted @seancorfield
2020:08:31 02:23:26 seancorfield @hadilsabbagh18 It's a good idea to not cross-post questions here.
2020:08:31 02:23:39 seancorfield You've already gotten a response in another channel on this.
2020:08:31 02:26:51 hadils @seancorfield Sorry my bad
2020:08:31 13:09:33 Petrus Theron How to constrain the size of two dependent coll-of specs, A & B so that (<= (count (concat A B)) 40)? AFAICT, I need to use g/bind to constrain the :max-count of B's sample based on the length of A, but I'm having some trouble getting this to work in Spec2.
@alexmiller you mentioned you are considering basic logic resolution. It would be awesome if it was possible to do something like:
(let [a (s/coll-of digit? :max-count 30)
b (s/coll-of digit? :max-count (- 40 (s/count a))]
(g/fmap (fn [a b] (apply str a ":" b) [(s/gen a) (s/gen b)]))
2020:09:02 04:30:07 johanatan I always use gen/let for this sort of thing. You can use it just like a normal let except with generators on the rhs and generated values bound to the lhs. if you need a regular value you can use a gen/return to effectively raise them to that level such that both regular values and generated values can be bound in the same gen/let.
2020:09:02 04:31:45 johanatan in this particular case, you could generate an int with int-in 1 40 and then compute a second int-in 40 - the first. then use the two ints as exact size parameters to your collection generators.
2020:09:02 04:16:33 johanatan does anyone see why the following wouldn't work? I'm getting an error: sym not defined on the stest/check line.
2020:09:02 04:43:15 seancorfield @johanatan You might have to ask in #clojurescript -- that seems an unfortunate and gratuitous difference in the implementations.
2020:09:02 12:54:20 borkdude I've also ran into this a couple of times
2020:09:02 21:18:24 johanatan [massive time saver once you start to accumulate many of these]
2020:09:03 01:59:25 kennytilton Is clojure.spec a timid retreat from dynamism? In part bowing before the sacred cow of testing?
"tests are essential to quality" --https://clojure.org/about/spec
It seems like Clojure ventured bravely forth from the suffocating safety of Java's strong static typing, caught one gust of untyped wind, and scurried back to the shelter of Java's skirts.
Or was this just a pragmatic choice to make Clojure more palatable in tall buildings?
2020:09:03 02:00:22 kennytilton Not trolling, just trying to decide whether to buy into the burden of spec.
2020:09:03 02:08:50 seancorfield You certainly sound like you're trolling @hiskennyness but I know enough of your posts to know that's just kind of your "teasing" style 🙂
2020:09:03 02:09:46 seancorfield We use Spec very heavily at work, in production code, to do data validation. It is not any sort of type system so, no, it is not any sort of "scurrying back to Java's skirts" 🙂
2020:09:03 02:10:41 seancorfield Have a read of https://corfield.org/blog/2019/09/13/using-spec/ and see the many ways we're using Spec at work -- none of which are in any way as a substitute for types.
2020:09:03 02:13:06 seancorfield We do find Spec to be very helpful in testing (and to some extent in development). We use specs to generate test data, to help support property-based testing, to valid test results...
2020:09:03 02:13:54 kennytilton Thx! I'll check it out. But reading that intro I kept hearing "OK, we should have objects with defined properties; maps are too loosey-goosey". Lemme see what HisCorfieldNess is up to...
2020:09:03 02:17:06 seancorfield (I'm going off to make dinner but I'm certainly happy to discuss Spec in great depth when I get back -- or tomorrow 🙂 )
2020:09:03 02:38:26 kennytilton OK, nice, short and sweet. Actually readable! But we likely disagree on testing, so no progress can be made there. As for runtime parameter checking, spec could be a handy way to code up what folks do anyway, but then that could be trivial little DSL I knock off in an hour then evolve easily over time. spec seems to be a handful at times, with people asking for help expressing different things. Sounds like a classic case of a good idea run amok. As for code generation from spec, now you are scaring me. 🙀 We have macros for DSLs, and I write the macro that parses a tree of symbols whose structure can be as user-friendly as I like? Think "CL loop" 🙂. Why parse a spec definition, whose structure I cannot completely control? Even if I can make the latter work, it now feels as if I am in places coding my app in spec so it can be generated, when I already have a fine language to work with (Clojure!). Maybe this is why people get stuck using spec, because they ineluctably get drawn into more and more intense usage trying to program with it.
So how widely used is spec?
2020:09:03 03:53:52 seancorfield Back from dinner now @hiskennyness... I'd say Spec is pretty widely used. Lots of libraries provide optional Specs in a separate namespace so folks on Clojure 1.8 or earlier can avoid it and anyone on Clojure 1.9 or later can opt in if they wish.
2020:09:03 03:55:01 seancorfield As for generating code from Specs, that's about having one "source of truth" and it might as well be the Spec (esp. since mostly it's macros in Spec 1 and they're hard to deal with programmatically -- that will change in Spec 2).
2020:09:03 03:55:24 seancorfield And we generate code from Specs we write.
2020:09:03 03:58:45 seancorfield Spec has a lot of different uses. Some people do try to use it like a type system (bad idea, IMO, and that's not what it is designed for). Given your skepticism, I can only assume you've not watched any of the (many) talks about Spec over the years? Spec has been available for about four, four and a half years, since 1.9.0-alpha1 in May 2016.
2020:09:03 12:18:26 kennytilton I'll google up some vids. Yes, well aware of spec for a while. Just ran into it at work for the first time, so it is time to get to know it. Thx for the feedback!
2020:09:03 16:47:08 seancorfield Feel free to ping with any Qs if you want 🙂
2020:09:03 09:33:49 borkdude > Lots of libraries provide optional Specs in a separate namespace
I think this is good practice also for programs written for > 1.9
2020:09:07 18:06:05 ho0man Hi everyone,
Can I write a multi-spec for the component/config segment of this data based on the component/type which is not part of component/config's corresponding value? :
2020:09:07 18:07:06 ho0man the component/config part has to be extensible ... so different components of the system across different namespaces define their own internal config map
2020:09:07 18:31:57 borkdude Is it possible to inspect an evaluated regex spec?
2020:09:07 20:37:01 tekacs definitely not ideal, but that works?
2020:09:07 20:37:03 borkdude I kinda wish that I could have user/x resolved to clojure.core/int? as well, since this information is available in the spec itself
2020:09:07 20:38:25 tekacs right — though you wouldn’t usually use def to define a sub-spec
2020:09:07 20:38:57 tekacs you’d usually have done:
2020:09:07 20:41:13 tekacs I’ve since moved to malli but I’m excited for this! 🙂
2020:09:07 20:41:33 tekacs I saw that you posted in #malli too 🙂
2020:09:07 20:42:17 tekacs I also forked aave for malli to add CLJS support and some additional features I haven’t seen elsewhere (https://github.com/tekacs/aave), so I should probably pay attention to the clj-kondo type spec 🙂
2020:09:07 20:43:19 borkdude @tekacs That's cool, you can use the same kind of trick indeed
2020:09:07 20:52:36 borkdude @tekacs
2020:09:07 20:56:05 kenny This is awesome. Support for s/keys would be a huge productivity win. We have many functions that take large maps as params and it's so easy to forget or misspell one.
2020:09:07 20:56:19 kenny Thank you for working on this @borkdude 😀
2020:09:07 21:24:34 borkdude :)
2020:09:07 21:25:09 kenny OMG
2020:09:07 21:26:54 kenny So freaking cool. An optional toggle for something like https://github.com/bhauman/spell-spec would epic (I could see some people not liking it).
2020:09:07 21:27:52 kenny I wonder how many places in our code base we have a :req key and aren't actually passing it but it happens to magically work in prod 😅
2020:09:07 21:28:19 borkdude Note that clj-kondo can only reliably detect this right now in places where literal maps are passed
2020:09:07 21:28:36 borkdude there are some attempts around assoc et al, but not fully worked out
2020:09:07 21:31:29 borkdude but I guess something is better than nothing :)
2020:09:07 21:31:43 kenny Ah, right - makes sense.
And totally!
2020:09:07 21:32:38 kenny Wonder how far you could go with it... e.g.,
Technically seems like you could infer that do-moar-stuff will (probably) not get passed the right stuff, assuming the specs are correct.
2020:09:07 21:35:37 borkdude do-stuff can still required a map which key ::b, although it's not required, so I don't see anything wrong here
2020:09:07 21:35:54 borkdude what we could however do is, when do-stuff has ret int? and do-moar-stuff expects a map, give an error
2020:09:07 21:36:34 borkdude clj-kondo already has support for this, it just needs to be mapped from spec to the right format.
2020:09:07 21:36:49 kenny The specs for do-stuff and do-moar stuff don't align and they should. By happenstance, do-stuff could always return ::b but that's just luck.
2020:09:07 21:37:34 kenny A future refactoring to do-stuff could result in ::b getting removed which will cause downstream do-moar-stuff to break.
2020:09:07 21:40:55 borkdude the clj-kondo "type system" will only complain if it's sure something is wrong, so in this case it would not complain I think
2020:09:07 21:41:17 kenny I think that particular scenario would fall under the warning category.
2020:09:07 21:41:28 borkdude type-wise, but maybe it could have a spec linter that gave a warning about this
2020:09:07 21:42:06 borkdude not sure if I want to go very deep into s/keys, since spec2 will have different ways of doing
2020:09:07 21:42:28 borkdude but basic type warnings as you type are already a free win
2020:09:08 06:25:24 ikitommi @borkdude if that’s of any value, there is one decent spec form parser in spec-tools. Doesn’t cover regexs, but lot of other dirty details. There is also a comprehensive regression & progression test suite. e.g. testing that the form bugs are not fixed, together with link to the JIRA issue. Feel free to c&p any parts that you find useful.
2020:09:09 19:56:30 borkdude Bit ugly, but one of those things that could have been deemed: yeah, Rich intended it like this all along, it was all from a design originating back to 2002 ;)
2020:09:09 19:58:14 alexmiller except it's the opposite of what Rich intends :)
2020:09:09 19:58:39 borkdude tell me more
2020:09:09 19:59:45 alexmiller this just plays to the familiarity from static typing langs, which is just not what Rich is trying to do with spec
2020:09:09 20:01:35 borkdude This would not be limited to simple annotations. You could do:
2020:09:09 20:04:57 alexmiller it looks like static typed function signatures, but it's not
2020:09:09 20:05:07 borkdude so?
2020:09:09 20:05:24 alexmiller so making things that are different look the same causes confusion
2020:09:09 20:07:02 alexmiller there are other considerations as well - compilation, evaluation, reuse across arities, combination with other features that don't exist yet, etc
2020:09:09 20:07:41 alexmiller Rich is looking at all that and considering a wide variety of options including the one above
2020:09:09 20:07:59 borkdude I bet
2020:09:09 20:08:15 alexmiller I just think this one is probably unlikely based on what I've heard
2020:09:14 17:42:47 tvalerio Hi! I have this spec in a namespace and a test so I can validate it:
Why can’t I use the spec like ::rates instead of the example in the code? If I try yo use it, I get an unable to find spec error.
Also, when I watch the result of explain-data I can see that it begins with:
#:clojure.spec.alpha{:problems...
And not like:
#:rates{:problems...
As I saw in the examples.
I’m trying to verify which function got error to return a proper error message but I’m getting a hard time to do so. Trying Spec for the first time. How could I do that?
2020:09:15 09:28:21 anthony-galea @tvalerio See the following under the Literals section at https://clojure.org/reference/reader
> A keyword that begins with two colons is auto-resolved in the current namespace to a qualified keyword
So when you use ::rates inside rates.clj it’s as if you’ve written :develop.rates/rates but when you do the same inside rates-test.clj you get develop.rates-test/rates and not :develop.rates/rates
2020:09:15 12:01:04 tvalerio Ok, I understood. Thanks @U446AB17F! =D
2020:09:15 17:35:25 kenny It seems the conformed result of the pred passed to s/coll-of is getting disregarded. Is this behavior of expected?
2020:09:15 17:59:01 seancorfield (the API URL was incorrect)
2020:09:15 18:05:04 alexmiller @kenny I think because you are using :kind map? you're into the map-specific conforming logic which (by default) does not conform keys
2020:09:15 18:07:42 alexmiller You can use :conform-keys to change that default:
That behavior seems correct. Curious if there's a way to indicate to not include the key though. Perhaps only way is to s/and in a conformer that removes nil keys?
2020:09:16 17:52:47 alexmiller no way to do that
2020:09:16 17:53:37 alexmiller in general, use of conformers to do arbitrary transforms is considered an anti-pattern (conformer really exists for building new spec types primarily)
2020:09:16 17:55:54 kenny Yep - definitely in a gray area. Thanks.
2020:09:21 19:27:31 johanatan hi, is there any way to override the comparator used for :distinct calculations for s/coll-of ?
2020:09:21 19:27:55 johanatan or is the best bet just to use s/and with a custom predicate ?
2020:09:21 19:40:49 alexmiller no way to do that currently
2020:09:21 19:41:05 alexmiller but s/and custom pred would work
2020:09:21 19:41:11 johanatan :+1:
2020:09:21 20:46:11 adamfrey is there a way to do something like :gen-max on a regex s/cat spec?
2020:09:21 21:10:07 alexmiller not currently
2020:09:21 21:11:29 alexmiller you can s/& a predicate to only accept smaller colls and that will have the same effect, but it's still producing the larger coll and that at least has memory implications (and may prevent such-that from succeeding at all)
2020:09:24 15:53:17 Jeff Evans From the guide: https://clojure.org/guides/spec#_custom_generators
I’m trying to figure out whether it’s possible to refer to kw-gen (which was just defined in the previous section) here, but I can’t quite figure out the right syntax. For instance, this doesn’t work:
2020:09:24 16:39:18 alexmiller kw-gen here is a generator so you need to wrap it in a no-arg function that returns it (not invokes it)
2020:09:24 17:30:36 Jeff Evans thanks! I misunderstood something fundamental, since I thought #(kw-gen) was equivalent to (fn [] kw-gen)
2020:09:24 17:34:34 alexmiller the former is the same as (fn [] (kw-gen))
2020:09:24 17:35:37 alexmiller I guess you could also do (constantly kw-gen)
2020:09:26 02:39:40 johanatan has anyone given much thought to (for the purposes of using multi-spec liberally across many namespaces) dynamically constructing "qualified keywords" at reader / compile time ?
2020:09:26 02:39:51 johanatan (in clojurescript)
2020:09:26 02:40:16 johanatan i've attempted both a plain macro and a tagged literal reader
2020:09:26 02:41:22 johanatan even the latter apparently only gets expanded and not eval'd at reader time
2020:09:26 02:42:42 johanatan such that:
(s/keys :opt-un [#my/reader "blah/blaz"]) will not work
2020:09:26 02:43:57 johanatan where as I had expected it to get eval'd to: (s/keys :opt-un [:the-current-namespace/blah/blaz]) before keys expansion time
2020:09:26 15:24:19 udit I am trying to use clojure.spec for validating the inputs to my API handlers. What’s the idiomatic pattern for usage here?
I was thinking of creating a generic fn which takes in spec and the data to be validated. This fn can throw an exception when the validation fails. This fn would be called as the very first thing in the handlers, thus if the data is not valid the exception will act like an early exit. But this is using exceptions for control flow.
The other approach I can think of is wrapping all my handlers with an if-else block that do this validation. This is fine, except it sort of makes my handlers less readable.
What’s a clean approach that people have discovered for validating (and coercing) data using spec.
2020:09:26 15:45:00 Jeff Evans Can’t you have a single wrapper fn that basically does
(let [res (s/conform input)]
(if (:s/invalid? res)
(build-error-reponse)
(real-handler input) ; real-handler could be an input to this fn
)
)
Then the real-handler doesn’t have to worry about whether the input is valid, and your outermost layer (i.e. the one that first receives the request) can call this instead of real-handler directly. And if you want more details out of build-error-response you could use explain-str or explain-data instead of conform. Or perhaps I’m misunderstanding the situation.
2020:09:26 16:45:54 udit This works @jeffrey.wayne.evans! Thanks!
Although I would like to have different spec for different handlers, but I presume that can be passed in as a parameter to this fn a well, right?
2020:09:26 16:54:07 Jeff Evans Sure! You would still have to somehow organize the specs per handler. Not sure if there is an established pattern for that but you could always have a keyword based map for them (handler function name as key?)
2020:09:28 13:11:06 arohner When using stest/instrument in tests, how do you clean up after a test? e.g.
`
(deftest foo
(stest/instrument foo {:stub foo})
2020:09:28 13:11:48 alexmiller unstrument
2020:09:28 13:11:57 arohner right, but you don’t know what the previous state was
2020:09:28 13:12:08 arohner was it instrumented, or stub+ instrumented?
2020:09:28 13:13:17 alexmiller Sorry, not understanding the question
2020:09:28 13:14:50 arohner We have some code that is always instrumented. During a test, if I call (instrument foo {:stub foo}), that overwrites the existing instrument with a stub+instrument. Now if I call unstrument at the end, the next test will run with foo un instrumented
2020:09:28 13:16:30 alexmiller Instrument remembers the old var and unstrument replaces it iirc
2020:09:29 08:39:04 Eugen hi, is there some tooling to generate spec code from XSD schema files ?
I would like to validate with clojurescript (if possible) some OFX XML I generate
2020:09:29 14:16:53 fadrian I tried both of the XSD parsers I found using Google search (which looks like they came from the same root). Neither of them seemed to work. clj-xml will parse an XSD, but moving on from the parsed XML to a spec seems to still be a work-in-progress.
2020:09:29 16:13:45 mpenet I have a case where I generate specs, and sometimes I have fwd declarations in spec alises: (s/def ::foo ::bar) where ::bar is not declared/speced yet, so it blows up of course. A dirty way to do that is to do (s/and ::bar) as spec for instance but I am sure there's a better way (short of building a dep graph and specing in order)?
2020:09:29 16:15:07 alexmiller Forward references are actually intended to be ok in specs - the spec alias case (s/def ::foo ::bar) is one known exception to that (there is a ticket for this)
2020:09:29 16:15:32 alexmiller not sure if you've tried (s/spec ::bar) - that might also work
2020:09:29 16:15:46 alexmiller would be slightly less dirty if so
2020:09:29 16:16:21 mpenet I tried s/spec, doesn't work
2020:09:29 16:16:44 alexmiller well, can't say I have a better alternative for you then
2020:09:29 16:16:46 mpenet "unable to resolve spec ::bar "
2020:09:29 16:17:08 mpenet alright, thanks for the info. If it's a bug I'll take the s/and trick as acceptable for now
2020:09:29 16:19:43 mpenet I cannot find the jira for it, but I am not a good at jira'ing
2020:10:02 15:23:53 ashnur thanks. Yeah, I was told not to use spec2 in production by someone
2020:10:05 07:58:09 Jim Newton I'm not a spec user, but I have a few questions about the API. Is it true that spec designates certain symbols as patterns? How can I know whether a given symbol designates a spec pattern? As I understand, given such a symbol, I can call (s/valid? pattern-designator some-value) to determine whether a given value conforms to the pattern. Are such pattern designators always available globally? or can the be defined lexically and thus be invisible to other applications?
2020:10:05 08:01:29 Jim Newton Is s/valid? side-effect free assuming the given pattern is side-effect free?
2020:10:05 08:02:22 Jim Newton Under which situations might s/valid? throw an exception? Do I need to wrap calls in try/catch ?
2020:10:05 13:05:36 alexmiller By “pattern”, I assume you mean “symbolic spec”. Specs have both a spec form (comprised of keyword names, list forms, sets, functions, etc) and a compiled instance form. Spec forms refer to macros that expand to code that emits spec instances. The details of this are quite a bit different in spec 1 and 2. But in both cases the spec forms are available globally as they are defined by macros. s/valid? does not have side effects of its own. s/valid? itself won’t throw but if it includes a predicate, that might throw on a bad input
2020:10:05 13:17:40 Jim Newton Is there a predicate I can use to know whether a given form is a valid "symbolic spec" ?
2020:10:05 13:21:44 Jim Newton Am I correct that s/valid? will throw an error if you give it an invalid symbolic spec?
2020:10:05 13:43:00 alexmiller I think s/spec will either give you a spec instance or an error
2020:10:05 13:43:06 alexmiller s/valid? should do the same
2020:10:05 13:44:30 Jim Newton that's reasonable, yes. So It could be nice if my application could ask spec whether the given spec instance is valid or not, before putting it in a place where at some time in the future when s/valid? is eventually called, spec will throw an error.
2020:10:05 13:45:58 Jim Newton OH, am I correct in assuming that s/valid? checks whether the given data matches the spec? i.e., it does not validate the spec, but rather validates the data...
2020:10:05 14:52:47 alexmiller yes
2020:10:05 15:32:48 Jim Newton so which function should I use to detect whether a candidate given to my by the function who called me, is a valid symbolic spec?
2020:10:05 15:39:59 alexmiller spec 2 is a bit better in this area (but is not ready for use yet)
2020:10:05 15:49:03 alexmiller that's not what he's asking for afaict
2020:10:05 15:59:10 Jim Newton Without knowing spec, it is hard to know really what I'm asking for. But Id like to know whether what I have in variable x is something I can pass as the first argument of s/valid?.
2020:10:05 15:59:28 Jim Newton I suppose my first implementation doesn't to need to verify its values, just trust the caller, and let spec sort it out.
2020:10:05 16:02:14 Jim Newton looks like everything in that registry is either symbol or a keyword. (distinct( map (fn [x] (or (symbol? x) (keyword? x))) (keys (clojure.spec.alpha/registry)))) returns (true)
2020:10:05 16:02:34 borkdude symbols are used for fdefs
2020:10:05 16:59:00 alexmiller s/valid? takes spec names, spec forms, and spec objects - there is no method that validates all of those
2020:10:06 21:34:44 Michael Stokley is there a good way to spec out just some of the arguments in s/fdef's :args? like :args (s/cat :first-param (constantly true) :second-param int?)?
2020:10:06 21:35:02 ghadi any?
2020:10:06 21:35:14 Michael Stokley thanks!
2020:10:12 23:27:12 johnjelinek is there a way to make a spec validate a reference to another key's value? let's say you have a (s/coll double?) and you want to make a key where the predicate is something like (s/and double? #(= % (apply + ,,,)) where the ,,, is that (s/coll double?)
2020:10:12 23:28:02 johnjelinek maybe your schema is like (s/schema [::double-list ::sum])
2020:10:12 23:28:22 johnjelinek I'm prolly not doing a great job of explaining this, does it vaguely make sense?
2020:10:12 23:28:38 alexmiller Can you give an example of valid data?
2020:10:12 23:29:15 alexmiller The example you gave should basically work so I’m trying to make sure I understand what you mean
2020:10:12 23:29:59 alexmiller Are you talking about in a map?
2020:10:12 23:30:01 johnjelinek ya
2020:10:12 23:30:33 alexmiller You can s/and a predicate that validates whatever you want
2020:10:12 23:31:50 johnjelinek ok, so let's say concretely, I've got something like: {::prices '(1 2 3 4) ::total 10} and so total's predicate has to match the collection of prices (except doubles instead of ints)
2020:10:12 23:33:45 johnjelinek so, how would a (s/def ::total)'s predicates (apply + on ::prices?
How do I do generative testing with this kind of spec? any attempt to generate an example of ::grid makes my OS super laggy until I kill the JVM process...
2020:10:13 21:37:17 alexmiller that's not a valid spec? what are you expecting -> to do there?
2020:10:14 05:14:35 vlaaad Oh, sorry, I added threading in the message for readability, actual spec doesn't have it
2020:10:13 21:39:03 alexmiller I guess maybe it would work in the spec 1 macrology, but I find that very confusing to parse (and it won't in spec 2 I think)
2020:10:13 21:39:41 alexmiller in general, setting :gen-max is useful for coll specs though to bound nested coll size
2020:10:14 05:33:19 vlaaad Thanks, I'll try that
2020:10:14 07:04:40 vlaaad It did help, thank you!
2020:10:13 22:27:56 kenny We often build our own spec "types" by doing something like this.
2020:10:13 23:15:13 kenny It's like with-gen needs to take an argument with :gen overrides or implicitly use a dynamic variable.
2020:10:13 23:41:43 kenny Can't figure out any way around this other than to override the top level spec (in my example ::my-custom-map). That can be quite painful with complex specs.
2020:10:14 02:56:13 johnjelinek question regarding preference: do you add your specs to an existing namespace of a domain model or do you have a separate spec namespace to share your specs around?
2020:10:14 17:14:37 alexmiller we do not recommend using regex specs to parse strings
2020:10:15 05:35:50 Ivan Fedorov ack, thanks!
2020:10:18 10:16:57 mike_ananev @alexmiller do you have any plans to release spec2? The winter is coming, and I'll write to Santa my wishes: new Clojure release and Spec2 release in new year. 😆
2020:10:18 12:46:31 alexmiller Sorry, no plans either way
2020:10:18 17:39:04 thom Is there some list of milestones that need to be achieved before we ever see a non alpha Spec? Anything the community should be helping with?
2020:10:18 18:02:09 alexmiller no, it is mostly design work that Rich is working through at the moment
2020:10:18 18:22:24 vlaaad I'm curious what's missing in current spec vision that requires more design work...
2020:10:18 18:36:16 alexmiller redoing function specs
2020:10:18 18:36:43 alexmiller there are other things on the list but that's the big one atm
2020:10:24 17:16:04 wcalderipe hey, is there a way to reuse the same spec for different map keys without having to redefine them over and over?
2020:10:26 20:30:11 gfredericks No. It's your example result a problem?
2020:10:26 20:30:44 waffletower Yes, this is a contrived example, trying to limit stack overflows with a recursive spec
2020:10:26 20:31:37 gfredericks I think recursive specs are generated in a different way and have their own depth control.
Not 100% sure
2020:10:26 20:33:09 waffletower Wrapping the collections within my component specs using recursive-gen has made stack overflows statistically less likely than not using that wrapping. So I thought they might be helpful if they could be further controlled
2020:10:26 20:35:05 waffletower Are there other depth controls beside s/*recursion-limi*t?
2020:10:26 20:40:36 alexmiller no, that's it - there are a few places where they are not being properly checked I think (b/c the gen code wrongly doesn't go through the place it's checked)
2020:10:26 20:40:47 alexmiller that's just a suspicion right now but seems like I've seen that before
2020:10:26 20:41:34 alexmiller and that's only for spec generators, won't affect directly constructed test.check generators
2020:10:30 20:54:28 teodorlu Good evening!
I've written specs to validate a small math language with addition and multiplication. Now I want to generate values! But I'm hitting Stack Overflow. I assume that's because my specs/generators are mutually recursive.
Any tips for writing mutually recursive spec generators that don't blow the stack?
Code attached.
2020:10:30 22:35:16 teodorlu Idea: write generators as functions rather than directly on specs. Argument: recursion depth. Call generator functions from the spec.
2020:10:30 22:51:42 teodorlu Followed through on a generator namespace with recursion limit. That fixed my StackOverflow.
Sample for add and multiply:
2020:10:31 13:48:58 teodorlu Note: Tree depth can also be controlled probabilistically. Make it more probable to take the non-recursive path than the recursive path. Example from the docs:
The probability of the non-recursive path would have to counterbalance the fanout for the recursive paths.
2020:10:31 14:23:13 teodorlu From reading test.check source, I see that many generators accept a size parameter. I suspect that I could have used that instead of inventing my own depth.
2020:10:30 22:52:32 borkdude Grasp now supports finding keywords using clojure specs while also preserving location metadata!
https://github.com/borkdude/grasp#finding-keywords
This should come in handy for finding re-frame events and subscriptions in CLJS apps.
I also made a #grasp channel
2020:10:31 14:02:21 borkdude The ::invalid keyword occurs 67 times in the spec repo:
2020:10:31 14:26:46 teodorlu And you're finding both fully qualified :clojure.spec.alpha/invalid and shorthand ::invalid / ::s/invalid uses? Nice!
I'm curious about use-cases for grasp in tooling for refactoring.
2020:10:31 15:01:45 borkdude yes
2020:10:31 16:55:19 kirill.salykin Hi
please advice, how I can write a spec for :fileds
2020:10:31 19:00:49 kirill.salykin I understand and agree
thought there is a dark magic to get indecies of non-unique elements, but apparently no
thanks for the answer
2020:10:31 19:01:12 borkdude There might be, but I don't know if it's pretty
2020:10:31 19:03:47 kirill.salykin maybe there is a way shape spec output in a way as it reports per failed element (eg with s/problems, path (`:in []`) and predicate?)
2020:10:31 19:10:06 borkdude @kirill.salykin Maybe something like this:
user=> (s/def ::unique-names (s/and #(group-by :name %) (s/every-kv any? #(= 1 (count %)))))
:user/unique-names
user=> (s/explain ::unique-names [{:name :foo} {:name :foo}])
Execution error (UnsupportedOperationException) at user/fn (REPL:1).
nth not supported on this type: PersistentArrayMap
but I'm getting an exception
2020:10:31 19:11:14 kirill.salykin interesting…
so (s/and ) behaves like ->?
2020:10:31 19:11:31 Dmytro Bunin no afaik
2020:10:31 19:16:39 borkdude It needed s/conformer around the first spec
2020:10:31 19:15:45 borkdude @kirill.salykin Ah, I needed s/conformer:
2020:10:31 19:16:17 kirill.salykin you are magician! thank you so much!
2020:10:31 20:23:57 borkdude I've made binary versions of grasp available in the #grasp channel now. Clojure spec on the command line, yeah :)
2020:11:03 12:39:11 borkdude Is there any existing work combining clojure-spec conform + core.match or datalog? Like: I want ?x and ?y from (s/conform ...)? I could see this being useful for grasp
2020:11:03 12:58:48 borkdude thanks. can you describe the difference between meander and yours?
2020:11:03 13:01:40 delaguardo • no macro
• less power -> cleaner pattern syntax (this is just my opinion, btw)
• much smaller codebase -> much easier to read through and explore
2020:11:03 13:02:32 borkdude 👍
2020:11:03 14:34:46 borkdude @U04V4KLKC How do I express this meander usage in your lib?
2020:11:03 15:57:40 borkdude Is there a way to get a list of only the symbols like with meander?
2020:11:03 16:00:01 delaguardo sorry, I was too fast copypasting )
instead of mc/scan please use mc/each
2020:11:03 16:00:43 delaguardo scan is for inspecting items in collection individually (in separate decision branches)
each is for walking over a collection
2020:11:03 17:51:27 delaguardo another pusher to polish documentation 😉
2020:11:03 12:41:21 borkdude Maybe meander is a good match for this
2020:11:03 15:27:04 Jim Newton I'm starting looking at clojure.spec.alpha can someone explain the different between the s/valid? function and the s/spec? function? Their docstrings look pretty similar.
2020:11:03 15:27:58 borkdude @jimka.issy spec? is more like an instance check, valid? validates a value against a spec
2020:11:03 15:28:36 borkdude in fact, it is an instance check :)
2020:11:03 15:33:40 Jim Newton As I understand the normal use case is the user associates certain tags, normally starting with :: with some DSL object which tells how to check validity? Does spec? check whether the symbol has been associated with a spec object, or does it check whether this is such an object?
2020:11:03 15:34:01 Jim Newton How can I know whether a symbol such as ::xyzzy designates a spec or not?
2020:11:03 16:09:29 alexmiller s/describe returns a shorter form, useful for printing (but not fully resolved so not too useful as data)
2020:11:03 16:10:38 Jim Newton seems like s/form does what I want. But it's not certain.
2020:11:03 16:11:03 alexmiller research question - if you are using specs with aliases, do you tend to use one alias, a few, or many in any given namespace?
2020:11:03 16:12:18 borkdude what do you mean by an alias?
2020:11:03 16:12:41 borkdude like (alias 'foo 'bar) and then ::bar/spec ?
2020:11:03 16:13:05 borkdude @alexmiller I think grasp could be used to find this out
2020:11:03 16:13:39 Lennart Buit one reason for me to use many aliases would be if many keys in a s/keys require a same generator. For one, we have a present string spec, with corresponding generator that we alias a lot
2020:11:03 16:17:34 Lennart Buit ah nevermind then, I interpreted this as an alias:
I'm guessing that the function #(> % 1000) is a closure in its current lexical context. Right? So later when I call s/form I don't get back that object, but some new text, which doesn't know the original lexical context. Right?
2020:11:03 17:26:38 borkdude @jimka.issy The text #(...) is not representable as an s-expression, it's something built into the reader which expands to (fn [...] ...)
2020:11:03 17:27:23 borkdude What simply happens is that s/def is a macro which captures the input and by that time the s-expression has already expanded to the fn form
2020:11:03 17:36:14 alexmiller the spec function will capture the lexically scoped values at the point of definition (but the form will not - it will have the names)
2020:11:03 17:36:54 alexmiller ^^ form is broken there
2020:11:03 17:42:05 alexmiller this is all somewhat more consistent in spec 2 (namely, this will not work at all as the divide between symbolic specs and spec objects is clearer) but there is built-in support for creating parameterized specs (like the one above) if needed
2020:11:04 08:51:46 Jim Newton @alexmiller, thanks for the explanation. however, i'd like to get the actual function object which spec would use to validate with. For example given something like (s/and int? #= % x)), I can use int? to get the int? function, but I can't use the s-expression form of the closure to recuperate the closure. However, I'm pretty sure spec has the closure somewhere, otherwise s/valid? would not work.
2020:11:04 08:58:52 Jim Newton What I'm trying to do is figure out which spec incantations can be expressed in a simple type system which I have implemented, called genus , normally abbreviated gns/.
For example, a naïve first attempt is able to translation ::big-even as follows
clojure-rte.rte-core> (gns/canonicalize-type '(spec ::big-even))
(and
(or Long Integer Short Byte)
(satisfies clojure.core/even?)
(spec (clojure.core/fn [%] (clojure.core/> % 1000))))
So genus can unwind the spec into types and predicates, if the predicates are symbolic. But what's left (clojure.core/fn [%] (clojure.core/> % 1000)) is not something which is useable.
2020:11:04 09:00:26 Jim Newton I could try to write a translator/compiler for expressions such as (clojure.core/fn [%] (clojure.core/> % 1000)) which try to convert them back to a closure, in the case that they don't have any free variables. But that work seems redundant since spec already has the closure in its internal data structure.
2020:11:04 09:09:04 Jim Newton I just realized something interesting. If I eval the list (clojure.core/fn [%] (clojure.core/> % 1000)) then I get a function object which seems to have the same semantics as the original function, in case there are no free variables.
Perhaps that is good enough for my proof-of-concept.
2020:11:04 09:10:56 Jim Newton And if there's a free variable, eval throws an exception
clojure-rte.rte-core> (eval '(clojure.core/fn [%] (clojure.core/> x 1000)))
Syntax error compiling at (clojure-rte:localhost:51477(clj)*:45:51).
Unable to resolve symbol: x in this context
2020:11:04 14:11:53 alexmiller Well eval is literally the function that takes a form and returns a function
2020:11:04 10:05:39 Jim Newton Yet another question about spec.
This time about the semantics of the sequence designators such as s/alt, s/+, s/* etc.
an element such as (s/+ even?) means (if I understand correctly, a sequence of 1 or more objects for which the even? predicate is true.
but I can substitute a spec name for even? and say (s/+ ::big-even) and that will be a sequence of 1 or more objects each of which validates the ::big-even spec.
However, what does (s/+ (s/alt :x even? :y prime?)) mean. I would guess that it means a sequence of 1 or more elements each of which either satisfy even? or satisfy prime?
But if I define
what does this mean? (s/+ ::even-or-prime)
It is a sequence of 1 or more sequences (each of which contains elements which are either even or prime), or is it a sequence of 1 or more integers each of which is either even or prime?
2020:11:04 10:06:38 Jim Newton I'm pretty sure spec has a way to designate both, as they are both possible things the user might want to express. right?
2020:11:04 10:07:46 borkdude @jimka.issy s/alt is for regexes, use s/or for normal predicates
2020:11:04 10:08:48 borkdude (s/+ (s/alt ....)) means a sequence of one or more alternatives
2020:11:04 10:09:36 borkdude it doesn't mean each, this can be expressed with s/every or s/coll
2020:11:04 10:11:23 Jim Newton Sorry, my question too convoluted. Let me ask simpler.
If ::foo is a spec defined somewhere using s/def , what does (s/+ ::foo) mean? Does it mean a sequence of 1 or more objects which each validate ::foo ?
2020:11:04 10:11:48 borkdude yes
2020:11:04 10:12:49 Jim Newton So it is not referntially transparent?
2020:11:04 10:13:07 Jim Newton I.e. substituting the expression for ::foo in place, changes the meaning?
2020:11:04 10:13:32 borkdude I don't see why that would be the case?
2020:11:04 10:17:21 Jim Newton suppose
(s/def ::foo (s/alt :x even? :y prime?))
Now
(s/+ (s/alt :x even? :y prime?)) matches a sequence of 1 or more integers each of which are either even or prime, right?
but (s/+ ::foo) matches a sequence of 1 or more objects which each satisfy :foo , i.e., a sequence of sequences each of which contain even or prime integers.
2020:11:04 10:20:34 borkdude @jimka.issy Please give a working example. E.g.:
2020:11:04 10:24:57 Jim Newton yes, but you answer avoids my question. I'm not asking how to write a good spec. I'm asking about the semantics.
2020:11:04 10:25:29 borkdude And I asked you about a counter-example of referential transparency. A working one without imaginary functions like prime? which does not exist in core.
2020:11:04 10:25:54 Jim Newton Ok here is a very simple one.
2020:11:04 10:26:17 vlaaad I mean you don’t really need prime? source to know its semantics
2020:11:04 10:26:28 Jim Newton (s/+ ::foo) matches a sequence for which all the elements are either even or negative.
2020:11:04 10:26:55 vlaaad looks right
2020:11:04 10:27:15 borkdude Your words are ambiguous to me. Do you mean: every element is a ..., or on a case by case basis?
2020:11:04 10:27:52 Jim Newton lets stick with integers and sequences thereof.
2020:11:04 10:28:35 Jim Newton Now define
(s/def ::foo2 (s/+ ::foo))
2020:11:04 10:29:31 Jim Newton what does (s/+ ::foo2) mean. It does not mean a sequence of 1 or more objects which of which match ::foo2
2020:11:04 10:30:00 borkdude What does ::foo mean in your code here?
2020:11:04 10:30:38 borkdude some kind of integer that's either this or that?
2020:11:04 10:30:54 Jim Newton
(s/def ::foo (s/or :x even? :y neg?))
2020:11:04 10:31:22 borkdude then ::foo2 means multiple even or negs, but they can be mixed. it's not all or nothing
2020:11:04 10:31:23 vlaaad I would. say (s/+ ::foo2) is a seq of 1 or more objects each of which is is ::foo2, why would you say it isn’t that?
2020:11:04 10:38:00 Jim Newton Now ::foo matches even and negative integers
2020:11:04 10:38:27 Jim Newton (s/+ ::foo) matches a non-empty sequence of even and negative integers
2020:11:04 10:39:00 vlaaad sounds right
2020:11:04 10:39:40 Jim Newton but (s/+ ::foo2)does not match a non-empty sequence of objects which match ::foo2
2020:11:04 10:40:16 vlaaad ah, I think that’s because of how regex ops nest
2020:11:04 10:40:54 Jim Newton
2020:11:04 10:41:07 Jim Newton yess!!!!!!!!
2020:11:04 10:41:36 Jim Newton this was my original question (s/+ x) does not match a non empty sequence of things which match x
2020:11:04 10:41:48 borkdude @jimka.issy If you want to do this, you need to use s/spec around the ::foo2
2020:11:04 10:42:09 borkdude This has to do with regex nesting indeed, not with referential transparency. Read the spec guide, which explains this
2020:11:04 10:42:59 borkdude > When regex ops are combined, they describe a single sequence. If you need to spec a nested sequential collection, you must use an explicit call to spec to start a new nested regex context.
2020:11:04 10:47:45 Jim Newton so does this mean that if a sequence operator is directly within another sequence operator it has this special modal/merging meaning. otherwise it has its normal meaning?
2020:11:04 10:47:46 borkdude (or maybe not that one, but the paper where that paper is based on)
2020:11:04 10:48:13 borkdude > When regex ops are combined, they describe a single sequence. If you need to spec a nested sequential collection, you must use an explicit call to spec to start a new nested regex context.
2020:11:04 10:48:36 borkdude s/alt returns a regex op, s/or is for combining predicates
2020:11:04 10:48:46 borkdude so s/or is not a regex op.
2020:11:04 10:52:53 Jim Newton the system I devised (in 2016) and after, is called RTE (regular type expressions).
2020:11:04 10:53:18 borkdude cool!
2020:11:04 10:53:19 Jim Newton What i'm trying to do now is figure how similar or different it is from spec at a theoretical leve.
2020:11:04 10:53:32 Jim Newton So I'm trying to see if I can "compile" a spec into an RTE.
2020:11:04 10:54:13 Jim Newton if so, then an RTE can me validated in linear time with no backtracking.
2020:11:04 10:54:36 Jim Newton as I understand it, a spec takes exponential time worst case, so users avoid worst cases.
2020:11:04 10:55:13 Jim Newton however, since I have found no academic papers on spec, and I am not an expert of spec, I'm just guessing. Maybe I'm completely wrong about my supposition
2020:11:04 10:55:15 borkdude @jimka.issy I'm interested in that for clj-kondo as well. Clj-kondo has a tiny type system in which I try to avoid backtracking of type checking arguments. Currently by just not allowing sophisticated constructs :)
2020:11:04 10:56:43 Jim Newton I submitted a paper recently to the Dynamic Languages Symposium where I introduced a Simple Type System for use in dynamic languages. The paper was rejected. One of the reasons for rejection was the revewers didn't see practical applications.
2020:11:04 10:57:45 Jim Newton a simple type system allows you to reason about types (where type is defined as any set of values) . Furthermore intersections, unions, and complements of types are also types.
2020:11:04 10:58:19 Jim Newton If your type logic can conclude that a type is empty, then you've usually found a bug in the user's code.
2020:11:04 10:58:29 borkdude It sounds like it could be very useful for clojure. Are you aware of the work done in core.typed?
2020:11:04 10:58:42 Jim Newton Thats Ambrose?
2020:11:04 10:58:47 borkdude yes
2020:11:04 10:59:11 Jim Newton He's looked at my work and is interested in helping out. but everyone is busy.
2020:11:04 10:59:41 Jim Newton I'd also love to have a student researcher interested in working with clojure.
2020:11:04 10:59:44 borkdude It looks like your work could help improve clj-kondo's type system since linear checking is necessary for performance
2020:11:04 11:00:33 Jim Newton what's clj-kondo written in? is it written in clojure?
2020:11:04 11:00:40 borkdude yes
2020:11:04 11:01:02 Jim Newton are you more of a hacker, or more of an academic?
2020:11:04 11:01:23 Jim Newton or more of an engineer? I don't mean that in any way insulting. sorry if it sounds as such....
2020:11:04 11:01:29 borkdude haha. I do have a CS master degree, but not a Phd in type systems ;)
2020:11:04 11:02:24 borkdude clj-kondo focuses on usefulness though, whereas something like core.typed is maybe more an academic project
2020:11:04 11:02:35 borkdude I don't care about publishing papers, I want the tool to help me
2020:11:04 11:04:13 Jim Newton I'm not sure why my next best step should be.
For the moment I have an implementation of genus, (the simple type system in clojure) and RTE (regular type expressions) which allows genus to represent sets of sequences which match regular expressions in terms of types. Genus claims to be extensible.
For the moment my experimental task is see if I can indeed extend genus (in user space) to incorporate the spec type. I.e., given a spec, the type will be the set of all objects which validate the spec.
2020:11:04 11:05:33 Jim Newton spec types will be treated as opaque types such as arbitrary predicate. except in the case where I can compile a spec into more fundamental types in which case the system can reason about them.
2020:11:04 11:05:46 borkdude Another tool I made recently also uses clojure.spec for searching code:
https://github.com/borkdude/grasp
This could potentially made faster
Another project which tries to use spec at compile time:
https://github.com/arohner/spectrum
It's experimental in nature.
2020:11:04 11:06:10 Jim Newton for example it will be able to decide whether the set of objects satisfying spec1 and also spec2 is empty. or decide whether two given specs are equivelent.
2020:11:04 11:07:53 Jim Newton in order for RTE to represent the regular type expression as a deterministic finite automaton, it needs to be able to determine whether two given types are disjoint. This is currently not possible with spec in the most general case, but could be possible in many particular cases.
2020:11:04 11:08:50 Jim Newton As I understand spec's internal data structure uses non-deterministic finite automata, thus the exponential behavior in the worst case.
2020:11:04 11:09:42 Jim Newton ANYWAY, maybe now you understand a bit better the reason for my strange beginner questions about the semantics of spec ???
2020:11:04 11:09:54 borkdude yes, thanks for explaining!
2020:11:04 11:10:26 borkdude The authors of spec usually emphasise that spec is not a replacement for a type system, but a runtime validation system.
2020:11:04 11:10:43 Jim Newton in my opinion there is no difference
2020:11:04 11:14:42 borkdude yes, if you define it like that, it makes sense. but this set of values can often not be known at compile time, which makes the difference between static compile time checking
2020:11:04 11:15:45 Jim Newton what is s/fdef ?
2020:11:04 11:16:54 borkdude clojure.spec supports defining specs for checking function arguments and return values using fdef
2020:11:04 11:17:39 Jim Newton yes many times you can indeed know a lot about a type at compile time. in those cases, compilers like the SBCL common lisp compiler can create more efficient code, or tell the user about errors or unreachable code.
2020:11:04 11:18:17 Jim Newton so an fdef decribes what a valid function call site looks like?
2020:11:04 11:18:40 borkdude yes
2020:11:04 11:19:45 Jim Newton I can see how the predicate based assumption of spec, could be a bottle neck for a code analyzer like kondo.
2020:11:04 11:20:15 borkdude I initially used spec in clj-kondo, but I found the performance not good enough
2020:11:04 11:20:42 Jim Newton one think that I've noticed is that many many many of these predicates can be rewritten as a simple type check. And I have code which automates this.
2020:11:04 11:21:02 Jim Newton ahhh, so you have abandoned that piste now?
2020:11:04 11:22:04 borkdude the format now uses something more explicit: you provide the seq of types per arity. so you don't have to do any backtracking to to match a seq based on some s/alt expression.
which makes more sense to me considering the structure of multi-arity functions.
2020:11:04 11:22:31 borkdude and a seq of types is usually a fixed number of things or a fixed number of thing + a variable amount of things of the same type
2020:11:04 11:22:46 borkdude I think this matches 99% of cases how people call functions in clojure
2020:11:04 11:23:06 Jim Newton for example
(gns/canonicalize-type '(satisfies int?))
==> (or Long Integer Short Byte)
(or ...) is a union type. when intersected and unioned with other types, it can be arranged so that redundant checks are eliminated.
2020:11:04 11:23:37 borkdude clj-kondo also supports union types, it's just #{:int :string}
2020:11:04 11:24:12 borkdude it also tries to infer returns types and threads types through let expressions
2020:11:04 11:24:58 borkdude Example:
2020:11:04 11:25:02 Jim Newton so it has type inferencing?
2020:11:04 11:25:43 borkdude It has inferencing yes, but limited to the things it knows, else it's any? and it won't complain
2020:11:04 11:26:05 Jim Newton I wrote something pretty similar some years ago for SKILL++ which was a proprietary lisp used by the company I worked for for many years.
2020:11:04 11:28:50 Jim Newton in my system, called "loathing" it would see that foo returns type string (because of the definition), and the call site inc expects an integer. So the type at that point is (and integer string) which is the empty-type.
2020:11:04 11:29:47 borkdude clj-kondo does something similar. but it just has a simple graph of things that are reachable or not: you can never go from string to int, so this is an error
2020:11:04 11:30:09 borkdude but you can go from any to int, or any to string, so this is not an error
2020:11:04 11:30:16 Jim Newton however, if inc had input type (or integer string), then the intersection would be (and string (or string integer)) = string != empty-set
2020:11:04 11:30:31 borkdude yep, same thing in kondo
2020:11:04 11:31:37 borkdude e.g.: this will not give a warning:
2020:11:04 11:32:11 Jim Newton You mentioned that you are not interested in publications. But I need to make publications. It's part of my job.
2020:11:04 11:32:29 borkdude Well, I meant that this is not the goal of clj-kondo.
2020:11:04 11:32:36 Jim Newton I need to get several publications out of this work.
2020:11:04 11:32:39 borkdude It's not the product of academic research
2020:11:04 11:32:51 borkdude I have nothing against making publications
2020:11:04 11:33:05 borkdude I do have my name on one publication ;)
2020:11:04 11:33:33 Jim Newton and I think there are several interesting results to publish. But i'm a very poor publicist. I don't do well at convincing people of the interest or novelty of my work.
2020:11:04 11:34:11 borkdude If you could get practical benefit out of your work by e.g. improving clj-kondo or a similar tool, I'm sure it will convince people
2020:11:04 11:35:32 borkdude @jimka.issy I'm sharing this for fun, but it's also related:
https://borkdude.github.io/re-find.web/
Find functions based on example in and outputs
2020:11:04 11:35:46 borkdude This uses the speculative specs to search for matches
2020:11:04 11:37:01 Jim Newton the clojure community is friendly for the most part. although I do see some snide comments now and then.
2020:11:04 11:37:11 Jim Newton So it is usally pretty easy to get questions answered.
2020:11:04 11:37:40 Jim Newton I would love to have some spec examples for test cases.
2020:11:04 11:37:44 borkdude I would say the clojure community is better in that respect than some others
2020:11:04 11:38:27 Jim Newton Could I solicit you to give me some spec examples, simple and complex. some which are trivial, and some which are very slow to validate?
2020:11:04 11:38:53 Jim Newton For the moment I'm just worried about sequences, not functions or maps or data structures. My code is not advanced enough to handle those.
2020:11:04 11:39:20 Jim Newton I want to try to compile these to RTE and see if I can come up with some cases where RTE is faster.
2020:11:04 11:39:35 Jim Newton if RTE is never faster, then i'm in trouble
2020:11:04 11:39:37 borkdude You could take a look at speculative, which has more than hundred specs
2020:11:04 11:40:59 borkdude I don't have a particular example of a slow spec, but I guess you can fabricate one based one which does backtracking
2020:11:04 11:41:18 Jim Newton when analyzing code statically, i suppose you are usually looking at nested sequences, not really at maps and such.
2020:11:04 11:41:58 borkdude This tool: https://github.com/borkdude/grasp searches through code. It applies the spec at every sub-tree of an s-expression
2020:11:04 11:42:09 borkdude So performance improvements would be really noticeable there
2020:11:04 11:42:49 borkdude I have to go now. speak to you later
2020:11:04 11:45:20 Jim Newton ok, thanks. talk later
2020:11:04 11:45:40 Jim Newton BTW i filed an issue for kondo, was it helpful?
2020:11:04 11:48:05 borkdude Yes, thank you. I'm not sure if clj-kondo will be able to support this, as it sees one file as a standalone unit and in-ns doesn't really match with this model. So either using in-ns with :refer on the original namespace requires you to help clj-kondo with some annotations or config, or clj-kondo has to make non-trivial changes.
2020:11:04 11:48:44 borkdude I will think about this for a while
2020:11:04 11:49:13 Jim Newton gocha
2020:11:04 13:35:44 Jim Newton I don't think s/& is a regular expression operation. It doesn't seem regular. as far as I can tell, it requires the implementation to remember the sequence being tested. a regular expression demands that every decision be based only one the current object of the sequence, and a state. whereas only a finite number of states are allowed. In this case if there are N states, you cannot test a sequence of length N+1.
2020:11:04 10:31:33 Jim Newton Wait. I think I'm confused. Let me work out a more consistent example. And come back to you in 5 minutes. OK?
2020:11:04 10:31:39 borkdude ok
2020:11:04 10:35:28 Jim Newton taking this conversation to a thread. I hope everyone follows.
2020:11:04 11:47:42 Jim Newton another spec question. Are all spec definitions global? If I'm writing test cases, I'd love to introduce local definitions that don't pollute the global space.
2020:11:04 11:54:25 borkdude @jimka.issy Yes, spec has one global registry (compare spec keywords to RDF uris for example). Malli (#malli) which aims to be an alternative schema library has local registries as well.
2020:11:04 11:55:48 Jim Newton I suppose for test cases I can use ::xyzzy to keep the symbols in my local namespace.
2020:11:04 11:59:00 Jim Newton Do I understand correctly that there is no intersection equivalent of s/alt for regex semantics. I.e., s/or is to s/alt as s/and is to what?
2020:11:04 12:08:25 Jim Newton syntactically does s/& work like s/alt and s/cat in that I need to tag the components? or like s/and where no tags are used?
2020:11:04 12:09:27 borkdude @jimka.issy In the guide, search for:
> Spec also defines one additional regex operator, &, which takes a regex operator and constrains it with one or more additional predicates.
to see an example
2020:11:04 12:19:07 Jim Newton ahh so s/& is not really analogous to s/alt it is yet a third syntax and semantic.
2020:11:04 12:19:48 borkdude I thought you were asking analogous to s/and.
> I.e., s/or is to s/alt as s/and is to what?
2020:11:04 12:25:59 borkdude s/alt and s/& are supposed to be used to build regexes, s/and and s/or are just general ways of combining predicates
2020:11:04 12:26:06 borkdude but sometimes they do the same thing
2020:11:04 12:26:42 Jim Newton It looks (from the documentation) that s/& is not analgous to s/and in the same way that s/alt is analogous to s/or. if they were analogous I'd expect s/& to work like this.
2020:11:04 12:49:41 Jim Newton It looks to me (on first and second reading of the specification) that (s/& pattern predicate) means that the subsequence which matches the pattern must as a sequence match the predicate. So the following
matches a sequence which begins with an even number of strings and ends with an odd number of integers.
Is my interpretation correct?
I don't think these are really regular expressions any more. I have to think about it, but my suspicion is that this cannot be solved without using a stack.
2020:11:04 14:00:50 Jim Newton How can I recognize an object such as the one returned by
because when foo gets called by the application, the name space might not have required [clojure.spec.alpha :as s]
Instead, I did the following and it seems to work.
because when foo gets called by the application, the name space might not have required [clojure.spec.alpha :as s]
Instead, I did the following and it seems to work.
2020:11:05 17:06:20 andy.fingerhut Note that you have no * after clojure.spec.alpha/ I do not know if that is a typo in chat, or reflects your actual code.
2020:11:05 17:06:41 Jim Newton yes a copy/paste error. corrected, should be /* everywhere
2020:11:05 17:07:41 andy.fingerhut Now I see clojure.spec.alpha. Did you intend to use clojure.spec.alpha/* instead?
2020:11:05 17:08:48 andy.fingerhut defn foo is in the namespace some-ns ? If so, how does some part of your code call foo without first require'ing some-ns ?
2020:11:05 17:09:00 Jim Newton important observation is that the behavior of resolve depends on the value of *ns* when foo is called, not when foo is defined.
2020:11:05 17:09:57 Jim Newton the caller of foo has required some-ns, but he might not have aliased closure.spec.alpha to s
2020:11:05 17:10:06 andy.fingerhut Got it. Yes, that is true, and resolution of aliases like s definitely depends upon the namespace in which they are resolved.
2020:11:05 17:11:25 Jim Newton i found out because I ran my tests from the repl where I had required spec and aliased to s. But when I tested from lien test at the comand line, it ran the tests with the user namespace, which I never tested in.
2020:11:05 17:11:31 andy.fingerhut Alias s could be not an alias in some namespace, be used for clojure.spec.alpha in another namespace, and be used for clojure.set in yet another namespace. Not recommended for developer sanity to use the same alias for different namespaces, but Clojure allows it.
2020:11:05 17:12:37 Jim Newton besides, when I develop the code, I use some convention. But I don't impose that convention on my customers who might have very different programming styles. or they theoretically might even be calling my code from java or some such
2020:11:05 17:12:59 andy.fingerhut The doc string for resolve gives a pretty strong hint about this dependence upon the current value of *ns*
2020:11:05 17:14:04 Jim Newton I think all calls to resolve or suspect within code.. I'm thinking of refactoring every call in my program and replacing with ns-resolve so I'll have to think about which ns to use. Then test from repl and also from lein command line as a general pratice
2020:11:05 17:14:24 Jim Newton it's on my to-do list.
2020:11:05 17:15:45 andy.fingerhut Using ns-resolve instead of resolve sounds like a good idea, if you want to force yourself to be explicit about in which namespace the resolving is performed.
2020:11:05 17:17:39 Jim Newton of course nothing prohibits me from using (ns-resolve *ns* foo) if that's the appropriate one to use.
2020:11:07 20:39:02 borkdude What was the take on spec and coercions again - spec isn't meant to do coercions of values (during conformation), rather it only validates or conforms values, right?
Then what is the opinion on a library like this one? https://github.com/exoscale/coax
2020:11:07 21:01:59 seancorfield My reading of comments by Alex, at least, is that Specs should not do coercions, but libraries that infer coercions from Specs, and do those coercions "outside" Spec are a good way to do stuff.
2020:11:07 21:03:28 seancorfield Our "web specs" library does the coercions internally in the Specs ("bad") but I've been looking at moving to spec-coerce instead (or maybe coax) and deprecating our library @borkdude
2020:11:08 06:10:52 mpenet coax is used heavily in production, so far so good :) we fixed a number of bugs from the original spec coerce fork and made it more flexible and performant. I am not sure I would still call coax a fork, the source is now totally different.
2020:11:08 06:17:18 mpenet spec-coerce outlined some good ideas but it felt a bit like a POC, I am not sure it is/was used seriously.
2020:11:09 14:53:50 Jim Newton how can I recognize an object such as the one returned from (s/or :1 int? :2 number?) . I see that s/regex? , s/get-spec, and s/spec? all return nil. I notice however that the class of the object has (type (s/or :1 int? :2 number?)) in its ancestors?
2020:11:09 14:56:45 Jim Newton Next question, how can I extract int? and number? from the object? I see that (keys (s/or :1 int? :2 number?)) throws an Exception.
(keys (s/or :1 int? :2 number?))
Execution error (IllegalArgumentException) at clojure-rte.genus-spec-test/eval17912 (form-init5322407590801644377.clj:30310).
Don't know how to create ISeq from: clojure.spec.alpha$or_spec_impl$reify__2118
I see that (s/describe (s/or :1 int? :2 number?)) returns the list (or :1 int? :2 number?) . However, to recognize the object as one which has a describe method.
2020:11:09 15:07:31 alexmiller it's important to be clear in talking about the two worlds of spec - spec forms and spec objects. (s/or :1 int? :2 number?) is a spec form, which when evaluated, returns a spec object (something that satisfies the spec protocol). Trying to nail down concrete types for the latter part is going to be a bad time - the important thing is the protocol which is inherently polymorphic.
2020:11:09 15:08:19 Jim Newton so how do I know whether I have an object which satisfies the spec protocol?
2020:11:09 15:09:30 alexmiller spec 1 does not have great answers for extracting information from either spec objects (which are opaque) or spec forms. One path to this is to write specs for spec forms and then use s/conform to "parse" the forms. A significant stab at this exists in https://clojure.atlassian.net/browse/CLJ-2112 but I think the future direction is really making a more data-centric representation which is one thing we are doing in spec 2 (but that's still very much a work in progress)
2020:11:09 15:11:05 alexmiller spec? is the predicate for that
2020:11:09 15:11:25 Jim Newton yes but spec? returns nil.
2020:11:09 15:13:00 Jim Newton aaaaaaaaahhhhhhhhh!!!!!! yikes. s/spec? returns non-nil. That's EXCELLENT I have a local function named gs/spec? which just asks whether it is a sequence whose first element is spec. spec? was my first guess. just was shadowed by a local function. Cool. thanks.
2020:11:09 15:13:17 alexmiller yeah, your predicate is asking a symbolic spec form question
2020:11:09 15:13:25 Jim Newton and I just typed spec? at the REPL.
2020:11:09 15:13:42 Jim Newton REPL issue.
2020:11:09 15:17:41 Jim Newton is there a way to undef a function. Sometimes I rename a function while debugging, and the old function (of course) is still defined in vm, and if my code accidentally calls it because I didn't find and change all references, that introduces bugs which are hard to find.
2020:11:09 15:26:07 borkdude @jimka.issy You can use (s/fdef foo/bar nil)
2020:11:09 15:26:33 borkdude (I assume, I know (s/def ::foo nil) at least works)
2020:11:09 15:29:08 alexmiller yes, s/def to nil
2020:11:09 15:29:12 Jim Newton what is s/ ?
2020:11:09 15:29:19 alexmiller clojure.spec.alpha
2020:11:09 15:29:31 Jim Newton you mean use spec to redefine a function?
2020:11:09 15:29:46 alexmiller are you asking about function specs or functions?
2020:11:09 15:30:05 alexmiller I think @borkdude and I assumed you meant specs since we're in the spec channel here
2020:11:09 15:30:06 Jim Newton funtions. Sorry if I mistyped before?
2020:11:09 15:30:16 Jim Newton sorry, its probably my fault.
2020:11:09 15:30:47 alexmiller you can use ns-unmap
2020:11:09 15:30:47 Jim Newton (def name nil) works of course.
2020:11:09 15:31:00 alexmiller ^^ that doesn't unmap, just redefines
2020:11:09 15:31:07 Jim Newton ahhh ns-unmap. thats the cleaner way.
2020:11:09 15:31:41 borkdude @jimka.issy
user=> (def x 1)
#'user/x
user=> (ns-unmap *ns* 'x)
nil
user=> x
Syntax error compiling at (REPL:0:0).
Unable to resolve symbol: x in this context
user=> user/x
Syntax error compiling at (REPL:0:0).
No such var: user/x
2020:11:09 15:32:16 borkdude I recently discovered ns-unmap also works for imported classes. I need to fix a bug in sci which doesn't support that yet :/ ;)
2020:11:09 15:32:23 Jim Newton I have several functions which are memoized. it speeds up my program 1000 fold. But when debugging, it can be a source of errors, because I forget that the function might not really be called.
2020:11:09 15:32:45 alexmiller the double-edged blade of memoization :)
2020:11:09 15:33:13 Jim Newton indeed. So sometimes I really want to set-the-d*mn-function to undefined
2020:11:09 15:33:21 Jim Newton to make sure it's not a problem of memoization
2020:11:09 15:33:23 borkdude @jimka.issy A solution to that problem might be to call the original function foo* and the memoized one foo
2020:11:09 15:34:09 Jim Newton Here's the macro I'm using.
(defmacro defn-memoized
[[public-name internal-name] docstring & body]
(assert (string? docstring))
`(let []
(declare ~public-name) ;; so that the internal function can call the public function if necessary
(defn ~internal-name
2020:11:09 15:35:52 borkdude for tests you can also use with-redefs which doesn't require your vars to be dynamic
2020:11:09 15:36:11 Jim Newton never heard of with-redefs. what's that.
2020:11:09 15:36:25 borkdude (doc with-redefs)
2020:11:09 15:38:08 Jim Newton indeed. so that just saves needing to declare them as dynamic? or does it do something else?
2020:11:09 15:38:38 borkdude dynamic bindings aren't visible to all threads, with-redefs changes the root binding of vars temporarily
2020:11:09 15:39:46 alexmiller (note that there are issues using with-redefs with concurrency, including future etc)
2020:11:09 15:40:28 Jim Newton @U064X3EF3 are these the same issues with any dynamic variable?
2020:11:09 15:40:43 borkdude yeah, when running tests concurrently this may bite you
2020:11:09 15:41:15 Jim Newton Aren't dynamic variables thread-local in clojure?
2020:11:09 15:41:44 borkdude yes, but with-redefs doesn't use thread-local, it's just a global mutation that gets restored afterwards
2020:11:09 15:42:14 Jim Newton ah ha.
2020:11:09 15:42:43 borkdude Personally I don't have any test suites that run into problems with this, but then again, I hardly use with-redefs at all
2020:11:09 15:43:29 borkdude There could be a performance penalty to marking all your vars dynamic, but not if they haven't been dynamically re-bound yet since there's a pretty efficient check for that happy path
2020:11:09 16:00:16 alexmiller we regularly see people run into failing test suites when using with-redefs
2020:11:09 16:00:58 alexmiller because they don't understand what it does, which is why I mention this
2020:11:09 16:01:04 borkdude 👍
2020:11:09 16:01:15 Jim Newton :+1::skin-tone-2:
2020:11:09 15:52:02 Jim Newton why doe s/describe return lists using and and or rather than clojure.spec.alpha/and and clojure.spec.alpha/or ? that makes programmatic usage more difficult.
2020:11:09 15:53:53 borkdude s/describe isn't intended for programmatic usage. use s/form rather than that
2020:11:09 15:54:39 Jim Newton Excellent. that does the trick. much more program friendly
2020:11:09 16:01:34 alexmiller s/describe is primarily useful for printing shorter specs for humans (used in doc for example)
2020:11:09 16:01:49 Jim Newton makes sense.
2020:11:09 20:48:46 amorokh I just gotta ask, is there a way to create a shorthand alias for a namespace to use in fully-qualified keys without actually creating that namespace (using the ns form)?
2020:11:09 20:51:35 borkdude @amorokh
I think this is a pretty common pattern:
Rumor has it that core team is working on making this easier
2020:11:09 20:53:24 amorokh @borkdude thanks, not sure I want to do that but at least I know about that possibility
2020:11:10 16:35:17 Jim Newton I'm getting an error from spec that I don't understand. Maybe someone can help me.
The following example, which I've based on an example in https://clojure.org/guides/spec, works fine, and valid? returns true.
Execution error (ClassCastException) at (REPL:1).
null
is there a way to all s/valid? with a programmatically generated spec?
2020:11:10 16:37:07 borkdude @jimka.issy An s-expression is not yet a spec. I would assume most people would write a macro for creating specs programmatically
2020:11:10 16:37:15 alexmiller Yes, but you need to understand more about how spec works
2020:11:10 16:37:42 alexmiller valid? works on spec objects
2020:11:10 16:38:01 alexmiller Spec forms need to be evaluated into spec objects first
2020:11:10 16:38:53 alexmiller In the case above, you could do that with eval
2020:11:10 16:39:07 borkdude or generate the form with a macro
2020:11:10 16:39:58 alexmiller Even so, you’re still leaning on eval
2020:11:10 16:40:08 borkdude yep
2020:11:10 16:41:44 borkdude Made some fun macros the other day:
This allows you to write (g/cat int? string?), when you're not interested in the conformed value
2020:11:10 16:43:07 alexmiller Spec 2 has a lot more tools for all this stuff
2020:11:10 16:43:20 borkdude defop right?
2020:11:10 16:43:29 alexmiller That’s one
2020:11:10 16:43:31 borkdude any docs I could read yet?
2020:11:10 16:43:54 alexmiller The wiki pages if you haven’t seen those
2020:11:10 16:44:30 alexmiller But not everything is in there and it’s missing a lot of the internal design
2020:11:10 16:44:39 borkdude I'll just wait, no hurry
2020:11:10 16:44:53 alexmiller There is now an intermediate map data form now that you can work on directly
2020:11:10 16:45:05 alexmiller But will almost certainly change
2020:11:10 16:45:18 borkdude Working with maps is a lot easier than writing macros (since macros beget more macros)
2020:11:10 16:46:34 alexmiller Some day rich or I will do a talk about the internals - there are several interesting aspects of it
2020:11:10 16:47:49 borkdude Looking forward
2020:11:10 16:52:49 Jim Newton macros wont help in this case. the origin of my spec is not sitting in an evaluation position.
2020:11:10 17:04:04 alexmiller thus, eval
2020:11:10 17:04:19 alexmiller evaluating a spec form gives you a spec object
2020:11:11 17:10:37 holyjak What is the state of / plan for Spec 2? Or is there I can check for the answers? 🙏
2020:11:11 17:23:17 seancorfield @holyjak It's still in the design phase -- Rich is rethinking the whole function spec thing, as far as I can tell from comments Alex has made.
2020:11:11 17:23:38 seancorfield So it's still very alpha and subject to (a lot of) change.
2020:11:11 17:24:10 holyjak thank you!
2020:11:11 17:24:31 seancorfield I'm chomping at the bit to use it -- for a while I had a branch of our code at work tracking Spec 2 but it just had too much churn and too many bugs to keep up with it, so I abandoned that branch after several months.
2020:11:11 17:24:57 seancorfield It definitely has some nice usability improvements in it, and the whole schema/`select` thing is much better than keys
2020:11:11 18:20:52 alexmiller sorry, don't have any concrete plan
2020:11:11 18:21:06 alexmiller it continues to get slices of attention is about all I can tell you :)
2020:11:11 18:42:53 borkdude Without encouraging more delay, I must say I kind of envy the patience that Rich has before releasing anything ;)
2020:11:12 20:39:55 Ronny Li Hi, does anyone have advice for how to read the following error message from plumatic/schema? My understanding is that the arity doesn't conform to spec but where did the error occur? (`(demo/data.clj:77:1)` is simply where I defined the function)
Error refreshing environment: Syntax error macroexpanding clojure.core/defn at (demo/data.clj:77:1).
Call to clojure.core/defn did not conform to spec. clojure.lang.ExceptionInfo: Call to clojure.core/defn did not conform to spec. #:clojure.spec.alpha{:problems ({:path [:fn-tail :arity-1 :params], :pred clojure.core/vector?, :val :-, :via [:clojure.core.specs.alpha/defn-args :clojure.core.specs.alpha/params+body :clojure.core.specs.alpha/param-list :clojure.core.specs.alpha/param-list], :in [1]} {:path [:fn-tail :arity-n :bodies], :pred (clojure.core/fn [%] (clojure.core/or (clojure.core/nil? %) (clojure.core/sequential? %))), :val :-, :via [:clojure.core.specs.alpha/defn-args :clojure.core.specs.alpha/params+body :clojure.core.specs.alpha/params+body], :in [1]}), :spec #object[clojure.spec.alpha$regex_spec_impl$reify__2509 0x406684fa "
2020:11:12 20:42:02 borkdude Maybe you could post the function itself
2020:11:12 20:53:50 seancorfield Sounds like a defn that has :- instead of an argument vector. I'm guessing you used defn (from core) with Schema syntax instead of Schema's own variant of defn? @ronny463
2020:11:12 20:54:30 Ronny Li that's exactly what happened, thank you 🙂
2020:11:12 20:54:19 seancorfield (based on :pred clojure.core/vector?, :val :- in the first line of the exception)
2020:11:13 16:15:20 borkdude What is the reason spec chose a custom deftype LazyVar over the built-in delay in its implementation?
2020:11:13 16:17:08 alexmiller for which?
2020:11:13 16:17:32 borkdude I'm trying to come up with something that would work in Clojure for this case:
such that you don't have to write @input but just input so the first time it's derefed by Clojure it starts its realization. Just a naked line-seq doesn't work, since that starts reading once it's def-ed.
2020:11:13 16:17:41 borkdude @alexmiller in the gen namespace
2020:11:13 16:19:06 borkdude I think in Clojure I would have to write a subclass of clojure.lang.Var to make that work.
2020:11:13 16:19:33 alexmiller sorry, I don't actually see what you're talking about
2020:11:13 16:20:35 borkdude And maybe I should just go with @input and not do any magic... This lead to the question: why is gen/LazyVar not just a delay but a custom data structure.
Ah right.. sorry, the LazyVar is something in CLJS, not CLJ. facepalm
2020:11:13 16:22:38 borkdude I'll ask over there in #clojurescript
2020:11:13 16:23:09 alexmiller ah
2020:11:13 18:39:26 socksy I am confused about the syntax for (or) and (and) in s/keys. My understanding of it was that anything that is passed to or could be correct, and anything that passed into and has to be present. Did I misunderstand? My intuition does not work for the following example:
2020:11:13 18:51:51 socksy ok it looks like when you have the same naked keyword (not sure of the right terminology, keyword without the NS) then it will always take the last one defined. That's a huge bummer, since I wanted to be able to spec something like "In this case, do this, in this other case, do this + another piece of data", and the way that was being specced before was by storing everything as a tuple
2020:11:13 18:52:38 borkdude @socksy can't you use s/or for either case?
2020:11:13 18:52:52 borkdude more verbose
2020:11:13 18:53:12 seancorfield @socksy You'll need to wrap s/keys with s/and and add your rules via a predicate -- or use s/or around s/keys as @borkdude suggests.
2020:11:13 18:53:22 socksy yes I think it might be possible. I'll go hit my head on it
2020:11:13 18:53:36 socksy i am thinking my own predicate will probably be the most readable but let's see
2020:11:13 18:54:11 borkdude maybe spec2 has a better answer to this... although I'm not sure if s/select can solve this case
2020:11:13 18:54:45 seancorfield Another option would be a multi-spec I think?
2020:11:13 18:59:05 borkdude Now I wonder how you would use this in spec2. I think you would define ::baz as the thing with all possible keys and then use s/select for either case?
2020:11:14 14:55:12 dvingo I wanted the required and optional keys to be reused in different parts of the app without needing to maintain two lists of these keys
2020:11:14 15:29:21 alexmiller at some point you have to ask yourself - is this easier than just saying the keys twice?
2020:11:14 20:04:37 Rob Hanlon I think I have my answer here—I’ll just repeat the keys. Thank you 🙏
